Text recognition method and system based on hamiltonian shape conservation network, medium and device

Abstract

The application provides a character recognition method and system based on a Hamiltonian shape conservation network, a medium and equipment, belonging to the technical field of character recognition based on deep learning. The system comprises an input module, a feature extraction module, an initial feature generation module, an energy conversion network ETN, a Hamiltonian dynamics evolution network HDNN and a Transformer decoder. The input module acquires an optical character recognition (OCR) image and performs preprocessing; the feature extraction module performs feature extraction on the preprocessed OCR image to obtain a spatial feature map F; the initial feature generation module performs serialization processing and semantic conversion on the spatial feature map F based on Mamba technology to obtain initial generalized coordinates and initial generalized momentum of a Hamiltonian system; the Hamiltonian dynamics evolution network HDNN performs Hamiltonian dynamics evolution according to and according to the Hamiltonian energy conservation, and outputs an evolution result; and the Transformer decoder obtains a character recognition result through decoding, thereby improving the character recognition accuracy in a complex writing scenario.

Description

Technical Field

[0001] This application relates to the field of character recognition technology based on deep learning, and in particular to a character recognition method, system, medium and device based on Hamiltonian semantic conservation network. Background Technology

[0002] Traditional optical character recognition (OCR) models (such as CNN-Transformer, CNN-CTC, etc.) typically treat character recognition tasks as a one-way mapping process from image to text. That is, the model focuses more on extracting visual features from the image and directly generating the corresponding character sequence, while paying insufficient attention to the inherent coupling relationship between the stroke morphology features and semantic information in the text image, which can easily lead to a decrease in recognition accuracy under complex writing systems.

[0003] Especially in agglutinative writing systems such as Arabic, the coupling relationship between morphology and semantics is shown in Table 1: Table 1. Coupling of stroke structure and semantics in complex writing systems

[0004] As can be seen from Table 1, a small change in the stroke structure (such as adding a dot, lengthening, or bending a hook) can lead to a change in semantics (pronunciation / meaning). This strong mapping relationship from form to meaning is called a coupling relationship.

[0005] Therefore, there is a need to provide an improved technical solution that utilizes the inherent coupling relationship between stroke morphology features and semantic information to ensure that the model retains both the morphological information of characters and the semantic information during the recognition process. Summary of the Invention

[0006] The purpose of this application is to provide a character recognition method, system, medium, and device based on Hamiltonian semantic conservation networks to solve or alleviate the problems existing in the prior art.

[0007] To achieve the above objectives, this application provides the following technical solution: This application provides a character recognition system based on Hamiltonian semantic conservation network, including: an input module, a feature extraction module, an initial feature generation module, a Hamiltonian dynamics evolution network (HDNN), and a Transformer decoder; The input module is used to acquire OCR images for text recognition and perform preprocessing; The feature extraction module is used to extract features from the preprocessed OCR image to obtain a spatial feature map F; The initial feature generation module is used to serialize the spatial feature map F based on Mamba technology to obtain morphological features and perform semantic transformation to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; Used to characterize the initial morphological features of characters. Used to characterize the initial semantic features of text; The Hamiltonian dynamics evolution network (HDNN) is used to determine the initial generalized coordinates. and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution results are output. ; The Transformer decoder is used to determine the evolution result. Decode the text to obtain the text recognition results.

[0008] Preferably, the feature extraction module is a MobileNetV3 network.

[0009] Preferably, the initial feature generation module includes a serialization submodule and a momentum generation module; The serialization submodule is used to generate a local input vector by performing image patching serialization on the spatial feature map F, followed by feature flattening and linear mapping. and the local input vector The input is fed into the Mamba model for structured state-space SSM processing, and the output vector obtained from the SSM processing is used as the initial generalized coordinates of the Hamiltonian system. ; The momentum generation module is used to process the initial generalized coordinates. By performing latent semantic mapping through learnable linear transformations and injecting positional information, the corresponding initial generalized momentum is obtained. .

[0010] Preferably, the system further includes an Energy Conversion Network (ETN), which is positioned after the initial feature generation module and used to process the initial generalized coordinates. Initial generalized momentum With the local input vector Channel splicing is performed, followed by feature reconstruction using two fully connected layers and a nonlinear activation function to obtain the initial state of the Hamiltonian dynamics. ; Accordingly, the Hamiltonian dynamics evolution network (HDNN) is used to start from the Hamiltonian dynamics initial state. As the final input, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution result is output. .

[0011] Preferably, the expression for the generalized Hamiltonian energy function is as follows: , In the formula, For form potential energy, For semantic kinetic energy, It is a nonlinear interaction energy. These represent generalized coordinates and generalized momentum, respectively.

[0012] Preferably, the Hamiltonian dynamics evolution network (HDNN) consists of two independent fully connected neural network sub-modules, referred to as the first sub-module and the second sub-module, respectively. The first submodule is used to calculate the generalized Hamiltonian energy function pair. The partial derivatives are used to obtain the shape driving force; The second submodule is used to calculate the generalized Hamiltonian energy function pair. The partial derivatives are used to obtain semantic driving force; The Hamiltonian energy is updated jointly by the morphological driving force and the semantic driving force until a preset number of iterations is reached, at which point the evolution result is output. .

[0013] Preferably, during the training phase of the character recognition system based on the Hamiltonian semantic conservation network, the total loss is obtained by weighting the recognition main loss and the energy conservation loss, wherein the recognition main loss is the label smoothing cross-entropy loss, and the expression for the energy conservation loss is as follows: , In the formula, Loss due to energy conservation The total number of characters to be recognized. For Hamiltonian energy at the encoding end, This refers to the Hamiltonian energy at the decoding end.

[0014] Secondly, this embodiment provides a character recognition method based on a Hamiltonian form-meaning conservation network. This method is executed by the character recognition system based on a Hamiltonian form-meaning conservation network provided in any of the above embodiments, and includes the following steps: Acquire text recognition OCR images and perform preprocessing; Feature extraction is performed on the preprocessed OCR image to obtain the spatial feature map F; Based on Mamba technology, the spatial feature map F is serialized and semantically transformed to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; According to the initial generalized coordinates and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to the conservation of Hamiltonian energy. During the evolution process, the energy conversion network (ETN) is used to update the Hamiltonian energy, and the evolution result is output. ; Using a Transformer decoder, based on the evolution results Decode the text to obtain the text recognition results.

[0015] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps of the character recognition method based on Hamiltonian semantic conservation network provided in any of the above embodiments.

[0016] This embodiment provides a computer-readable storage medium storing a computer program / instructions thereon, which, when executed by a processor, implements the steps of the character recognition method based on Hamiltonian semantic conservation network provided in any of the above embodiments.

[0017] The technical solution of this application embodiment has the following beneficial effects: This embodiment provides a character recognition system based on Hamiltonian form-semantic conservation network. Based on the semantic energy conservation theoretical framework and Hamiltonian network modeling idea, this solution describes the balanced transmission relationship of information flow based on the energy conservation principle in Hamiltonian dynamic system. It adopts a bidirectional energy conservation mechanism of form and semantics. By simultaneously constraining the balanced evolution of morphological energy and semantic energy in the recognition process, it preserves the morphological information of characters without losing semantics, thereby improving the recognition accuracy under complex writing systems. Attached Figure Description

[0018] Figure 1 This is a logic diagram of a character recognition system based on Hamiltonian semantic conservation networks according to some embodiments of this application.

[0019] Figure 2 This is a schematic diagram of the structure of a computer device provided in some embodiments of this application. Detailed Implementation

[0020] The embodiments of this application will now be described with reference to the accompanying drawings.

[0021] Example 1 This application provides a character recognition system based on a Hamiltonian semantic conservation network, comprising: an input module, a feature extraction module, an initial feature generation module, an energy conversion network (ETN), a Hamiltonian dynamics evolution network (HDNN), and a Transformer decoder. The input module acquires and preprocesses the character recognition OCR image; the feature extraction module extracts features from the preprocessed OCR image to obtain a spatial feature map F; and the initial feature generation module performs serialization and semantic transformation on the spatial feature map F based on Mamba technology to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; Hamiltonian Dynamics Evolutionary Network (HDNN) is used to determine the initial generalized coordinates. and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to the conservation of Hamiltonian energy. During the evolution process, the Hamiltonian energy is updated using an energy conversion network (ETN), and the evolution result is output. Transformer decoder, used to determine the evolution results Decode the text to obtain the text recognition results.

[0022] It should be noted that in Hamiltonian dynamics, p represents generalized momentum and q represents generalized coordinates. In this embodiment, the morphological features of the text to be recognized are mapped to the generalized coordinates of the Hamiltonian system, and the semantic features are mapped to the generalized momentum of the Hamiltonian system. Therefore, the initial generalized coordinates... and initial generalized momentum As the starting point of the dynamic evolution of the Hamiltonian system, these represent the initial morphological features and initial semantic features of the characters, respectively.

[0023] The input module is responsible for the system's input data, namely, acquiring the OCR image to be recognized (also known as the original document image) based on the original document, and using it as the input image for standardized preprocessing.

[0024] Preprocessing includes: sequentially performing grayscale conversion and pixel value normalization on the original document image, mapping the pixel values ​​to... or The interval is used to eliminate the interference of uneven illumination and accelerate model convergence.

[0025] Subsequently, the image size was adjusted to a fixed value using bilinear interpolation. The input tensor is obtained and represented as: ,in: , These are the height and width of the image, respectively. For batch size, This represents the number of channels.

[0026] After image preprocessing, morphological features are extracted using a feature extraction module. Specifically: Image The input is fed into the backbone of a lightweight convolutional neural network (CNN) to extract stroke morphology features (i.e. stroke structure information).

[0027] For example, in this embodiment, a network structure such as MobileNetV3 is adopted. Shallow texture features are extracted through convolutional layers at the network front end, and then abstracted layer by layer by subsequent inverted residual blocks to finally obtain a high-dimensional spatial feature map containing structural information, denoted as... ,in, and The size is determined by the total number of downsampling steps in the CNN backbone network, that is: , This represents the total number of downsampling steps.

[0028] This is called feature (spatial) resolution, where, and The size can be adjusted based on the stroke structure information. The specific setting standard is to ensure the height of the output feature map. and width It is sufficient to cover more than half of the minimum character width, thereby preventing the loss of stroke structure information due to excessive downsampling.

[0029] To capture the spatial dependencies of characters, this application sets up an initial feature generation module. This module is based on Mamba technology and semantic transformation. After obtaining the spatial feature map F, it generates the two-dimensional feature map. The input to the Mamba module is serialized using the standard procedure to obtain the structural feature vector. Also known as morphological features, and because the output is in sequence form, it is also called structural feature sequence. Since this feature is also used as input to the Hamiltonian system, the output of this module is also called the initial generalized coordinates. and initial generalized momentum ,in, Indicates the current character position, superscript This indicates the initial step of Hamiltonian dynamics evolution.

[0030] Furthermore, the initial feature generation module includes a serialization submodule and a momentum generation module. The serialization submodule is used to serialize the spatial feature map F using image patches, then input it into the Mamba model for structured state-space SSM processing. The output vector obtained from the SSM processing is used as the initial generalized coordinates of the Hamiltonian system. The momentum generation module is used to generate initial generalized coordinates. By performing latent semantic mapping through learnable linear transformations and injecting positional information, the corresponding initial generalized momentum is obtained. .

[0031] For example, the execution steps of the serialization submodule are as follows: (1) Serialization Patch Partition: Set the patch window size to... , feature map Divided in spatial dimension Non-overlapping local patches Among them, sequence length The following constraints must be satisfied: , (2) Feature flattening and linear mapping: each three-dimensional feature block Flattened into a one-dimensional vector, and mapped to a local input vector through a linear layer. : , In the formula, This is the first learnable projection matrix, used to compress high-dimensional image features to an intermediate dimension. ; This indicates a flattening operation, that is, a flattening operation with dimensions of The block is converted to a dimension of ; This is the corresponding bias vector.

[0032] (3) Mamba input embedding projection: Projecting each local input vector Further linear projection to the input embedding vector adapted to the Mamba model : , In the formula, For the second learnable projection matrix, For the corresponding bias vector, is the dimension of the hidden layer in the Mamba model.

[0033] (4) Adding position encoding yields: , In the formula, For the corresponding number The position embedding vector of each sequence position, and The same dimension is used to indicate the order of character blocks in the original image.

[0034] (5) The structured state-space model (SSM) input to Mamba is discretized as follows: , In the formula, This is the hidden state vector. The StateExpansion Factor is used to remember historical sequence information. It is a discretized state transition matrix that controls the evolution and decay of the hidden state over time steps; To discretize the input matrix and control the current input information The strength of the injected hidden state; The output projection matrix is ​​used to map the hidden states back to the feature space; For residual connection parameters (Direct feedthrough); This is the output vector of the SSM unit. The execution details of the above Mamba technology can be found in the existing Mamba technology architecture, and will not be elaborated upon here.

[0035] Output of the Mamba model As structural feature vectors .

[0036] 3. Perform semantic feature initialization (momentum construction).

[0037] To construct the bivariate phase space required for Hamiltonian dynamics, it is necessary to consider each morphological feature. Initialize a corresponding conjugate semantic momentum .

[0038] Since a global contextual semantic flow has not yet been formed in the initial stage of recognition, this embodiment adopts a strategy that combines local mapping and explicit enhancement to infer the initial semantic latent features from the morphological feature space.

[0039] As mentioned earlier, the final structural features of Mamba output As the initial generalized coordinates of the system, denoted as The corresponding initial generalized momentum The construction process includes the following two sub-steps: Step 1: Latent semantic mapping.

[0040] By mapping structural features to the semantic dimension through a learnable linear transformation, the fundamental momentum vector is obtained. : , In the formula, Initialize the matrix for semantics. This corresponds to the bias. This step maps the structural information in the visual space to an initial local guess in the semantic space.

[0041] Step 2: Location information injection.

[0042] Despite morphological features While some sequence information is already implicitly included, in order to explicitly reinforce the absolute word order and relative positional relationships of characters in semantic momentum (which is crucial for language models), positional encoding is injected again in this embodiment. : , In the formula, , which represents an embedding vector containing absolute position index and relative order information.

[0043] Thus, the initial Hamiltonian state set of the system has been constructed, namely: , Morphological characteristics With semantic features The initial input state variables constitute the Hamiltonian dynamical network (HDNN), and the two will undergo nonlinear coupling evolution and contextual interaction in subsequent steps through the Verlet integrator.

[0044] The system provided in this embodiment also includes an Energy Conversion Network (ETN), which is set after the initial feature generation module and used to process the initial generalized coordinates. Initial generalized momentum With the local input vector Channel splicing is performed, followed by feature reconstruction using two fully connected layers and a nonlinear activation function to obtain the initial state of the Hamiltonian dynamics. Accordingly, the Hamiltonian dynamics evolution network (HDNN) is used to initialize the Hamiltonian dynamics network with the initial state. As the final input, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution result is output. .

[0045] Initial generalized coordinates Initial generalized momentum Local input vector The data is fed into the Energy Transformation Network (ETN). The ETN is specifically constructed as a residual MLP structure, which aims to further nonlinearly map the linearly initialized semantic features, projecting them onto a high-dimensional feature space that is adapted to the subsequent Hamiltonian dynamics evolution.

[0046] To compensate for high-frequency visual details that may be lost during deep feature extraction, this embodiment employs a multi-scale feature fusion strategy to construct a joint input vector. The input to ETN includes the following: (1) High-dimensional semantic state The aforementioned initial semantic momentum and morphological generalized coordinates represent high-level abstract features.

[0047] (2) Local visual features That is, the local input vector that has not been abstracted by the Mamba layer in the aforementioned steps. This feature is introduced to supplement the local stroke and texture information that may be weakened during the Mamba abstraction process, thereby improving the ability to preserve detailed structure in subsequent feature evolution stages.

[0048] The specific calculation process for ETN is as follows: First, the above features are concatenated along the channel dimension to obtain a joint input vector. : , Subsequently, feature reconstruction is performed using two fully connected layers and a nonlinear activation function.

[0049] Let the weight of the first layer be... The weight of the second layer is To adapt to gradient propagation in deep networks, the activation function... GELU (Gaussian Error Linear Element) is selected, and its smooth nonlinear characteristics near the origin are used to introduce a stochastic regularization effect. The calculation formula is as follows: , In the formula, The result of feature reconstruction, i.e., the energy correction term, , This is the corresponding bias.

[0050] It is important to note that, in order to achieve dimension alignment, the weight of the last layer needs to be set. The output dimension is strictly equal to and The sum of dimensions (i.e.) ).

[0051] Finally, a residual connection mechanism is used to superimpose the calculated energy correction terms back onto the initial state to obtain the projected initial dynamic state. : , Here, and This is the formal input state of the Hamiltonian Dynamics Network (HDNN).

[0052] Next, using the Hamiltonian dynamics initial state As the final input, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution result is output. .

[0053] 4. Energy evolution of the Hamiltonian system.

[0054] In this embodiment, the Hamiltonian Dynamics Evolution Network (HDNN) includes two independent fully connected neural network sub-modules, referred to as the first sub-module (sub-module). ) and the second submodule (submodule) The first submodule is used to calculate the generalized Hamiltonian energy function. The partial derivatives are used to obtain the morphological driving force; the second submodule is used to calculate the generalized Hamiltonian energy function. The partial derivatives are used to obtain the semantic driving force; the morphological driving force and the semantic driving force jointly drive the update of the Hamiltonian energy until the preset number of iterations (such as L iterations) is reached, and the evolution result is output. .

[0055] Initial state of Hamiltonian dynamics In the state space, this application no longer uses a simple quadratic form matrix to constrain the energy, but instead constructs a generalized Hamiltonian energy function based on neural network parameterization. The total energy of the system (i.e., the generalized Hamiltonian energy function) is formally defined as: , in, Form potential energy reflects the complexity of visual structure; Semantic dynamics reflect the underlying trends of the linguistic context; It is a nonlinear interaction energy, implicitly modeled by the coupling parameters in the network, used to capture the complex nonlinear mapping relationship between form and meaning, and realize the information interaction between form and semantics.

[0056] During the energy evolution phase, this embodiment constructs a system containing... Hamiltonian dynamics network (HDNN) with layer-by-layer iterative updates.

[0057] To clearly distinguish between sequence time and network depth, subscripts are defined. ( () represents the time step of the character sequence, with the superscript indicating the time step. ( ) represents the iterative hierarchy of the dynamic evolution. The update of the characteristic states follows the second-order symplectic geometric Verlet integral.

[0058] During the feature update process of HDNN, the system does not directly calculate the above. , , Instead of calculating the scalar values ​​of the three energies, we calculate the energy function. Regarding state variables The partial derivatives (i.e., gradient field / conservative force field) drive the state evolution.

[0059] Among them, for The partial derivatives of the submodule Fitting, for The partial derivatives of the submodule Fitting, submodule and submodules They are two independent fully connected neural networks, each consisting of two fully connected neural network layers and a non-linear activation function.

[0060] Specifically, submodule yes The function is denoted as: This function is used for fitting. This refers to the form-driving force generated by both form potential energy and interaction energy. Due to the energy function... Only in China and Include Item, therefore, submodule The output automatically corresponds in a physical sense to these two energy parameters. The sum of gradients.

[0061] submodule yes The function is denoted as: This function is used for fitting. That is, the semantic driving force generated by the combined semantic kinetic energy and interaction energy, due to the energy function Only in China and Include Item, therefore, submodule The output automatically corresponds in a physical sense to these two energy parameters. The sum of gradients. By controlling the network's input variables, it is structurally guaranteed that it fits the gradient of a specific energy term, rather than random data.

[0062] The update of the characteristic state follows a second-order symplectic geometric Verlet integral. In the... Layer to the first During the evolution of the layers, a learnable gradient fitting network (i.e., the aforementioned sub-module) is used. and submodules This achieves a "half-step - full-step - half-step" state update. The specific evolution formula is as follows: (1) Semantic momentum half-step pre-inference: , (2) Morphological generalized coordinate full-step reconstruction: , (3) Semantic momentum half-step fine-tuning: , In the formula, Step Size is the evolution step size factor used to control the magnitude of feature modification and the rate of dynamic evolution in a single-layer network.

[0063] go through After alternating layer evolution, the system output completes the feature state of deep semantic coupling. .

[0064] After evolution is complete, morphological and semantic features are concatenated along the channel dimension, and the input feature sequence for the decoder is generated through linear projection. : , In the formula, The projection matrix maps the feature dimensions to the hidden layer dimensions that match the Transformer decoder.

[0065] 6. Decoding stage using Transformer structure.

[0066] The decoding stage employs a standard Transformer decoder architecture. The decoder receives the feature sequence. As a context encoding (i.e., key and value), and using an autoregressive mechanism to generate character sequences, the text recognition result (i.e., the output recognized character sequence) is obtained.

[0067] To rigorously map the physical conservation properties generated by the aforementioned Hamiltonian dynamics evolution to the character generation process of the decoder, and to monitor and constrain the physical state of the system in real time, in this embodiment, during the training phase of the character recognition system based on the Hamiltonian semantic conservation network, the total loss is obtained by weighting the recognition main loss and the energy conservation loss. The recognition main loss is the label smoothing cross-entropy loss, and the expression for the energy conservation loss is as follows: , In the formula, Loss due to energy conservation The total number of characters to be recognized. For Hamiltonian energy at the encoding end, This refers to the Hamiltonian energy at the decoding end.

[0068] This energy conservation loss can measure and suppress energy drift that may occur during the actual inference process. In other words, by adding an energy consistency constraint (energy conservation loss) to the training objective and minimizing this energy conservation loss, it is possible to ensure that the Hamiltonian energy at the encoding end is consistent with the Hamiltonian energy at the decoding end, thus satisfying the lossless and accurate conservation principle of the cross-modal information transmission process of Hamiltonian dynamics.

[0069] In the above expression for energy conservation loss, and All values ​​were calculated using the EnergyNet energy evaluation network. Specifically, EnergyNet is deployed in parallel on the side bypass of the Transformer decoder. This energy evaluation network does not interfere with the character generation process of the Transformer decoder backbone network. Instead, it acts as an observer to receive intermediate variables from the decoder and uses a weight-sharing strategy to achieve energy alignment between the encoder and decoder.

[0070] In terms of network composition, EnergyNet employs a multilayer perceptron (MLP) structure, including linear projection layers and nonlinear activation functions, with an output dimension of 1, used for the energy function. By performing fitting and end-to-end learning, the high-dimensional features of the input are mapped to scalar energy values, which include morphological potential. Semantic kinetics and nonlinear interaction energy The Hamiltonian total energy scalar.

[0071] Among them, Hamiltonian energy at the encoding end Using the current context vector As input to EnergyNet, the current context vector is calculated through nonlinear feature mapping and scalar projection. This is during the generation of the Transformer decoder. When dealing with a number of characters, a cross-attention mechanism is used to process the feature sequence. Perform weighted aggregation to generate the current context vector. The specific generation process can be referred to the existing Transformer technology, and will not be elaborated here. The specific calculation process includes the following two stages: Phase 1: Nonlinear Feature Reconstruction. Utilizing the first-layer weight matrix of EnergyNet. context vector Project onto the hidden layer space and apply the GELU activation function. Activation is performed to capture the complex nonlinear coupling relationship between morphology and semantics, and to fit the interaction energy term in the Hamiltonian system.

[0072] Phase 2: Energy scalar output. This utilizes the second-layer weight matrix of EnergyNet. The reconstructed features are mapped to a one-dimensional real number field, thereby quantifying the generalized Hamiltonian total energy value of the current state.

[0073] The above will use the current context vector Input EnergyNet, the expression is as follows: , In the formula, For the activation function GELU, , This is the weight matrix. , This is the corresponding bias.

[0074] Hamiltonian energy at the decoding end This refers to the top-level hidden state vector of the Transformer decoder before the output Softmax layer. The energy is calculated as input to EnergyNet. To ensure consistency in energy metric standards, a weight-sharing strategy is employed. The input is exactly the same as the encoder (i.e., sharing the same set of weight parameters). In EnergyNet, the expression for calculating its reconstruction energy (i.e., the energy at the decoding end) is: , With EnergyNet, the system can achieve the same time step Below, energy estimates derived from the encoder's visual flow are obtained respectively. Energy estimation derived from the decoder language stream According to the conservation principle of Hamiltonian dynamics, if the cross-modal information transmission process is lossless and accurate, the two energy scalars mentioned above should theoretically maintain numerical consistency.

[0075] Label smoothing cross-entropy loss is denoted as The Chinese version is also written as: Label Smoothing-Cross-Entropy Loss Function. The specific calculation formula can be found in existing technology and will not be elaborated here.

[0076] Therefore, the total loss function The formula is as follows: , in, This is a balancing coefficient used to adjust the weight of physical constraints during training. Preferably, its value range is set to... The specific values ​​are dynamically adjusted based on the convergence of the validation set. By minimizing this loss function, the model improves character recognition accuracy while forcing the conservation of semantic energy flow in the feature space during cross-modal mapping, thereby enhancing the model's robustness in ambiguous and degenerate text scenarios.

[0077] In summary, this embodiment provides a character recognition system based on Hamiltonian form-semantic conservation networks. This scheme, based on the semantic energy conservation theoretical framework and Hamiltonian network modeling ideas, describes the balanced transmission relationship of information flow based on the energy conservation principle in Hamiltonian dynamic systems. It adopts a bidirectional energy conservation mechanism of form and semantics, and by simultaneously constraining the balanced evolution of morphological energy and semantic energy in the recognition process, it preserves the morphological information of characters without losing semantics, thereby improving the recognition accuracy under complex writing systems.

[0078] The system provided in this embodiment not only maintains semantic consistency in the semantic space, but also preserves the stroke morphological features of characters in the visual structural space, and through interactive capabilities... By establishing a reversible mapping relationship between the two, semantic information can inversely constrain morphological feature reconstruction, while morphological information can also positively enhance semantic reasoning, thus achieving semantic-driven morphological repair and morphological feedback-based semantic enhancement. Unlike the unidirectional semantic conservation in traditional Hamiltonian systems, the scheme provided in this embodiment is a bidirectionally adjustable energy conservation system that dynamically maintains the information flux conservation of visual and linguistic features during the recognition process, thereby significantly improving the model's recognition accuracy and robustness in complex handwritten characters, fuzzy characters, and multilingual mixed scenarios.

[0079] In the technical solution of this embodiment, the morphological-semantic bidirectional energy conservation mechanism is established between the morphological space and the semantic space, so that the energy distribution of visual features and semantic representations is kept constant, which can compensate for information loss caused by stroke blurring, font deformation, etc.

[0080] Based on the same inventive concept, this embodiment also provides a character recognition method based on a Hamiltonian semantic conservation network. This method is executed by the character recognition system based on a Hamiltonian semantic conservation network provided in any of the above embodiments, and includes: Acquire text recognition OCR images and perform preprocessing; Feature extraction is performed on the preprocessed OCR image to obtain the spatial feature map F; Based on Mamba technology, the spatial feature map F is serialized and semantically transformed to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; According to the initial generalized coordinates and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to the conservation of Hamiltonian energy. During the evolution process, the energy conversion network (ETN) is used to update the Hamiltonian energy, and the evolution result is output. ; Using a Transformer decoder, based on the evolution results Decode the text to obtain the text recognition results.

[0081] The character recognition method based on Hamiltonian semantic conservation network provided in this embodiment can achieve the same technical purpose and effect as the character recognition system based on Hamiltonian semantic conservation network provided in any of the above embodiments, and will not be described in detail here.

[0082] The embodiments of this application can be applied to Figure 2 Among the electronic devices shown, the electronic devices may be, but are not limited to, mobile terminals such as mobile phones, tablets, handheld computers, and personal digital assistants (PDAs), smart home devices such as smart TVs and smart cameras, wearable devices such as smart bracelets, smartwatches, and smart glasses, or other computer devices such as desktop, laptop, notebook, ultra-mobile personal computer (UMPC), netbook, and smart screen.

[0083] like Figure 2 As shown, the electronic device 200 may include one or more of the following components: a processor 201, a memory 203, a communication interface 202, and a communication bus 204. The memory 203 can be connected to the processor 201 via the bus 204. The bus can transfer data between the processor 201 and the memory 203. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0084] Processor 201 may include one or more processing cores. Processor 201 can connect to various parts within the electronic device 200 using various interfaces and lines. It performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 203, and by calling data stored in memory 203. For example, processor 201 may include an application processor (AP), a modem processor, a CPU, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a field-programmable gate array (FPGA), a programmable logic array (PLA), and / or a neural network processing unit (NPU). The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed; the NPU implements artificial intelligence (AI) functions; and the modem handles wireless communication. Different processing units can be independent devices or integrated into one or more processors. For example, the multiple processing units shown above are all integrated into a single SoC, or the AP is a separate semiconductor chip, while other processing units are integrated into a single SoC. This application does not limit this to any particular type.

[0085] The memory 203 may include random access memory (RAM), read-only memory (ROM), or non-transitory computer-readable storage medium. The memory 203 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 203 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system or instructions for at least one function, such as a character recognition method based on a Hamiltonian semantic conservation network. The data storage area may store data created based on the use of the electronic device 200, such as acquired text OCR images, network structure definition files, hyperparameters, etc.

[0086] In addition, those skilled in the art will understand that the structure of the electronic device 200 shown in the above figures does not constitute a limitation on the electronic device 200. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device 200 may also include components such as a microphone, speaker, radio frequency circuit, sensor, audio circuit, power supply, and Bluetooth module, which will not be described in detail here.

[0087] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A character recognition system based on Hamiltonian semantic conservation networks, characterized in that, include: Input module, feature extraction module, initial feature generation module, Hamiltonian dynamics evolution network (HDNN) and Transformer decoder; The input module is used to acquire OCR images for text recognition and perform preprocessing; The feature extraction module is used to extract features from the preprocessed OCR image to obtain a spatial feature map F; The initial feature generation module is used to perform serialization and semantic transformation on the spatial feature map F based on Mamba technology to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; Used to characterize the initial morphological features of characters. Used to characterize the initial semantic features of text; The Hamiltonian dynamics evolution network (HDNN) is used to determine the initial generalized coordinates. and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution results are output. ; The Transformer decoder is used to determine the evolution result. Decode the text to obtain the text recognition results.

2. The system according to claim 1, characterized in that, The feature extraction module is specifically the MobileNetV3 network.

3. The system according to claim 1, characterized in that, The initial feature generation module includes a serialization submodule and a momentum generation module; The serialization submodule is used to generate a local input vector by performing image patching serialization on the spatial feature map F, followed by feature flattening and linear mapping. and the local input vector The input is fed into the Mamba model for structured state-space SSM processing, and the output vector obtained from the SSM processing is used as the initial generalized coordinates of the Hamiltonian system. ; The momentum generation module is used to process the initial generalized coordinates. By performing latent semantic mapping through learnable linear transformations and injecting positional information, the corresponding initial generalized momentum is obtained. .

4. The system according to claim 3, characterized in that, The system also includes an Energy Conversion Network (ETN), which is configured after the initial feature generation module and used to process the initial generalized coordinates. Initial generalized momentum With the local input vector Channel splicing is performed, followed by feature reconstruction using two fully connected layers and a nonlinear activation function to obtain the initial state of the Hamiltonian dynamics. ; Accordingly, the Hamiltonian dynamics evolution network (HDNN) is used to start from the Hamiltonian dynamics initial state. As the final input, Hamiltonian dynamics evolution is performed according to Hamiltonian energy conservation, and the evolution result is output. .

5. The system according to claim 1, characterized in that, The expression for the generalized Hamiltonian energy function is as follows: ， In the formula, As for potential energy, For semantic kinetic energy, It is a nonlinear interaction energy. These represent generalized coordinates and generalized momentum, respectively.

6. The system according to claim 5, characterized in that, The Hamiltonian dynamics evolution network (HDNN) consists of two independent fully connected neural network sub-modules, referred to as the first sub-module and the second sub-module, respectively. The first submodule is used to calculate the generalized Hamiltonian energy function pair. The partial derivatives are used to obtain the shape driving force; The second submodule is used to calculate the generalized Hamiltonian energy function pair. The partial derivatives are used to obtain semantic driving force; The Hamiltonian energy is updated jointly by the morphological driving force and the semantic driving force until a preset number of iterations is reached, at which point the evolution result is output. .

7. The system according to claim 1, characterized in that, During the training phase of the character recognition system based on the Hamiltonian semantic conservation network, the total loss is obtained by weighting the recognition main loss and the energy conservation loss. The recognition main loss is the label smoothing cross-entropy loss, and the energy conservation loss is expressed as follows: ， In the formula, Loss due to energy conservation The total number of characters to be recognized. For Hamiltonian energy at the encoding end, This refers to the Hamiltonian energy at the decoding end.

8. A character recognition method based on a Hamiltonian form-meaning conservation network, wherein the method is executed by the character recognition system based on a Hamiltonian form-meaning conservation network as described in any one of claims 1 to 7, comprising the following steps: Acquire text recognition OCR images and perform preprocessing; Feature extraction is performed on the preprocessed OCR image to obtain the spatial feature map F; Based on Mamba technology, the spatial feature map F is serialized and semantically transformed to obtain the initial generalized coordinates of the Hamiltonian system. and initial generalized momentum ;in, Indicates the current character position; According to the initial generalized coordinates and initial generalized momentum Based on the generalized Hamiltonian energy function, Hamiltonian dynamics evolution is performed according to the conservation of Hamiltonian energy. During the evolution process, the energy conversion network (ETN) is used to update the Hamiltonian energy, and the evolution result is output. ; Using a Transformer decoder, based on the evolution results Decode the text to obtain the text recognition results.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 8.

10. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method of claim 8.

Images