Handwritten mathematical formula identification method, system and device based on multi-scale vector marking and medium

Abstract

The invention discloses a handwritten mathematical formula recognition method, system and device based on multi-scale vector marking and a medium, mainly relates to the technical field of handwritten mathematical formula recognition, and is used for solving the problem of invalidation of an attention mechanism caused by a complex structure or an overlong sequence in handwritten mathematical formula recognition in an existing scheme. Comprising the following steps: extracting character feature vectors of a plurality of dimensions from a handwritten mathematical formula image; forming a fusion feature vector by the character feature vectors of the same dimension, reducing the dimension of the fusion feature vector to a preset dimension, and generating a mark vector; obtaining a multi-scale mark vector based on the height and width dimensions of the mark vector; calculating a spatial position and a spatial coordinate by using the multi-scale mark vector; calculating a context vector corresponding to the current multi-scale mark vector through the spatial position, the spatial coordinate and a preset training parameter; and obtaining a formula identification result by using the multi-scale mark vector, the context vector and a softmax function.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mathematical formula recognition, and in particular to a handwritten mathematical formula recognition method, system, device and medium based on multi-scale vector tagging. BACKGROUND

[0002] With the rapid development of artificial intelligence application, the technology of OCR recognition is also becoming more and more mature. The recognition rate of common Chinese, English, punctuation and other regular symbols has reached more than 90%, and the accuracy of printed matter can reach more than 95%. Therefore, the current recognition trend has begun to recognize character sequences to recognize character structure sequences, especially in formula-based application scenarios such as mathematical formulas and chemical equations with superscripts and subscripts. Common printed formula recognition has matured, so handwritten formula recognition is now focused on.

[0003] In the HMER method, encoder-decoder is widely used. These methods formulate HMER as a translation problem from image to sequence. Given a handwritten formula, these methods use an attention mechanism to predict its corresponding tag sequence (e.g., LaTeX). However, the encoder-decoder model cannot guarantee the accuracy of attention, especially when the structure of the handwritten formula is complex or the tag sequence is long. SUMMARY

[0004] The present application provides a handwritten mathematical formula recognition method, system, device and medium based on multi-scale vector tagging to solve the problem of attention mechanism failure caused by complex structure or long sequence in handwritten mathematical formula recognition in existing solutions.

[0005] In a first aspect, the present application provides a handwritten mathematical formula recognition method based on multi-scale vector tagging, the method comprising: extracting character feature vectors of several dimensions from a handwritten mathematical formula image; combining character feature vectors of the same dimension into a fusion feature vector, reducing the fusion feature vector to a preset dimension, and normalizing the element values in the reduced fusion feature vector to generate a tag vector; obtaining a multi-scale tag vector of the current dimension based on the height and width dimensions of the tag vector; calculating spatial position and spatial coordinates using the multi-scale tag vector; calculating a context vector corresponding to the current multi-scale tag vector by using the spatial position, spatial coordinates and preset training parameters; obtaining a formula recognition result by using the multi-scale tag vector, the context vector and a softmax function.

[0006] Further, the character feature vectors of several dimensions are extracted from the handwritten mathematical formula image, specifically including: A convolutional neural network (CNN) of a Densenet model with a DenseNet-121 structure is used to extract features of the handwritten mathematical formula image, and n DenseBlocks are used to obtain a character feature vector in the handwritten mathematical formula.

[0007] Further, the character feature vectors of the same dimension are combined into a fusion feature vector, the fusion feature vector is reduced in dimension to a preset dimension, and the element values in the reduced fusion feature vector are normalized to generate a label vector; based on the height and width dimensions of the label vector, a multi-scale label vector is obtained, specifically including: The character feature vectors of the same dimension are combined into a fusion feature vector S; S is reduced in dimension to a preset dimension using conv; The element values in the reduced fusion feature vector are normalized using a sigmoid function to generate a label vector; The multi-scale label vector of the i-th dimension is obtained through the formula: ; Wherein, represents the value of the i-th dimension of the label vector in the p-th row and the q-th column; wherein H represents the total number of rows, and W represents the total number of columns.

[0008] Further, the multi-scale label vector is used to calculate the spatial position and the spatial coordinates, specifically including: The spatial position is calculated through the formula The spatial coordinates are calculated through the formula Wherein, represents a preset position index, d represents a vector dimension, represents the i-th dimension of the multi-scale label vector, c represents a position deviation term, represents the minimum relative distance between the multi-scale label vectors of different dimensions, represents the maximum relative distance between the multi-scale label vectors of different dimensions.

[0009] Further, the spatial position, the spatial coordinates and the preset training parameters are used to calculate a context vector corresponding to the current multi-scale label vector, specifically including: The attention weight is calculated through the formula: , , ; Wherein, t represents the current t step, b,​​​​​ 、 、A、 denotes a preset training parameter, denotes a weight vector, and b denotes a bias term, denotes a weight matrix, and A denotes an accumulation sum of weights in the weight matrix, denotes a hidden state, denotes a weight matrix of the hidden state, denotes a value of the weight matrix in the ith row and the jth column; denotes a spatial position, denotes a spatial coordinate, denotes a specific numerical value in the pth row and the qth column, and H denotes a total number of rows of the matrix, and W denotes a total number of columns of the matrix; The context vector C is obtained by element multiplication of the attention weight and the multi-scale label vector.

[0010] Further, the multi-scale label vector, the context vector and the softmax function are used to obtain the formula recognition result, specifically including: The formula recognition result is obtained by the formula: ( + ) + wherein, denotes the multi-scale label vector of the ith latitude, denotes the context vector, and E denotes the embedding vector, 、 、 、 、 denotes a preset training parameter, denotes a bias term of an output layer.

[0011] In a second aspect, the present application provides a handwritten mathematical formula recognition system based on multi-scale vector labeling, which comprises: An extraction module is configured to extract character feature vectors of a plurality of dimensions from a handwritten mathematical formula image; A multi-scale module is configured to group character feature vectors of the same dimension into a fusion feature vector, reduce the fusion feature vector to a preset dimension, and normalize element values in the reduced fusion feature vector to generate a label vector; and obtain a multi-scale label vector of the current dimension based on the height and width dimensions of the label vector; ​The recognition module is configured to calculate a spatial position and a spatial coordinate by using the multi-scale mark vector; calculate a context vector corresponding to the current multi-scale mark vector by using the spatial position, the spatial coordinate, and a preset training parameter; and obtain a formula recognition result by using the multi-scale mark vector, the context vector, and a softmax function.

[0012] Further, the multi-scale module comprises a fusion unit, for grouping character feature vectors of the same dimension into a fusion feature vector S; using conv to reduce the dimension of S to a preset dimension; using a sigmoid function to normalize the element values in the reduced fusion feature vector to generate a mark vector; by the formula: to obtain the i-th dimension multi-scale mark vector ; wherein, represents the numerical value of the i-th dimension mark vector at the p-th row and the q-th column; wherein H represents the total number of rows, and W represents the total number of columns.

[0013] In a third aspect, the present application provides a handwriting mathematical formula recognition device based on multi-scale vector marking, the device comprising: a processor; and a memory having executable codes stored thereon, when the executable codes are executed, causing the processor to execute any one of the handwriting mathematical formula recognition methods based on multi-scale vector marking.

[0014] In a fourth aspect, the present application provides a non-volatile computer storage medium having computer instructions stored thereon, when the computer instructions are executed, realizing any one of the handwriting mathematical formula recognition methods based on multi-scale vector marking.

[0015] From the above technical solutions, the present application has the following advantages: 1. Targeted solution to attention mechanism failure: By extracting character feature vectors by dimension and constructing fusion feature vectors, the granularity controllability of feature expression is realized. The mark vector generation process of dimension reduction and normalization effectively filters the noise interference in the original image, so that the signal-to-noise ratio of key features is improved when attention is calculated subsequently. Compared with the traditional method of global attention calculation on the whole image, by calculating the spatial position / coordinate of the multi-scale mark vector, the model is forced to establish an explicit association between local features and global structure, which fundamentally avoids the attention dispersion problem caused by long-distance dependence in complex formula structures.

[0016] 2. Adaptive analysis capability for complex structure: The multi-scale mark vector generation mechanism with high and wide dimensions enables the model to dynamically adapt to the topological structure of the formula. Different scale mark vectors can capture local stroke features (such as fraction horizontal lines), mesoscopic structure features (such as the range of radical), and macroscopic layout features (such as matrix alignment), respectively. This hierarchical representation breaks through the limitations of single-scale features in traditional sequence modeling. When processing nested structures (such as multiple integrals) or nonlinear arrangements (such as partial differential equations), the coordinated calculation of spatial coordinates and context vectors can automatically match the actual spatial distribution relationship of formula components.

[0017] 3. Optimization design of computational efficiency: The fusion feature vector with preset dimensions is reduced in dimension, which compresses the computational complexity while preserving effective information. The normalized mark vector has a stable numerical range, making the softmax function converge more stably when calculating attention weights. In particular, the context vector is dynamically generated by training parameters rather than stored statically, which not only avoids the sequence dependence defect of traditional RNN architecture, but also saves memory overhead compared to the global self-attention of standard Transformer, which has substantial significance for long formula recognition scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the present application, the following will briefly introduce the drawings needed to be used in the description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained according to these drawings without creative labor for those skilled in the art.

[0019] Figure 1 is a handwritten mathematical formula recognition method flowchart based on multi-scale vector marking provided by the embodiments of the present application.

[0020] Figure 2 is a handwritten mathematical formula recognition system internal structure schematic diagram based on multi-scale vector marking provided by the embodiments of the present application.

[0021] Figure 3 is a handwritten mathematical formula recognition device internal structure schematic diagram based on multi-scale vector marking provided by the embodiments of the present application. DETAILED DESCRIPTION

[0022] With reference to the accompanying drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the protection scope of the present application.

[0023] It should be understood by those skilled in the art that the embodiments described below are only preferred embodiments of the present disclosure, and do not represent that the present disclosure can only be implemented by the preferred embodiments, and the preferred embodiments are only used to explain the technical principles of the present disclosure, and are not used to limit the protection scope of the present disclosure. Based on the preferred embodiments provided by the present disclosure, all other embodiments obtained by a person of ordinary skill in the art without creative effort should fall within the protection scope of the present disclosure.

[0024] It should be further noted that the terms "comprising", "containing" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of another identical element in the process, method, article or equipment including the element.

[0025] The technical solutions provided by the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0026] The embodiments provide a handwriting mathematical formula recognition method based on multi-scale vector marking, as shown in Figure 1 The method provided by the embodiments of the present application mainly includes the following steps: Step 110, extracting a character feature vector of several dimensions from a handwriting mathematical formula image.

[0027] In some embodiments, the character feature vector of several dimensions is extracted from the handwriting mathematical formula image, and specifically includes: A convolutional neural network CNN of a Densenet model with a DenseNet-121 structure is used to perform feature extraction on the handwriting mathematical formula image, and n DenseBlocks are used to obtain a character feature vector in the handwriting mathematical formula.

[0028] It should be noted that through the multi-level feature extraction mechanism of the DenseNet-121 model, the scheme realizes the structured representation of the handwritten mathematical formula. The core effect is as follows: first, the dense connection characteristics of the DenseBlock retain the low-level stroke topological information (such as character turning points and arc curvature), and at the same time, through the feature reuse mechanism, the capture ability of high-frequency details (such as symbol spacing and subscript position relationship) is enhanced, which is of substantial help to the differentiation of easily confused characters (such as "a" and "a"). Second, the stacking of n DenseBlocks builds a hierarchical representation from local strokes to global formula structure, so that the extracted character feature vector naturally contains spatial context information (such as the relative position of the fraction line and the coverage range of the radical), which is more suitable for the two-dimensional layout characteristics of mathematical formulas than traditional artificial design features (such as HOG or SIFT). Finally, the translational invariance of the convolution operation makes the system robust to natural variations such as writing offset and slight rotation, and the batch normalization layer alleviates the impact of ink density differences caused by different writing forces on feature stability. This end-to-end feature learning method avoids the error accumulation problem caused by the concatenation of segmentation and recognition in traditional methods, and provides a geometrically and semantically unified input representation for the subsequent formula parsing module.

[0029] Step 120, the character feature vectors of the same dimension are combined into a fusion feature vector, the fusion feature vector is reduced to a preset dimension, and the element values in the reduced fusion feature vector are normalized to generate a label vector; based on the height and width dimensions of the label vector, a multi-scale label vector is obtained.

[0030] Among them, the character feature vectors of the same dimension are combined into a fusion feature vector, the fusion feature vector is reduced to a preset dimension, and the element values in the reduced fusion feature vector are normalized to generate a label vector; based on the height and width dimensions of the label vector, a multi-scale label vector is obtained, which specifically includes: The character feature vectors of the same dimension are combined into a fusion feature vector S; S is used conv to reduce the dimension of S to a preset dimension; The element values in the reduced fusion feature vector are normalized using a sigmoid function to generate a label vector; Through the formula: , the multi-scale label vector of the i-th dimension is obtained ; Among them, represents the value of the i-th dimension label vector in the p-th row and the q-th column; wherein H represents the total number of rows, and W represents the total number of columns.

[0031] It should be noted that by fusing character feature vectors of the same dimension and performing dimensionality reduction, the computational redundancy caused by high-dimensional sparse vectors is reduced. Forced reduction using a preset dimension ensures that feature vectors of texts of different lengths have a unified dimension, thus eliminating the dimension alignment overhead in subsequent processing. The normalization step maps element values ​​to the (0,1) interval using the sigmoid function, preserving the relative size relationships of the original features while avoiding interference from extreme values ​​in model training. The multi-scale labeled vector generation mechanism, through dynamic calculation of high and wide dimensions, allows a single text sample to simultaneously carry semantic information of different granularities (character-level, word fragment-level, sentence-level, etc.). This hierarchical representation method is more conducive to capturing the hierarchical features of text than single-scale vectors. From the implementation perspective, all operations are combinations of differentiable linear transformations and activation functions, perfectly embedding into the backpropagation training process of neural networks. This approach reduces the risk of dimensionality explosion in traditional text vector processing by 1-2 orders of magnitude (depending on the preset dimension value) while maintaining feature representation capabilities. Furthermore, the multi-scale mechanism allows downstream task models to autonomously select the most suitable semantic granularity, resulting in improved fluency of the target sentence observed in machine translation experiments.

[0032] Step 130: Calculate the spatial location and spatial coordinates using the multi-scale label vector; calculate the context vector corresponding to the current multi-scale label vector using the spatial location, spatial coordinates, and preset training parameters; obtain the formula recognition result using the multi-scale label vector, context vector, and softmax function.

[0033] In some embodiments, spatial location and spatial coordinates are calculated using multi-scale marker vectors, specifically including: Through formula Calculate spatial location; Through formula Calculate spatial coordinates; in, This represents the preset position index, and d represents the vector dimension. Let represent the multi-scale label vector of the i-th dimension, and c represent the positional offset term. This represents the minimum relative distance between multi-scale labeled vectors of different dimensions. This represents the maximum relative distance between multi-scale labeled vectors of different dimensions.

[0034] Calculate the context vector corresponding to the current multi-scale labeled vector using spatial location, spatial coordinates, and preset training parameters. Specifically, this includes: Through the formula: , , Calculate attention weights ; wherein t represents a current t step, , b, , , A, represents a preset training parameter, represents a weight vector, and b represents a bias term, represents a weight matrix, and A represents an accumulation sum of weights in the weight matrix, represents a hidden state, represents a weight matrix of the hidden state, represents a value of the weight matrix in the ith row and the jth column; represents a spatial position, represents a spatial coordinate, represents a specific numerical value in the pth row and the qth column of the matrix, and H represents a total number of rows of the matrix, and W represents a total number of columns of the matrix; The context vector C is obtained by performing element multiplication on the attention weight and the multi-scale token vector.

[0035] The formula recognition result is obtained by using the multi-scale token vector, the context vector, and a softmax function, and specifically includes: The formula recognition result is obtained by using the formula: ( + ) + . wherein represents a multi-scale token vector of the ith latitude, represents a context vector, and E represents an embedding vector, , , , , represents a preset training parameter, represents a bias term of an output layer.

[0036] It should be noted that through the cooperative calculation of the multi-scale mark vector and the spatial position / coordinate, the accurate analysis of the formula structure is realized. The core effect is reflected in three aspects: first, the double calculation mechanism of spatial position and coordinate can effectively distinguish the absolute position and relative layout relationship of the symbols in the formula, avoiding the positioning deviation of the nested formula caused by the single coordinate coding in the traditional method. Second, based on the context vector generation process of the preset training parameters, the dimension correlation of the attention weight matrix (matrix operation of HxW) is dynamically adjusted, so that the model can adaptively capture long-distance symbol dependency, especially for complex structures such as cross-line fraction and nested root. Finally, the joint calculation of the softmax function and the embedded vector can preserve the multi-scale features (vector operation of dimension d), and through the constraint of parameters, it can convert the discrete error of the traditional classifier into an optimizable problem in the continuous space.

[0037] In addition, the present application Figure 2 A handwritten mathematical formula recognition system based on multi-scale vector marking is provided for the embodiments of the present application. As Figure 2 shown, the system provided by the embodiments of the present application mainly includes: The extraction module 210 is used to extract several dimensional character feature vectors from the handwritten mathematical formula image.

[0038] The multi-scale module 220 is used to form a fusion feature vector by combining the character feature vectors of the same dimension, reduce the fusion feature vector to a preset dimension, and normalize the element values in the reduced fusion feature vector to generate a mark vector. Based on the height and width dimensions of the mark vector, a multi-scale mark vector of the current dimension is obtained.

[0039] The multi-scale module 220 includes a fusion unit, for combining the character feature vectors of the same dimension to form a fusion feature vector S; using conv to reduce the dimension of S to a preset dimension; using the sigmoid function to normalize the element values in the reduced fusion feature vector to generate a mark vector; by the formula: , the multi-scale mark vector of the i-th dimension ; wherein, represents the value of the i-th dimension mark vector in the p-th row and the q-th column; wherein H represents the total number of rows, and W represents the total number of columns.

[0040] The recognition module 230 is configured to calculate a spatial position and a spatial coordinate by using the multi-scale mark vector, calculate a context vector corresponding to the current multi-scale mark vector by using the spatial position, the spatial coordinate and preset training parameters, and obtain a formula recognition result by using the multi-scale mark vector, the context vector and a softmax function.

[0041] The above is a method embodiment of the present application. Based on the same inventive concept, the present application also provides a handwriting mathematical formula recognition device based on multi-scale vector marking. As shown in the formula (1), the device comprises a processor and a memory having executable codes stored thereon, wherein the executable codes, when executed, cause the processor to perform a handwriting mathematical formula recognition method based on multi-scale vector marking. Figure 3

[0042] Specifically, the server extracts character feature vectors of several dimensions from a handwriting mathematical formula image, groups the character feature vectors of the same dimension into a fusion feature vector, reduces the fusion feature vector to a preset dimension, and performs normalization processing on the element values in the reduced fusion feature vector to generate a mark vector. The device obtains a multi-scale mark vector based on the height and width dimensions of the mark vector, calculates a spatial position and a spatial coordinate by using the multi-scale mark vector, calculates a context vector corresponding to the current multi-scale mark vector by using the spatial position, the spatial coordinate and preset training parameters, and obtains a formula recognition result by using the multi-scale mark vector, the context vector and a softmax function.

[0043] In addition, the present application also provides a non-volatile computer storage medium having executable instructions stored thereon, wherein the executable instructions, when executed, implement a handwriting mathematical formula recognition method based on multi-scale vector marking.

[0044] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the present application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.​

Claims

1. A method for recognizing handwritten mathematical formulas based on multi-scale vector labeling, characterized in that, The method includes: Extract character feature vectors of several dimensions from images of handwritten mathematical formulas; Character feature vectors of the same dimension are combined to form a fused feature vector, which is then reduced to a preset dimension. The element values ​​in the fused feature vector after dimensionality reduction are normalized to generate a label vector. Based on the height and width dimensions of the label vector, a multi-scale label vector is obtained. The spatial location and spatial coordinates are calculated using multi-scale labeled vectors; the context vector corresponding to the current multi-scale labeled vector is calculated using the spatial location, spatial coordinates, and preset training parameters; and the formula recognition result is obtained using the multi-scale labeled vector, context vector, and softmax function.

2. The handwritten mathematical formula recognition method based on multi-scale vector labeling according to claim 1, characterized in that, From images of handwritten mathematical formulas, extract character feature vectors of several dimensions, specifically including: A convolutional neural network (CNN) using the DenseNet-121 structure of the DenseNet model is used to extract features from handwritten mathematical formula images. The character feature vectors in the handwritten mathematical formulas are obtained by using n DenseBlocks.

3. The handwritten mathematical formula recognition method based on multi-scale vector labeling according to claim 1, characterized in that, Character feature vectors of the same dimension are combined to form a fused feature vector, which is then reduced to a preset dimension. The element values ​​in the fused feature vector after dimensionality reduction are normalized to generate a label vector. Based on the height and width dimensions of the label vector, a multi-scale label vector is obtained, specifically including: Combine character feature vectors of the same dimension into a fused feature vector S; use conv reduces the dimension of S to the preset dimension; The sigmoid function is used to normalize the element values ​​in the dimensionality-reduced fused feature vector to generate a label vector; Through the formula: Obtain the multi-scale label vector of the i-th dimension. ; in, Let H represent the value of the label vector in the i-th dimension at row p and column q; where H represents the total number of rows and W represents the total number of columns.

4. The handwritten mathematical formula recognition method based on multi-scale vector labeling according to claim 1, characterized in that, Calculating spatial location and spatial coordinates using multi-scale labeled vectors specifically includes: Through formula Calculate spatial location; Through formula Calculate spatial coordinates; in, This represents the preset position index, and d represents the vector dimension. Let represent the multi-scale label vector of the i-th dimension, and c represent the positional offset term. This represents the minimum relative distance between multi-scale labeled vectors of different dimensions. This represents the maximum relative distance between multi-scale labeled vectors of different dimensions.

5. The handwritten mathematical formula recognition method based on multi-scale vector labeling according to claim 1, characterized in that, Calculate the context vector corresponding to the current multi-scale labeled vector using spatial location, spatial coordinates, and preset training parameters. Specifically, this includes: Through the formula: ， ， Calculate attention weights ; Where t represents the current step size t, b , A This indicates the preset training parameters. Let b represent the weight vector, and b represent the bias term. Let A represent the weight matrix, and let A represent the sum of the weights in the weight matrix. Indicates a hidden state. express The weight matrix, This represents the value of the weight matrix in the i-th row and j-th column; Indicates spatial location, Represents spatial coordinates, express The specific value in the p-th row and q-th column, H represents The total number of rows in the matrix, W represents The total number of columns in the matrix; The context vector C is obtained by element-wise multiplying the attention weights and the multi-scale label vector.

6. The handwritten mathematical formula recognition method based on multi-scale vector labeling according to claim 1, characterized in that, The formula recognition results are obtained by utilizing multi-scale labeled vectors, context vectors, and the softmax function, specifically including: Through the formula: ( + ) + To obtain the formula recognition result; in, This represents the multi-scale label vector for the i-th latitude. Let E represent the context vector and E represent the embedding vector. , , , , This indicates the preset training parameters. This represents the bias term of the output layer.

7. A handwritten mathematical formula recognition system based on multi-scale vector labeling, characterized in that, The system includes: The extraction module is used to extract character feature vectors of several dimensions from images of handwritten mathematical formulas; The multi-scale module is used to combine character feature vectors of the same dimension into a fused feature vector, reduce the dimensionality of the fused feature vector to a preset dimension, normalize the element values ​​in the dimensionality-reduced fused feature vector to generate a label vector, and obtain a multi-scale label vector of the current dimension based on the height and width dimensions of the label vector. The recognition module is used to calculate spatial location and spatial coordinates using multi-scale labeled vectors; calculate the context vector corresponding to the current multi-scale labeled vector using the spatial location, spatial coordinates and preset training parameters; and obtain the formula recognition result using the multi-scale labeled vector, context vector and softmax function.

8. The handwritten mathematical formula recognition system based on multi-scale vector labeling according to claim 7, characterized in that, The multi-scale module includes a fusion unit. Used to combine character feature vectors of the same dimension into a fused feature vector S; use conv reduces the dimension of S to the preset dimension; The sigmoid function is used to normalize the element values ​​in the dimensionality-reduced fused feature vector to generate a label vector; Through the formula: Obtain the multi-scale label vector of the i-th dimension. ; in, Let H represent the value of the label vector in the i-th dimension at row p and column q; where H represents the total number of rows and W represents the total number of columns.

9. A handwritten mathematical formula recognition device based on multi-scale vector labeling, characterized in that, The device includes: processor; And a memory storing executable code, which, when executed, causes the processor to perform a handwritten mathematical formula recognition method based on multi-scale vector labeling as described in any one of claims 1-6.

10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement a handwritten mathematical formula recognition method based on multi-scale vector labeling as described in any one of claims 1-6.

Images