Character recognition method and related apparatus

Abstract

The application discloses a character recognition method and related device, the character recognition method comprises: performing general feature extraction on a character image to obtain character general features; and performing size feature extraction on the character image to obtain character size features, the character size features representing sizes of each character in the character image; obtaining modulation weight features based on the character size features, modulation weight feature values in the modulation weight features being negatively correlated with corresponding character size feature values in the character size features; fusing the character general features and the modulation weight features to obtain character fusion features; and performing character prediction on the character fusion features to obtain a character recognition result. The above scheme can improve the recognition accuracy of micro characters, thereby improving the overall character recognition accuracy.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a character recognition method and related apparatus. Background Technology

[0002] Character recognition is used to identify characters in character images. Character recognition technology can be applied to many scenarios such as scene text understanding, document digitization, and product number digitization management. The general process of character recognition methods in related technologies can be described as follows: extracting general features from the character recognition image to obtain general character features; predicting these general character features to obtain the character recognition result.

[0003] Character images may suffer from mixed character sizes, meaning that the image contains characters of different sizes. For example, advertisements on billboards may include characters of different sizes; numbers with decimal points on displays (such as 23.5) may have decimal points that are different sizes from the numbers; and numbers with electricity units on power nameplates (such as 50Hz, 30℃) may have electricity units that are different sizes from the numbers.

[0004] However, character recognition methods in related technologies are not accurate enough for recognizing small characters, which affects the overall accuracy of character recognition. Summary of the Invention

[0005] This application provides a character recognition method and related apparatus, which can solve the problem that the character recognition method in the related art has insufficient recognition accuracy for small characters, which affects the overall character recognition accuracy.

[0006] This application provides a character recognition method, comprising: extracting general features from a character image to obtain general character features; extracting size features from the character image to obtain character size features, wherein the character size features characterize the size of each character in the character image; obtaining modulation weight features based on the character size features, wherein the modulation weight feature value in the modulation weight features is negatively correlated with the corresponding character size feature value in the character size features; fusing the general character features and the modulation weight features to obtain character fusion features; and performing character prediction on the character fusion features to obtain a character recognition result.

[0007] This application provides a character recognition device, comprising: a feature extraction module, an acquisition module, a fusion module, and a character prediction module; wherein, the feature extraction module is used to perform general feature extraction on a character image to obtain general character features; and to perform size feature extraction on the character image to obtain character size features, wherein the character size features characterize the size of each character in the character image; the acquisition module is used to acquire modulation weight features based on the character size features, wherein the modulation weight feature value in the modulation weight features is negatively correlated with the corresponding character size feature value in the character size features; the fusion module is used to fuse the general character features and the modulation weight features to obtain character fusion features; and the character prediction module is used to perform character prediction on the character fusion features to obtain a character recognition result.

[0008] This application provides an electronic device, including a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-described method.

[0009] This application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.

[0010] This application provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0011] The above scheme does not directly predict characters based on general character features. Instead, it extracts character size features from the character image, derives modulation weight features based on these features, and fuses the general character features with the modulation weight features to obtain character fusion features. Character prediction is then performed on these fusion features to obtain the character recognition result. On one hand, since a larger character size feature value indicates a larger character size, and the modulation weight feature value is negatively correlated with the character size feature value, fusing the modulation weight feature with the general character features allows for stronger modulation of the character feature values ​​corresponding to smaller characters. This effectively enhances the general character feature values ​​of smaller characters within the general character features, while simultaneously modulating the character feature values ​​corresponding to larger characters with weaker modulation, preserving as much of the original information of the general character feature values ​​of larger characters as possible. This improves the expressive power of the fused character features for smaller characters, thus enhancing the accuracy of character prediction for smaller characters and improving the overall character recognition accuracy. On the other hand, the extraction of character size features involves less computation, minimizing its impact on the overall computational cost of character recognition.

[0012] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0013] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0014] Figure 1 This is a flowchart illustrating an embodiment of the character recognition method provided in this application; Figure 2 This is a structural diagram of a specific example of the character recognition model of this application; Figure 3 This is a flowchart illustrating a specific example of the character recognition method provided in this application; Figure 4 This is a schematic diagram of the structure of an embodiment of the character recognition device provided in this application; Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application; Figure 6 This is a schematic diagram of the structure of an embodiment of the computer-readable storage medium of this application. Detailed Implementation

[0015] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0016] In the following description, specific details such as particular system architectures, interfaces, and technologies are presented for illustrative purposes rather than for limiting purposes, in order to provide a thorough understanding of this application.

[0017] In this document, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this document means two or more. The term "at least one" in this document means any combination of at least two of any one or more of a plurality of objects. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. Finally, the term "several" in this document means any integer greater than 0, such as 1, 2, 3, 4, 5, ...

[0018] Character recognition is used to identify characters in character images. Character recognition technology can be applied to many scenarios such as scene text understanding, document digitization, and product number digitization management. The general process of character recognition methods in related technologies can be described as follows: extracting general features from the character recognition image to obtain general character features; predicting these general character features to obtain the character recognition result.

[0019] Character images may suffer from mixed character sizes, meaning they contain characters of different sizes. For example, billboard advertisements may include characters of varying sizes; numbers with decimal points (e.g., 23.5) on a display screen may have decimal points that are different sizes from the digits; and numbers with electricity units (e.g., 50Hz, 30℃) on an electrical nameplate may have different unit sizes than the digits. Related character recognition methods often lack accuracy in recognizing smaller characters, affecting overall character recognition accuracy.

[0020] Through long-term research, the inventors of this application have discovered that the reason why character recognition methods in related technologies lack accuracy in recognizing small characters is that small characters lack detail and are easily overlooked during the general feature extraction stage, resulting in insufficient expressive power of the obtained general character features for small characters. For example: In some related technologies, character recognition methods are implemented using end-to-end models based on CNN-RNN (such as CRNN and its variants). CNN-RNN-based end-to-end models have a simple structure and fast character recognition speed, achieving high overall character recognition accuracy when there is no size mixing issue in the character image. However, when the character image has size mixing issues, the recognition accuracy for smaller characters is insufficient, leading to overall lower character recognition accuracy. This is because in CNN-RNN-based end-to-end models, the CNN extracts general character features from the character image, and the RNN uses these general features to predict the character and obtain the recognition result. During the extraction of general character features, the CNN performs multiple downsampling operations on the character image (e.g., convolution stride > 1). Multiple downsampling leads to significant loss of spatial information for smaller characters in the character image, resulting in blurred feature parts of smaller characters in the general character feature map and insufficient expressive power for smaller characters. Furthermore, RNNs have limited ability to model long sequences during character prediction and tend to ignore the contextual dependencies of smaller characters. Therefore, this leads to misidentification or missed identification of smaller characters, resulting in insufficient accuracy for recognizing smaller characters. Furthermore, the character recognition accuracy of end-to-end models based on CNN-RNN is also affected by the quality of the character image. For example, noise and lighting can cause a decrease in the quality of the character image, which will lead to a decrease in the overall character recognition accuracy.

[0021] In some related technologies, character recognition methods are implemented based on Attention-based Transformer models (such as SVTR, ViTSTR, and ABINet). In Attention-based Transformer models, a backbone network is used to extract general character features, and a self-attention mechanism is used for character prediction. Although the self-attention mechanism is better at modeling long sequences than RNNs, the general character features extracted by the backbone network are still insufficient for small characters, so the recognition accuracy for small characters remains insufficient.

[0022] Based on this, this application proposes a novel character recognition method. Its embodiments are described below: Figure 1 This is a flowchart illustrating an embodiment of the character recognition method provided in this application. Figure 1 As shown, in this embodiment, the character recognition method may include the following steps: S110: Perform general feature extraction on the character image to obtain general character features; and perform size feature extraction on the character image to obtain character size features.

[0023] Character size features characterize the size of each character in a character image.

[0024] The execution subject in this embodiment is a character recognition device, which can be any electronic device with character recognition capability.

[0025] A character image is an image containing several characters. These characters can be distributed in rows, columns, or other ways. In a row-based distribution, each row of characters can be considered a single character unit. In a column-based distribution, each column of characters can be considered a single character unit. Character images can be captured in real-time by a camera or scanner, or they can be captured in real-time but stored in a specific location.

[0026] In the character size feature, each character size feature value represents the size of the character to which it belongs. The larger the character size feature value, the larger the size of the character to which the character size feature value belongs.

[0027] S120: Obtain modulation weight features based on character size features.

[0028] The modulation weight feature value in the modulation weight feature is negatively correlated with the corresponding character size feature value in the character size feature.

[0029] The modulation weight feature characterizes the modulation intensity of each character. A larger character size feature value corresponds to a smaller modulation weight feature value, indicating a weaker modulation intensity for that character. Conversely, a smaller character size feature value corresponds to a larger modulation weight feature value, indicating a stronger modulation intensity for that character.

[0030] S130: The character general features are fused with the modulation weight features to obtain the character fusion features.

[0031] The fusion of character general features and modulation weight features involves using a larger modulation weight value to modulate the character general feature value belonging to the smaller character within the character general features, achieving a stronger modulation intensity and effectively / significantly enhancing the character general feature value belonging to the smaller character. Simultaneously, a smaller modulation intensity is used to modulate the character general feature value belonging to the larger character within the character general features, achieving a weaker modulation intensity and preserving as much of the original information of the character general feature value belonging to the smaller character as possible.

[0032] S140: Perform character prediction on the character fusion features to obtain the character recognition result.

[0033] The above scheme does not directly predict characters based on general character features. Instead, it extracts character size features from the character image, derives modulation weight features based on these features, and fuses the general character features with the modulation weight features to obtain character fusion features. Character prediction is then performed on these fusion features to obtain the character recognition result. On one hand, since a larger character size feature value indicates a larger character size, and the modulation weight feature value is negatively correlated with the character size feature value, fusing the modulation weight feature with the general character features allows for stronger modulation of the character feature values ​​corresponding to smaller characters. This effectively enhances the general character feature values ​​of smaller characters within the general character features, while simultaneously modulating the character feature values ​​corresponding to larger characters with weaker modulation, preserving as much of the original information of the general character feature values ​​of larger characters as possible. This improves the expressive power of the fused character features for smaller characters, thus enhancing the accuracy of character prediction for smaller characters and improving the overall character recognition accuracy. On the other hand, the extraction of character size features involves less computation, minimizing its impact on the overall computational cost of character recognition.

[0034] The following is a further expansion of S120: In some embodiments, S120 includes: performing a feature value inversion operation on the character size feature to obtain an initial modulation weight feature; and using the initial modulation weight feature as the final modulation weight feature.

[0035] The numerical inversion operation involves subtracting each character size feature value from the character size feature by 1 to obtain the corresponding modulation weight value in the modulation weight feature. This ensures that the modulation weight value is negatively correlated with the corresponding character size feature value.

[0036] In some embodiments, S120 includes: performing a feature value inversion operation on the character size feature to obtain an initial modulation weight feature; and adding the initial modulation weight feature to a preset value to obtain a final modulation weight feature. The preset value can be 1 or a value close to 1, such as 1.1 or 0.9.

[0037] It is understandable that adding the preset value can achieve modulation while avoiding the loss of the original information of the character's common features.

[0038] In some embodiments, S120 includes S121-S122. S121: Perform a feature value inversion operation on the character size features to obtain initial modulation weight features. S122: Obtain the final modulation weight features based on the initial modulation weight features and the character general features.

[0039] In some embodiments, S122 includes: normalizing the character general features; and fusing the normalized character general features with the initial modulation weight features to obtain the final modulation weight features.

[0040] In some embodiments, S122 includes S1221-S1222. S1221: Fusing the character general features with the character size features to obtain the gated weight features. S1222: Fusing the initial modulation weight features with the gated weight features to obtain the final modulation weight features.

[0041] It is understandable that the values ​​of each gate weight feature in the gate weight feature belong to [0, 1], and the larger the gate weight feature value, the larger the size of the corresponding character. By fusing the initial modulation weight feature with the gate weight feature, the initial modulation weight feature values ​​in the initial modulation weight feature can be dynamically adjusted according to the size of the corresponding character. This can reduce the difference between the modulation weight feature values ​​for larger and smaller characters, while limiting the range of the modulation weight feature values ​​to avoid overmodulation caused by excessively large modulation weight feature values.

[0042] In some embodiments, S1221 includes: concatenating the character general features and the character size features in the channel dimension to obtain concatenated features; sequentially performing channel dimension compression, nonlinear transformation, channel dimension compression, and normalization operations on the concatenated features to obtain gated weight features.

[0043] In this process, first-order compression and second-order compression can be implemented, but are not limited to, through convolution; nonlinear transformation can be implemented, but are not limited to, through the ReLU activation function; and normalization operations can be implemented, but are not limited to, through the Sigmoid activation function.

[0044] In some embodiments, the fusion process of character general features and character size features may omit the nonlinear transformation operation, and the first and second compressions of the channel dimension may be merged into a single compression of the channel dimension.

[0045] In some embodiments, S1222 includes: multiplying the initial modulation weight feature with the gating weight feature to obtain an intermediate modulation weight feature; and obtaining the final modulation weight feature based on the intermediate modulation weight feature.

[0046] In some embodiments, obtaining the final modulation weight feature based on the intermediate modulation weight feature includes: using the intermediate modulation weight feature as the final modulation weight feature.

[0047] In some embodiments, obtaining the final modulation weight feature based on the intermediate modulation weight feature includes: adding the intermediate modulation weight feature to a preset value to obtain the final modulation weight feature.

[0048] It is understandable that adding the preset value can achieve modulation while avoiding the loss of the original information of the character's common features.

[0049] In some embodiments, prior to S122, the method further includes: amplifying the initial modulation weighting features using a preset amplification factor.

[0050] The magnification factor is greater than 1, for example, it can be 3, 4, 5, etc.

[0051] It is understandable that by amplifying the data, the difference between the modulation weight feature values ​​of smaller characters and larger characters can be magnified.

[0052] The following is a further expansion of S130: In some embodiments, S130 includes: multiplying the character general features with the modulation weight features to obtain character fusion features.

[0053] In some embodiments, S130 includes: multiplying the character general features with the modulation weight features to obtain an initial character fusion feature; multiplying the residual weight parameters with the character general features to obtain a multiplied feature; and adding the initial character fusion feature and the multiplied feature to obtain the final character fusion feature.

[0054] It is understandable that the above process can be regarded as adding residual connections of character general features, so that even if the modulation fails, the original information of the character general features will not be lost in the final character fusion features.

[0055] Among them, the residual weight parameter is less than or equal to 1.

[0056] In some embodiments, the residual weight parameter is a fixed value, such as 0.1.

[0057] In some embodiments, the character recognition method is implemented based on a character recognition model, and the residual weight parameters are learnable parameters of the character recognition model during the training phase.

[0058] Understandably, the character recognition process is consistent between the training and application phases of the character recognition model. During training, after obtaining the predicted character recognition results, a loss function is constructed based on the difference between the predicted and actual character recognition results. The parameters of the character recognition model are then adjusted based on this loss function. Since the residual weight parameters are learnable parameters, they are adjusted along with other learnable parameters during parameter tuning. After training, the residual weight parameters, along with the other learnable parameters, are fixed and applied to character recognition in real-world scenarios. Compared to fixed residual weight parameters, using learnable parameters as residual weight parameters results in better residual connection performance.

[0059] In some embodiments, character size features belong to the size-aware prior information of the character recognition model and are not extracted by the character recognition model. Therefore, the extraction of character size features does not depend on the training of the character recognition model.

[0060] In some embodiments, character size features can also be extracted by a character recognition model.

[0061] The character size features of S110 are further expanded as follows: In some embodiments, the character size feature can be a character variance feature, where each character variance feature value represents the variance within the neighborhood of the corresponding pixel value in the character image. The variance is positively correlated with the character size within the neighborhood; that is, the larger the variance, the more drastic the change in pixel values ​​within the neighborhood, meaning a larger character size; conversely, the smaller the variance, the more gradual the change in pixel values ​​within the neighborhood, meaning a smaller character size.

[0062] In some embodiments, the character size feature can be a character height feature, which characterizes the height of each character in the character image. Each character height feature value in the character height feature represents the height of its respective character.

[0063] In some embodiments, the character size feature can be a character stroke width transform (SWT). The character stroke width transform represents the stroke width of each character in the character image. The larger the stroke width of a character, the larger the size of the character. The character stroke width feature value of each character represents the stroke width of the character to which it belongs.

[0064] In some embodiments, the character size feature is the result of fusing at least two of the character variance feature, character height feature, and character stroke width feature.

[0065] In some embodiments, the character size feature is a fusion of the character height feature and the character stroke width feature. Based on this, S120 includes: extracting stroke width features and height features from the character image respectively to obtain the character stroke width feature and character height feature; and fusing the stroke width feature and character height feature to obtain the character size feature.

[0066] In some embodiments, the fusion method for character stroke width features and character height features is weighted. When weighting, the weights of stroke width features and character height features can be the same (e.g., both are 0.5) or different (e.g., the weight of stroke width features is greater).

[0067] Understandably, compared to character variance features, character stroke width features, by analyzing stroke structure, can more accurately express character size. Character height features can compensate for the inaccuracies of certain character stroke width features in representing characters in certain fonts or characters with different case. Therefore, the character size feature obtained by fusing character stroke width features and character height features provides a more accurate representation of character size.

[0068] In some embodiments, prior to S110, the method further includes preprocessing the character image. Preprocessing includes at least one of image size normalization, grayscale conversion, and pixel value normalization. Image size normalization refers to scaling the character image to a standard size, for example, scaling the height to 32 pixels and scaling the width according to the height scaling ratio, so as to maintain the aspect ratio while scaling. Pixel value normalization refers to normalizing each pixel value in the character image to a specific pixel value range, such as [-1, 1] or [0, 1].

[0069] In some embodiments, for an image containing multiple character units, the character image can be divided into multiple character image blocks according to the character units, and each character image block can be preprocessed and character recognized separately.

[0070] To facilitate understanding, the character recognition method provided in this application is illustrated below with a specific example: Figure 2 This is a structural diagram of a specific example of the character recognition model of this application. For example... Figure 2 As shown, the character recognition model includes a general feature extraction module, a gating weight feature acquisition module, a modulation weight feature acquisition module, a modulation module, a deep feature extraction module, and a character prediction module. The general feature extraction module can be the backbone feature extraction network, which can be ResNet-18, VGG, or other ResNet series networks. The deep feature extraction module can be a multi-layer Transformer encoder.

[0071] Figure 3 This is a flowchart illustrating a specific example of the character recognition method provided in this application. For example... Figure 3 As shown, character recognition methods based on character recognition models include: 1. Obtain character images.

[0072] 2. Preprocess the character image.

[0073] 3. Perform feature extraction on the character images to obtain the general character feature F and the character size feature scale_factor. The scale of F is [B, C, H, W], where B, C, H, and W represent the batch size dimension, channel dimension, height dimension, and width dimension, respectively. The batch size dimension refers to the number of character images processed simultaneously. The scale_factor is [B, 1, H, W].

[0074] 3.1 Use the general feature extraction module to extract general features from the character image to obtain the general character feature F.

[0075] 3.2 Obtain the character size feature scale_factor of the pre-extracted character image.

[0076] (1) Extract the character stroke width feature s_swt from the character image. The extraction steps include: The process involves converting the character image to a grayscale image; detecting edges using the Canny algorithm; calculating the image gradient direction; tracing the stroke width along the gradient direction to generate initial character stroke width features; and post-processing the initial character stroke width features to obtain the final character stroke width features. The post-processing includes at least one of the following: removing unreasonable character stroke width feature values ​​(such as those that are too large or too small), filling in missing character stroke width feature values, and normalizing the character stroke width values ​​to the range [0, 1].

[0077] (2) Extract the character height feature s_height from the character image. The extraction steps include: Convert the character image into a binary image; sum the pixel values ​​of each row in the binary image to obtain the number of character pixels in each row. The number of character pixels in each row constitutes the row projection vector, and the length is the height of the character image; analyze the non-zero regions (i.e., character regions) in the row projection vector; estimate the overall height of the text line and the continuous length of the non-zero regions; local height estimation: analyze the horizontal projection curve using a sliding window; normalize the character height values ​​to the range [0, 1] to obtain the character height features.

[0078] (3) Upsample the character stroke width feature s_swt and the character height feature s_height to update the character stroke width feature to s_swt' (scale [B, 1, H, W]) and the character height feature to s_height' (scale [B, 1, H, W]) respectively.

[0079] (4) Weight the character stroke width feature s_swt' and the character height feature s_height' to obtain the character size feature scale_factor. The calculation formula is as follows: scale_factor=w_swt s_swt'+w_height s_height'; w_swt+w_height=1; Where w_swt and w_height represent the weights of the character stroke width feature and the character height feature, respectively.

[0080] 4. The gating weight feature acquisition module is used to fuse the character size feature scale_factor with the character general feature F to obtain the gating weight feature G.

[0081] (1) The character size feature scale_factor and the character general feature F are concatenated in the channel dimension to obtain the concatenated feature concat_feat, with a scale of [B, C+1, H, W].

[0082] (2) The concatenated feature concat_feat is compressed once by 1×1 convolution to obtain the concatenated feature after compression, with the scale being [B, C / 4, H, W].

[0083] (3) The nonlinear transformation of the spliced ​​features after one compression is performed by the ReLU activation function to obtain the spliced ​​features after nonlinear transformation, with the scale being [B, C / 4, H, W].

[0084] (4) The splicing features after nonlinear transformation are compressed twice by another 1×1 convolution to obtain the splicing features after compression, with the scale being [B, 1, H, W].

[0085] (5) The concatenated features after secondary compression are normalized by the Sigmoid activation function, so that the concatenated feature values ​​in the concatenated features after secondary compression are limited to the range of [0, 1], and the gated weight feature G is obtained with a scale of [B, 1, H, W]. The gated weight feature G represents the modulation intensity of the general feature value of each character as believed by the character recognition model, that is, how much modulation intensity should be applied to the general feature value of characters belonging to small characters and how much modulation intensity should be applied to the general feature value of characters belonging to large characters.

[0086] 5. The modulation weight feature acquisition module obtains the modulation weight feature based on the gated weight feature G and the character size feature scale_factor.

[0087] (1) Perform eigenvalue inversion on the character size feature scale_factor, and then multiply it by the amplification factor γ to obtain the initial modulation weight feature gain_from_scale. The calculation formula is as follows: gain_from_scale=γ (1.0-scale_factors).

[0088] Understandably, in gain_from_scale, the modulation weight eigenvalues ​​of smaller characters are larger, and those of larger characters are smaller. The amplification factor, after amplifying the result of the eigenvalue inversion operation, amplifies the difference between the modulation weight eigenvalues ​​of the larger and smaller characters.

[0089] (2) Multiply the initial modulation weight feature gain_from_scale by the gate weight feature G to obtain the intermediate modulation weight feature; add the intermediate modulation weight feature to 1 to obtain the final modulation weight feature modulation. The calculation formula is as follows: modulation = 1.0 + G gain_from_scale.

[0090] It is understandable that the values ​​of each gate weight feature in the gate weight feature are in the range of [0, 1]. Therefore, by restricting the gate weight feature, it is possible to avoid the situation where the modulation weight feature value is too large, thereby avoiding over-modulation.

[0091] 6. Modulate the character general feature F using the modulation module based on the modulation weight feature modulation to obtain the character fusion feature.

[0092] (1) Multiply the character general feature F with the modulation weight feature modulation to obtain the initial character fusion feature F_fused = F modulation. Due to the broadcast mechanism, the scale of F_fused is [B, C, H, W].

[0093] It is understandable that, since the modulation weight value of larger characters is smaller and the modulation weight value of smaller characters is larger, multiplying them by the modulation weight feature value can effectively enhance the general feature value of the character to which the smaller character belongs, while keeping the general feature value of the character to which the larger character belongs basically unchanged.

[0094] (2) Multiply the residual weight parameter residual_weight with the character general feature F to obtain the multiplied feature F. residual_weight.

[0095] Here, residual_weight is a learnable parameter of the character recognition model.

[0096] (3) Combine the initial character fusion feature F_fused and the multiplication feature F The final character fusion feature is obtained by adding the residual weights, and the calculation formula is as follows: F_fused = F_fused + F residual_weight.

[0097] It is understandable that adding the multiplicative features can, on the one hand, preserve the original information of the general features of the characters in the event of modulation failure, and on the other hand, improve the training stability (convergence speed, normal convergence, etc.) of the character recognition model during the training phase.

[0098] 7. Use the deep feature extraction module to encode the character fusion features to obtain the deep character features.

[0099] 8. Use the character prediction module to perform sequence modeling and prediction of deep character features to obtain character recognition results.

[0100] In the specific example above, firstly, conventional general feature extraction is performed on the character image to obtain general character features. Secondly, to address the issue of character size mixing in the character image, additional size feature extraction is performed on the character image to obtain spatial-level character size features (size-aware prior information). Then, the character size features are fused with the general character features to obtain gating weight features, and the gating weight features and character size features are combined to obtain modulation weight features. Finally, the general character features are modulated using the modulation weight features to obtain the character fusion features.

[0101] On the one hand, by modulating the weighted features, the general feature values ​​of smaller characters in the general features of characters can be significantly enhanced, thereby improving the expressive power of smaller characters and thus improving the recognition accuracy of smaller characters.

[0102] On the other hand, introducing gating weight features to obtain modulation weight features can improve the fit between modulation weight features and each character, and can also avoid over-modulation caused by excessively large modulation weight features, thereby improving the robustness and practicality of the character recognition method.

[0103] On the other hand, compared with character variance features, the character size features obtained by fusing character stroke width features and character height features are more accurate in expressing character size.

[0104] On the other hand, compared to directly concatenating or simply summing the initial modulation weight features with the character general features to obtain the character fusion features, multiplying the final modulation weight features with the character general features to obtain the character fusion features can not only enhance the character general feature values ​​of smaller characters, but also retain the original character general feature values ​​of larger characters as much as possible.

[0105] On the other hand, if character images with scale mixing issues are used as training data to train a character recognition model to improve the accuracy of recognizing smaller characters (a purely data-driven method), the improvement is limited by the scale of the training data, and the details of smaller characters in the training data are insufficient, resulting in a minor improvement. In the specific example above, the general character features are modulated directly based on the modulation weight features obtained from the character size features, enhancing the general character feature values ​​of smaller characters, thereby improving the accuracy of recognizing smaller characters (a physical model-guided method). This method is not limited by the scale of the training data and the improvement is significant.

[0106] On the other hand, compared with fixed residual weight parameters, introducing learnable residual weight parameters can not only ensure that the original information of the general features of the character is preserved when modulation fails, but also improve the training stability of the character recognition model and avoid performance degradation caused by extreme modulation.

[0107] Figure 4 This is a schematic diagram of the structure of an embodiment of the character recognition device provided in this application, as shown below. Figure 4 As shown, the character recognition device includes: a feature extraction module, an acquisition module, a fusion module, and a character prediction module. Wherein: The feature extraction module is used to perform general feature extraction on the character image to obtain general character features; and to perform size feature extraction on the character image to obtain character size features, which represent the size of each character in the character image.

[0108] The acquisition module is used to obtain modulation weight features based on character size features. The modulation weight feature value in the modulation weight feature is negatively correlated with the corresponding character size feature value in the character size feature.

[0109] The fusion module is used to fuse the general features of characters with the modulation weight features to obtain the character fusion features.

[0110] The character prediction module is used to predict characters based on character fusion features to obtain character recognition results.

[0111] For further detailed descriptions of this embodiment, please refer to the other embodiments described above, which will not be repeated here.

[0112] Figure 5 This is a schematic diagram of the structure of an embodiment of the electronic device of this application. Figure 5 As shown, the electronic device 50 includes a memory 51 and a processor 52. The processor 52 is used to execute program instructions stored in the memory 51 to implement the steps in any of the above method embodiments. In a specific implementation scenario, the electronic device 50 may include, but is not limited to, a microcomputer or a server. In addition, the electronic device 50 may also include a laptop computer, a tablet computer, or other carrier device, which is not limited here.

[0113] Specifically, processor 52 controls itself and memory 51 to implement the steps in any of the above method embodiments. Processor 52 may also be referred to as a CPU (Central Processing Unit). Processor 52 may be an integrated circuit chip with signal processing capabilities. Processor 52 may also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. A general-purpose processor may be a microprocessor or any conventional processor. Furthermore, processor 52 may be implemented using integrated circuit chips.

[0114] Please see Figure 6 , Figure 6 This is a schematic diagram of a computer-readable storage medium according to an embodiment of the present application. The computer-readable storage medium 60 stores program instructions 601 thereon, which, when executed by a processor, implement the steps in any of the above method embodiments.

[0115] This application also provides a computer program product comprising a computer program that, when executed by a processor, can implement the steps of the methods described in any of the foregoing embodiments. Specifically, the computer program product can be a software or program product containing a computer program, capable of running on a computing device or stored on any available medium.

[0116] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0117] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed methods and apparatus can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, units or components may be combined or integrated into another system, or some features may be ignored or not executed. In another image location, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical, or other forms.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Claims

1. A character recognition method, characterized in that, include: General feature extraction is performed on the character image to obtain the general character features; as well as, Size features are extracted from the character image to obtain character size features, which characterize the size of each character in the character image; Based on the character size feature, a modulation weight feature is obtained, wherein the modulation weight feature value in the modulation weight feature is negatively correlated with the corresponding character size feature value in the character size feature; The character general features are fused with the modulation weight features to obtain the character fusion features; Character prediction is performed on the character fusion features to obtain character recognition results.

2. The method according to claim 1, characterized in that, The step of obtaining modulation weight features based on the character size features includes: Perform a feature value inversion operation on the character size feature to obtain the initial modulation weight feature; The final modulation weight features are obtained based on the initial modulation weight features and the character general features.

3. The method according to claim 2, characterized in that, The process of obtaining the final modulation weight features based on the initial modulation weight features and the character general features includes: The character general features are fused with the character size features to obtain the gating weight features; The initial modulation weight feature is fused with the gating weight feature to obtain the final modulation weight feature; And / or, Before obtaining the final modulation weight features based on the initial modulation weight features and the character general features, the method further includes: The initial modulation weighting features are amplified using a preset amplification factor.

4. The method according to claim 3, characterized in that, The process of fusing the general character features with the character size features to obtain the gating weight features includes: The character general features and the character size features are concatenated along the channel dimension to obtain the concatenated features; The splicing features are sequentially subjected to a first compression of the channel dimension, a nonlinear transformation, a second compression of the channel dimension, and a normalization operation to obtain the gated weight features. And / or, The step of fusing the initial modulation weight feature with the gating weight feature to obtain the final modulation weight feature includes: The initial modulation weight feature is multiplied by the gate weight feature to obtain the intermediate modulation weight feature; The final modulation weight feature is obtained based on the intermediate modulation weight feature.

5. The method according to claim 4, characterized in that, The process of obtaining the final modulation weight feature based on the intermediate modulation weight feature includes: The intermediate modulation weight feature is added to a preset value to obtain the final modulation weight feature.

6. The method according to claim 1, characterized in that, The step of fusing the character general features with the modulation weight features to obtain character fusion features includes: Multiply the general character features by the modulation weight features to obtain the initial character fusion features; The residual weight parameters are multiplied with the character general features to obtain the multiplied features; The initial character fusion feature and the multiplication feature are added together to obtain the final character fusion feature.

7. The method according to claim 6, characterized in that, The character recognition method is implemented based on a character recognition model, and the residual weight parameter is a learnable parameter of the character recognition model during the training phase.

8. The method according to claim 1, characterized in that, The step of extracting size features from the character image to obtain character size features includes: The character image is subjected to stroke width feature extraction and height feature extraction respectively, resulting in character stroke width feature and character height feature; The stroke width feature and the character height feature are fused to obtain the character size feature.

9. An electronic device, characterized in that, It includes a memory and a processor, the processor being configured to execute program instructions stored in the memory to implement the method of any one of claims 1-8.

10. A computer-readable storage medium / program product, characterized in that, It stores program instructions / computer programs that, when executed by a processor, implement the method of any one of claims 1-8.

Images