Unsupervised training method of text recognition model and related equipment

Abstract

Embodiments of the present application disclose a kind of unsupervised training method of character recognition model and related equipment.The method comprises: obtaining original image, and the original image is the image containing character;Mask processing is carried out to part of pixel of original image, and mask image is obtained, and the mask image contains pixel masked area and pixel unmasked area;Mask image is input into neural network model, and the information of character of pixel masked area is predicted based on the information of character that pixel unmasked area has using neural network model, and prediction result is obtained, wherein, the information of character is the information of partial structure with single character and / or the information of several complete characters, or the information without character;Based on prediction result and the original image, the parameters of neural network model are updated.The present application solves the technical problem that existing unsupervised character recognition cannot finely learn the structure information of single character.

Description

Technical Field

[0001] This invention relates to the field of information recognition technology, and in particular to an unsupervised training method and related equipment for a character recognition model. Background Technology

[0002] In recent years, deep learning has been widely applied in fields such as image classification, object detection, and segmentation. However, this has also brought problems such as high data annotation costs and long processing times. How to effectively utilize large amounts of unlabeled data for self-supervised learning has become a current research hotspot. Self-supervised learning can extract its own supervisory information from large-scale unlabeled datasets and train the network using this constructed supervisory information, thereby learning representational information valuable for downstream tasks. In computer vision applications such as image classification, object detection, and segmentation, techniques utilizing self-supervised contrastive learning of representational information have achieved significant results. However, due to the special nature of text sequences, self-supervised learning has not been well applied in the field of text recognition.

[0003] Currently, most text recognition algorithms still rely on fully supervised learning, requiring large amounts of labeled data for modeling. Therefore, how to efficiently learn sequence-based text information representations without human supervision is a long-standing problem. Solving this problem can unlock the practical application value of large-scale unlabeled datasets. Based on this, existing technologies propose an unsupervised text recognition method based on contrastive learning, which leverages the practical application value of large-scale unlabeled datasets by learning sequence-based text information representations. The current technical strategies for unsupervised text recognition are primarily based on contrastive learning methods. However, within the current technological framework, unsupervised text recognition still faces the following challenges:

[0004] Unsupervised text recognition methods based on contrastive learning construct positive and negative samples by augmenting text images. Then, they learn text information representations by narrowing the feature distance between positive samples and widening the feature distance between negative samples. However, this method only models the degree of text similarity and cannot learn the structural information of individual characters in detail, such as the structural information of radicals and components. At the same time, it ignores the contextual semantic information of the entire text string and cannot efficiently learn text information representations based on sequence.

[0005] No effective solution has yet been proposed to address the above problems. Summary of the Invention

[0006] This invention provides an unsupervised training method and related equipment for a character recognition model, which at least solves the technical problem that existing unsupervised character recognition methods cannot learn the structural information of individual characters in detail.

[0007] According to one aspect of the present invention, an unsupervised training method for a character recognition model is provided, comprising:

[0008] Obtain the original image, which is an image containing text;

[0009] A mask image is obtained by performing masking processing on a portion of the pixels of the original image. The mask image includes masked areas and unmasked areas of pixels.

[0010] The masked image is input into a neural network model. The neural network model predicts the text information of the masked area of ​​the pixel based on the text information in the unmasked area of ​​the pixel, and obtains the prediction result. The text information is information with a partial structure of a single character and / or information of several complete characters, or information without any characters.

[0011] Based on the prediction results and the original image, the parameters of the neural network model are updated.

[0012] In some embodiments, the neural network model includes a feature aggregation module and a pixel prediction module, wherein, before inputting the mask image into the neural network model, the method further includes:

[0013] After the mask image is cut into multiple first strip images, the multiple first strip images are input into the neural network model. The first strip image includes a masked strip image belonging to the masked area of ​​the pixel and an unmasked strip image belonging to the unmasked area of ​​the pixel.

[0014] The steps of using the neural network model to predict the text information in the masked region of a pixel based on the text information in the unmasked region of the pixel, and obtaining the prediction result, include:

[0015] The feature aggregation module determines the global image features of the unmasked strip image and inputs the global image features into the pixel prediction module. The global image features include contextual semantic features, which are used to describe the contextual semantic information of the text information in the unmasked strip image.

[0016] The pixel prediction module predicts the pixel prediction values ​​of the masked strip image based on the global features of the image, and updates the parameters of the neural network model based on the pixel prediction values ​​and the original image.

[0017] In some embodiments, the step of updating the parameters of the neural network model based on the pixel prediction values ​​and the original image includes:

[0018] Based on the pixel prediction value, the pixels in the masked area of ​​the pixel in the mask image are reconstructed to obtain the reconstructed image;

[0019] Determine the pixel similarity between the original image and the reconstructed image;

[0020] The parameters of the neural network model are updated based on the pixel similarity.

[0021] In some embodiments, the neural network model further includes a label prediction module, wherein, before updating the parameters of the neural network model, the method further includes:

[0022] The global image features of the unmasked strip image are input into the label prediction module, and the global image features include the contextual semantic features;

[0023] The label prediction module predicts the category label of the text information in the masked strip image based on the contextual semantic features to obtain the category prediction label. The category label is a label used to mark the category to which the text information belongs after classifying the text information according to a preset classification rule.

[0024] Based on the cutting position and number of cuts of the masked image, the original image is cut into multiple second strip images, and the category true label of the text information of each second strip image is determined. Then, each masked strip image corresponds to a category true label, so that the parameters of the neural network model are updated based on the category prediction label and the category true label of each masked strip image.

[0025] In some embodiments, the step of updating the parameters of the neural network model based on the predicted category label and the true category label for each of the masked strip images includes:

[0026] Determine the label similarity between the predicted category label and the true category label for each of the masked strip images;

[0027] The parameters of the neural network model are updated based on the degree of label similarity.

[0028] In some embodiments, the neural network model further includes a feature extraction module, wherein, before determining the global image features of the unmasked strip image through the feature aggregation module, the method further includes:

[0029] The feature extraction module extracts a first image feature for each of the first strip images, wherein the first image feature is a feature containing the text information of the first strip image;

[0030] The step of determining the global image features of the unmasked strip image through the feature aggregation module includes:

[0031] The first image features are input into the feature aggregation module;

[0032] The feature aggregation module aggregates the text information to determine the contextual semantic information of the text information in each of the first strip images, thereby obtaining the global image features of the unmasked strip image.

[0033] In some embodiments, the neural network model further includes a text prediction module, and when masking a portion of the original image to obtain a masked image, the method further includes:

[0034] The original text is generated based on the text in the original image, and a portion of the text in the original text is masked to obtain masked text.

[0035] Before updating the parameters of the neural network model, the method further includes:

[0036] The masked text is input into the neural network model. The text prediction module predicts the masked text based on the unmasked text in the masked text and obtains the text prediction result. Based on the original text and the text prediction result, the parameters of the neural network model are updated.

[0037] In some embodiments, before the text prediction module predicts the masked text in the masked text based on the unmasked text in the masked text, the method further includes:

[0038] The original image is cut into multiple second strip images and input into the neural network model. The feature aggregation module determines the image sequence features of the second strip images. The image sequence features include contextual semantic features, which are used to describe the contextual semantic information of the text information in the second strip images.

[0039] The language sequence features of each unmasked character in the masked text are extracted using a predetermined language model. The language sequence features are features that contain the contextual semantic information of each character.

[0040] The steps of using the text prediction module to predict the masked text in the masked text based on the unmasked text in the masked text, and obtaining the text prediction result, include:

[0041] Obtain the image sequence features and the language sequence features;

[0042] The feature aggregation module aggregates the image sequence features and the language sequence features to obtain the global character features of each character.

[0043] Based on the global features of the characters, predict the masked characters in the masked text to obtain the character prediction result.

[0044] In some embodiments, before determining the image sequence features of the second strip image by the feature aggregation module, the method further includes:

[0045] The feature extraction module extracts a second image feature for each of the second strip images, the second image feature being a feature containing the text information of the second strip image;

[0046] The step of determining the image sequence features of the second strip image through the feature aggregation module includes:

[0047] The second image features are input into the feature aggregation module;

[0048] The feature aggregation module aggregates the text information of each second strip image to determine the contextual semantic information of the text information in each second strip image, thereby obtaining the image sequence features of the unmasked strip image.

[0049] In some embodiments, the step of updating the parameters of the neural network model based on the original text and the text prediction result includes:

[0050] Based on the text prediction results, the masked text in the masked text is reconstructed to obtain the reconstructed text;

[0051] Determine the degree of text similarity between the original text and the reconstructed text;

[0052] The parameters of the neural network model are updated based on the text similarity.

[0053] In some embodiments, the method further includes:

[0054] Obtain labeled images containing text that have been labeled with labeled data;

[0055] The parameters of the neural network model are adjusted based on the labeled image.

[0056] According to a second aspect of the present invention, an unsupervised character recognition method is also provided, comprising:

[0057] The system receives a text image to be recognized and uses a text recognition model to recognize the text in the text image. The text recognition model is trained using an unsupervised training method.

[0058] According to a third aspect of the present invention, an unsupervised training apparatus for a character recognition model is also provided. The unsupervised training apparatus for the character recognition model is used to implement the unsupervised training method for the character recognition model. The apparatus includes:

[0059] The acquisition module is used to acquire the original image, which is an image containing text;

[0060] A masking module is used to perform masking processing on a portion of the pixels of the original image to obtain a mask image, wherein the mask image includes masked areas and unmasked areas of pixels;

[0061] The prediction module is used to input the masked image into a neural network model, and use the neural network model to predict the text information of the masked area of ​​the pixel based on the text information of the unmasked area of ​​the pixel, and obtain the prediction result. The text information is the partial structure of a single character and / or information of several complete characters.

[0062] The parameter update module is used to update the parameters of the neural network model based on the prediction result and the original image.

[0063] According to a fourth aspect of the present invention, a computer device is also provided, comprising:

[0064] processor;

[0065] Memory used to store processor-executable instructions;

[0066] The processor is configured to execute the unsupervised training method of the character recognition model.

[0067] According to a fifth aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to execute the unsupervised training method of the character recognition model.

[0068] According to a sixth aspect of the present invention, a processor is also provided, the processor being configured to run a program, wherein the program, when running, executes the unsupervised training method of the character recognition model.

[0069] In this embodiment of the invention, a masked image obtained by masking a portion of the pixels of the original image is input into a neural network model. The neural network model uses the text information in the unmasked areas of the pixels in the masked image to predict the text information in the masked areas. Since this prediction process requires reconstructing the entire character based on the partial structure of a single character, the trained character recognition model can recognize the structural information of a single character, such as the structural information of radicals and components. If the entire character is masked, it is necessary to reconstruct the character based on the information of the remaining characters, which enables the character recognition model to recognize the semantic information of the text context and improves the recognition accuracy of the character recognition model. Attached Figure Description

[0070] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0071] Figure 1 This is a flowchart illustrating an unsupervised training method for a character recognition model according to an embodiment of the present invention.

[0072] Figure 2 for Figure 1 A flowchart illustrating the process of predicting pixel prediction values ​​in step S13.

[0073] Figure 3 for Figure 2 A schematic diagram illustrating the process of updating the parameters of a neural network model based on pixel prediction values ​​and the original image;

[0074] Figure 4 A flowchart illustrating the process of predicting global image prediction features for a masked stripe image;

[0075] Figure 5 for Figure 4 A schematic diagram illustrating the process of updating the parameters of a neural network model based on global image prediction features and the original image;

[0076] Figure 6 A flowchart illustrating the process of generating true category labels for raw images offline;

[0077] Figure 7 A flowchart illustrating the process of obtaining the text prediction structure;

[0078] Figure 8 A flowchart illustrating the process of updating the parameters of a neural network model based on the text-based betting results and the original text;

[0079] Figure 9 This is a schematic diagram of the structure of an unsupervised training device for a character recognition model provided in an embodiment of the present invention.

[0080] Figure 10 This is a system block diagram illustrating the overall flow of the unsupervised character recognition method provided in the seventh embodiment of the present invention.

[0081] Figure 11 This is a block diagram of the image acquisition module;

[0082] Figure 12 Here is a block diagram of the image cleaning module;

[0083] Figure 13 This is a block diagram of the unsupervised training module.

[0084] The above figures include the following reference numerals:

[0085] 10. Unsupervised training device; 11. Acquisition module; 12. Mask processing module; 13. Prediction module; 14. Parameter update module; 15. Image segmentation module; 16. Determination module; 17. Language extraction module; 18. Parameter adjustment module; 20. Image acquisition module; 21. Internet data collection unit; 22. Detection model inference unit; 23. Recognition matting unit; 30. Image cleaning module; 31. Recognition image processing unit; 32. ATT recognition model inference unit; 33. CTC recognition model inference unit; 34. Paddle recognition model inference unit; 35. Recognition image filtering unit; 40. Unsupervised training module; 41. Masked image reconstruction unit; 42. Masked image prediction unit; 43. Masked text reconstruction unit; 44. Recognition fine-tuning unit. Detailed Implementation

[0086] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0087] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, rather than to limit a specific order.

[0088] Unsupervised training refers to techniques that use machine learning methods to learn representations from unlabeled data. (See also: [link to related documentation]) Figure 1 The first embodiment of the present invention provides an unsupervised training method for a character recognition model, comprising the following steps:

[0089] Step S11: Obtain the original image, which is an image containing text. In this embodiment of the invention, the original image is obtained by filtering from a massive number of images collected from the Internet.

[0090] Step S12: Perform masking processing on a portion of the pixels of the original image to obtain a mask image. The mask image contains masked and unmasked pixel areas. That is, the original image is masked according to a predetermined ratio, such as masking one-third of the image area to obtain a mask image. The mask ratio and mask position can be different each time during training.

[0091] Step S13: Input the masked image into the neural network model. Using the text information in the unmasked areas of pixels, the neural network model predicts the text information in the masked areas of pixels and obtains the prediction result. The text information includes partial structural information of a single character and / or information of several complete characters, or information without any characters. Since this prediction process requires the neural network model to reconstruct the entire character based on the partial structural information, the neural network model can learn the structural information of individual characters, such as radicals and components. If the entire character is masked, the neural network model needs to reconstruct the masked character based on the information of the remaining characters. Thus, the neural network model can learn the semantic information of the text context.

[0092] Step S14: Update the parameters of the neural network model based on the prediction results and the original image. Training and updating the parameters of the neural network model according to steps S11 to S14 helps improve the downstream recognition performance of the character recognition model.

[0093] As can be seen, in this embodiment of the invention, a masked image obtained by masking a portion of the pixels of the original image is input into a neural network model. The neural network model uses the text information in the unmasked areas of the pixels in the masked image to predict the text information in the masked areas. Since this prediction process requires reconstructing the entire character based on the partial structure of a single character, the trained character recognition model can recognize the structural information of a single character, such as the structural information of radicals and components. If the entire character is masked, it is necessary to reconstruct the character based on the information of the remaining characters, which enables the character recognition model to recognize the semantic information of the text context and improves the recognition accuracy of the character recognition model.

[0094] The neural network model includes a feature aggregation module and a pixel prediction module. Before inputting the mask image into the neural network model, the unsupervised training method for the character recognition model provided in this embodiment of the invention further includes the following steps: after cutting the mask image into multiple first strip images, inputting the multiple first strip images into the neural network model. The first strip images include masked strip images belonging to the masked pixel regions and unmasked strip images belonging to the unmasked pixel regions. For further details, please refer to [link to relevant documentation]. Figure 2Step S13, which uses a neural network model to predict the text information in the masked region of a pixel based on the text information in the unmasked region of the pixel and obtains the prediction result, includes:

[0095] Step S131: Determine the global features of the unmasked strip image through the feature aggregation module, and input the global features into the pixel prediction module. The global features include contextual semantic features, which are used to describe the contextual semantic information of the textual information in the unmasked strip image.

[0096] Step S132: The pixel prediction module predicts the pixel prediction values ​​of the masked strip image based on the global features of the image. The parameters of the neural network model are then updated based on the pixel prediction values ​​and the original image. During this process, when a portion of the text structure is masked, the entire text needs to be reconstructed based on that partial information, allowing the model to learn structural information such as radicals and components of individual characters. When the entire text is masked, it needs to be reconstructed based on information from the remaining text, ensuring that the neural network model learns the contextual semantic information of the text.

[0097] Please see below. Figure 3 Step S132, which updates the parameters of the neural network model based on the pixel prediction values ​​and the original image, includes:

[0098] Step S321: Based on the pixel prediction values, reconstruct the pixels in the masked areas of the mask image to obtain the reconstructed image. This involves combining the pixel prediction values ​​with the mask image to obtain the reconstructed image.

[0099] Step S322: Determine the pixel similarity between the original image and the reconstructed image. In this embodiment of the invention, the mean squared error loss function can be used to measure the pixel similarity between the original image and the reconstructed image.

[0100] Step S323: Update the parameters of the neural network model based on pixel similarity. When updating the neural network model, the backpropagation algorithm can be used to update the parameters of the feature aggregation module and pixel prediction module.

[0101] In this embodiment of the invention, the neural network model further includes a label prediction module, wherein, please refer to... Figure 4 Before step S14 updates the parameters of the neural network model, the method provided in this embodiment of the invention further includes:

[0102] Step S21: Input the global image features of the unmasked strip image into the label prediction module. The global image features include contextual semantic features.

[0103] Step S22: The label prediction module predicts the category label of the text information in the masked strip image based on the contextual semantic features, thereby obtaining the category prediction label. The category label is a label used to mark the category to which the text information belongs after classifying it according to a preset classification rule. In this prediction process, the neural network model needs to have the ability to perform high-dimensional feature prediction based on contextual semantic information. This method allows it to learn the contextual semantic information of the text image, improves feature representation capabilities, and helps improve the downstream recognition performance of the text recognition model.

[0104] Step S23: Based on the cutting position and number of cuts in the mask image, the original image is cut into multiple second strip images, and the true category label of the text information in each second strip image is determined. Each masked strip image then corresponds to a true category label. The parameters of the neural network model are updated based on the predicted category label and the true category label of each masked strip image. Each masked strip image corresponds to a true category label because the cutting position and number of cuts in the original image and the mask image are the same. Therefore, each masked strip image corresponds to a second strip image at the corresponding position, facilitating similarity comparison training of the corresponding labels (i.e., predicted category label and true category label). The true category label is the category label corresponding to the actual text information in the second strip image.

[0105] In this embodiment of the invention, the neural network model further includes a feature extraction module. Before determining the global image features of the unmasked strip image through the feature aggregation module in step S21, the method provided in this embodiment further includes: extracting a first image feature for each first strip image through the feature extraction module. The first image feature is a feature containing text information present in the first strip image. Therefore, step S21, determining the global image features of the unmasked strip image through the feature aggregation module, includes: inputting the first image features into the feature aggregation module, aggregating the text information through the feature aggregation module, determining the contextual semantic information of the text information present in each first strip image, and obtaining the global image features of the unmasked strip image.

[0106] This invention embodiment can store multiple different category labels in a codebook so that the corresponding category labels can be called when needed. Step S23, determining the true category label of the text information of each second strip image, includes:

[0107] Step S31: Obtain the neural network predictor, which is a neural network predictor obtained after random initialization.

[0108] Step S32: Use a neural network predictor to generate feature vectors for each second strip image offline. These feature vectors are used to represent the text information in the second strip image. Since these feature vectors cannot be directly used to train the neural network model, in order to obtain the true category labels for training and updating the neural network model, the next step S33 needs to be performed based on these feature vectors.

[0109] Step S33: Obtain a codebook containing various category labels. Based on feature vectors, filter out category labels from the codebook that match the text information in the second strip image. These matching category labels are then designated as the true category labels for the second strip image. Category labels are used to mark the category to which the text information belongs after classifying it according to preset classification rules. These rules may include assigning one category label to each character, or assigning one category label to a portion of the text information (radicals, components, etc.). The codebook contains various category labels. Each category label can correspond to a single character or a portion of a character's information (radicals, components, etc.). The text information corresponding to a category label can be random, thereby enhancing the recognition accuracy of the trained text recognition model.

[0110] The neural network predictor provided in this embodiment predicts a d-dimensional feature vector for each second strip image. First, a codebook (k*d in size, containing k category labels for each d-dimensional feature vector) is randomly generated. Then, the predicted d-dimensional feature vector is used to find the category label in the codebook that has the highest similarity to the text information in the second strip image to which the feature vector belongs. This highest similarity category label is used as the true category label for each second strip image, and the neural network parameters are subsequently updated based on this true category label. Category labels include labels corresponding to both the masked and unmasked strip images. Since each original image may be trained multiple times, and the masked portion will be different each time, this embodiment generates all types of category labels for the text information of the entire original image at once. Before subsequent similarity calculation, the true category label of the corresponding second strip image is retrieved as needed based on the category labels stored in the codebook for similarity calculation.

[0111] Please see Figure 5 In step S23, the step of updating the parameters of the neural network model based on the predicted category label and the true category label of each masked strip image includes:

[0112] Step S221: Determine the label similarity between the predicted category label and the true category label for each masked bar image. Specifically, this embodiment of the invention uses the mean squared error loss function to measure the label similarity between the predicted category label and the true category label for each masked bar image.

[0113] Step S222: Update the parameters of the neural network model based on the label similarity. During the update, the parameters of the feature extraction module, label prediction module, and feature aggregation module of the neural network model are updated using the backpropagation algorithm.

[0114] To further improve the recognition performance of the text recognition model in downstream text recognition work, this embodiment of the invention masks some text characters in the recognition pseudo-labels corresponding to the image, and then uses the remaining characters and the original image information to recover the masked characters. This process requires restoring the original text based on the text image information and some text information, allowing for the learning of richer contextual semantic information. Specifically, the neural network model provided in this embodiment of the invention also includes a text prediction module. When masking some pixels of the original image to obtain a masked image in step S12, the method provided in this embodiment of the invention further includes: generating original text based on the text in the original image, and masking some characters in the original text to obtain masked text. Before updating the parameters of the neural network model in step S14, the masked text is input into the neural network model. The text prediction module predicts the masked characters in the masked text based on the unmasked characters in the masked text, and obtains the text prediction result. Based on the original text and the text prediction result, the parameters of the neural network model are updated.

[0115] Before the text prediction module predicts the masked text in the masked text based on the unmasked text in the masked text, the method provided in this embodiment of the invention further includes the following steps: cutting the original image into multiple second strip-shaped images and inputting them into a neural network model; determining the image sequence features of the second strip-shaped images through a feature aggregation module; the image sequence features include contextual semantic features, which are used to describe the contextual semantic information of the text information in the second strip-shaped images; and extracting the language sequence features of each unmasked text in the masked text using a predetermined language model; the language sequence features are features containing the contextual semantic information of each text. See also... Figure 7 The steps of using the text prediction module to predict the masked text in the masked text based on the unmasked text in the masked text, and to obtain the text prediction results, include:

[0116] Step S41: Obtain image sequence features and language sequence features.

[0117] Step S42: The image sequence features and language sequence features are aggregated by the feature aggregation module to obtain the global character features of each character. That is, the global character features are features that simultaneously include image sequence features and language sequence features.

[0118] Step S43: Predict the masked text in the masked text based on the global features of the characters, and obtain the text prediction result.

[0119] Before determining the image sequence features of the second strip image through the feature aggregation module, the method provided in this embodiment of the invention further includes: extracting second image features for each second strip image through a feature extraction module, wherein the second image features are features containing text information present in the second strip image. Then, the step of determining the image sequence features of the second strip image through the feature aggregation module includes: inputting the second image features into the feature aggregation module, aggregating the text information of each second strip image through the feature aggregation module, determining the contextual semantic information of the text information in each second strip image, and obtaining the image sequence features of the unmasked strip image.

[0120] In the embodiments of the present invention, please refer to Figure 8 The steps for updating the parameters of the neural network model based on the original text and the text prediction results include:

[0121] Step S51: Based on the text prediction results, reconstruct the masked text in the masked text to obtain the reconstructed text. This involves combining the text prediction results with the masked text to obtain the reconstructed text.

[0122] Step S52: Determine the degree of text similarity between the original text and the reconstructed text. Specifically, this embodiment of the invention uses the cross-entropy loss function to measure the degree of similarity between the original text and the reconstructed text.

[0123] Step S53: Update the parameters of the neural network model based on the text similarity. Specifically, in this embodiment of the invention, the backpropagation algorithm is used to update the parameters of the feature extraction module, feature aggregation module, and text prediction module of the neural network model.

[0124] The embodiments of the present invention can simultaneously reconstruct the entire text based on the text information of the original image and the original text. It can simultaneously learn the image sequence features of the original image and the contextual semantic features of the original text, thereby better assisting the text recognition model in downstream text recognition tasks, such as optical character recognition (OCR) tasks.

[0125] The method provided in this embodiment of the invention further includes: acquiring labeled images containing text with labeled data; and adjusting the parameters of the neural network model based on the labeled images. Compared to previous text recognition models that required a long training time to achieve good recognition results, this embodiment of the invention uses an unsupervised trained text recognition model to obtain a high-performance text recognition model in a shorter training time on labeled images. This saves training time on downstream recognition tasks and also achieves better recognition performance.

[0126] The second embodiment of the present invention also provides an unsupervised text recognition method, comprising: receiving a text image to be recognized, and using a text recognition model to recognize the text in the text image, wherein the text recognition model is trained by the unsupervised training method of the text recognition model provided in the first embodiment of the present invention. For details of the unsupervised training method of the text recognition model, please refer to the content provided in the first embodiment of the present invention. The embodiments of the present invention will not be repeated here.

[0127] For the third embodiment of the present invention, please refer to [link / reference]. Figure 9 Furthermore, an unsupervised training device 10 for a character recognition model is provided. This device implements the unsupervised training method for the character recognition model provided in the first embodiment of the present invention. The device includes an acquisition module 11, a masking module 12, a prediction module 13, and a parameter update module 14. The acquisition module 11 acquires an original image containing text. The masking module 12 performs masking processing on a portion of the original image to obtain a mask image, which includes masked and unmasked regions. The prediction module 13 inputs the mask image into a neural network model, uses the neural network model to predict the text information in the masked regions based on the text information in the unmasked regions, and obtains a prediction result. The text information includes partial structures of a single character and / or information about several complete characters. The parameter update module 14 updates the parameters of the neural network model based on the prediction result and the original image.

[0128] In some embodiments, the neural network model includes a feature aggregation module and a pixel prediction module. Before inputting the mask image into the neural network model, the apparatus provided in this embodiment further includes an image segmentation module 15. The image segmentation module 15 is used to segment the mask image into multiple first strip images, and then input these multiple first strip images into the neural network model. The first strip images include masked strip images belonging to the masked pixel regions and unmasked strip images belonging to the unmasked pixel regions. See also... Figure 2 The prediction module 13 uses a neural network model to predict the text information in the masked area of ​​a pixel based on the text information in the unmasked area of ​​the pixel, and obtains the prediction result through the following steps:

[0129] Step S131: Determine the global features of the unmasked strip image through the feature aggregation module, and input the global features into the pixel prediction module. The global features include contextual semantic features, which are used to describe the contextual semantic information of the textual information in the unmasked strip image.

[0130] Step S132: The pixel prediction module predicts the pixel prediction value of the masked strip image based on the global features of the image, so that the parameter update module 14 updates the parameters of the neural network model based on the pixel prediction value and the original image.

[0131] Please see Figure 3 The parameter update module 14 updates the parameters of the neural network model based on the pixel prediction values ​​and the original image, including the following steps:

[0132] Step S321: Based on the pixel prediction values, reconstruct the pixels in the masked areas of the mask image to obtain the reconstructed image. This involves combining the pixel prediction values ​​with the mask image to obtain the reconstructed image.

[0133] Step S322: Determine the pixel similarity between the original image and the reconstructed image. In this embodiment of the invention, the mean squared error loss function can be used to measure the pixel similarity between the original image and the reconstructed image.

[0134] Step S323: Update the parameters of the neural network model based on pixel similarity. When updating the neural network model, the backpropagation algorithm can be used to update the parameters of the feature aggregation module and pixel prediction module.

[0135] In this embodiment of the invention, the neural network model further includes a label prediction module. Before the parameter update module 14 updates the parameters of the neural network model, the apparatus provided in this embodiment of the invention further includes a determination module 16. The determination module 16 is used to determine the global image features of the unmasked strip image through the feature aggregation module, and input the global image features into the label prediction module. The global image features include the contextual semantic features of the text information in the unmasked strip image. The prediction module 13 uses the label prediction module to predict the category label of the text information in the masked strip image based on the contextual semantic features of the text information in the unmasked strip image, so as to obtain the category prediction label. The category label is a label used to mark the category to which the text information belongs after classifying the text information according to a preset classification rule. The image segmentation module 15 is further used to segment the original image into multiple second strip images based on the segmentation position and number of segments of the masked image. Then, it determines the true category label of the text information in each second strip image. Each masked strip image corresponds to a true category label, so that the parameter update module 14 updates the parameters of the feature aggregation module, pixel prediction module, and label prediction module of the neural network model based on the predicted category label and the true category label of each masked strip image. In this prediction process, the neural network model needs to have the ability to perform high-dimensional feature prediction based on contextual semantic information. This method allows it to learn the contextual semantic information of the text image, improve feature representation capabilities, and contribute to the improvement of downstream recognition performance of the text recognition model.

[0136] In this embodiment of the invention, the neural network model further includes a feature extraction module. Before the determining module 16 determines the global image features of the unmasked strip image through the feature aggregation module, the device provided in this embodiment of the invention extracts a first image feature for each first strip image through the feature extraction module of the neural network model. The first image feature is a feature containing text information of the first strip image. Therefore, the step of the determining module 16 determining the global image features of the unmasked strip image through the feature aggregation module includes: inputting the first image features into the feature aggregation module, aggregating the text information through the feature aggregation module, determining the contextual semantic information of the text information of each first strip image, and obtaining the global image features of the unmasked strip image.

[0137] Please see Figure 5 The parameter update module 14 further includes the following steps in updating the parameters of the neural network model based on the predicted category label and the true category label of each masked strip image:

[0138] Step S221: Determine the label similarity between the predicted category label and the true category label for each masked strip image.

[0139] Step S222: Update the parameters of the neural network model based on the label similarity. During the update, the parameters of the feature extraction module, label prediction module, and feature aggregation module of the neural network model are updated using the backpropagation algorithm.

[0140] The neural network model provided in this embodiment of the invention also includes a text prediction module. When the masking module 12 performs masking processing on some pixels of the original image to obtain a masked image, the masking module 12 also generates original text based on the text in the original image, and performs masking processing on some text in the original text to obtain masked text. Before the parameter update module 14 updates the parameters of the neural network model, the device provided in this embodiment of the invention inputs the masked text into the neural network model through the prediction module 13. The text prediction module predicts the masked text in the masked text based on the unmasked text in the masked text, and obtains the text prediction result, so that the parameter update module 14 updates the parameters of the neural network model based on the original text and the text prediction result.

[0141] The apparatus provided in this embodiment of the invention further includes a language extraction module 17. Before the text prediction module predicts the masked text based on the unmasked text in the masked text, the image segmentation module 15 segments the original image into multiple second strip-shaped images and inputs them into a neural network model. Then, the determination module 16 determines the image sequence features of the second strip-shaped images through a feature aggregation module. These image sequence features include contextual semantic features, which describe the contextual semantic information of the text information in the second strip-shaped images. The language extraction module 17 uses a predetermined language model (such as the BERT language model) to extract the language sequence features of each unmasked text in the masked text. These language sequence features are features containing the contextual semantic information of each text.

[0142] Please see Figure 7 The steps of prediction module 13, which uses a neural network model's text prediction module to predict the masked text in the masked text based on the unmasked text in the masked text, and to obtain the text prediction results, include:

[0143] Step S41: Obtain image sequence features and language sequence features.

[0144] Step S42: The image sequence features and language sequence features are aggregated by the feature aggregation module to obtain the global character features of each character. That is, the global character features are features that simultaneously include image sequence features and language sequence features.

[0145] Step S43: Predict the masked text in the masked text based on the global features of the characters, and obtain the text prediction result.

[0146] Before the determining module 16 determines the image sequence features of the second strip image through the feature aggregation module, the apparatus provided in this embodiment of the invention extracts the second image features of each second strip image through the feature extraction module of the neural network model. The second image features are features containing text information of the second strip image. The step of the determining module 16 determining the image sequence features of the second strip image through the feature aggregation module of the neural network model includes: inputting the second image features into the feature aggregation module, performing aggregation processing on the text information of each second strip image through the feature aggregation module, determining the contextual semantic information of the text information in each second strip image, so as to obtain the image sequence features of the unmasked strip image.

[0147] In the embodiments of the present invention, please refer to Figure 8 The parameter update module 14 updates the parameters of the neural network model based on the original text and the text prediction results, including the following steps:

[0148] Step S51: Based on the text prediction results, reconstruct the masked text in the masked text to obtain the reconstructed text. This involves combining the text prediction results with the masked text to obtain the reconstructed text.

[0149] Step S52: Determine the degree of text similarity between the original text and the reconstructed text. Specifically, this embodiment of the invention uses the cross-entropy loss function to measure the degree of similarity between the original text and the reconstructed text.

[0150] Step S53: Update the parameters of the neural network model based on the text similarity. Specifically, in this embodiment of the invention, the backpropagation algorithm is used to update the parameters of the feature extraction module, feature aggregation module, and text prediction module of the neural network model.

[0151] The embodiments of the present invention can simultaneously reconstruct the entire text based on the text information of the original image and the original text. It can simultaneously learn the image sequence features of the original image and the contextual semantic features of the original text, thereby better assisting the text recognition model in downstream text recognition tasks, such as optical character recognition (OCR) tasks.

[0152] The apparatus provided in this embodiment of the invention further includes a parameter adjustment module 18. The parameter adjustment module 18 is used to acquire labeled images containing text and to adjust the parameters of the neural network model based on the labeled images. Compared to previous text recognition models that required a long training time to achieve good recognition results, this embodiment of the invention uses an unsupervised trained text recognition model, which can be trained on labeled images in a shorter time to obtain a high-performance text recognition model. This saves training time on downstream recognition tasks and also achieves better recognition performance.

[0153] A fourth embodiment of the present invention also provides a computer device, which includes a processor and a memory for storing processor-executable instructions. The processor is configured to execute an unsupervised training method for a character recognition model. For details of this unsupervised training method, please refer to the first embodiment of the present invention; further details will not be repeated here.

[0154] The fifth embodiment of the present invention also provides a computer-readable storage medium, which includes a stored program, wherein, when the program is running, it controls the device where the computer-readable storage medium is located to execute an unsupervised training method for a character recognition model. For details of the unsupervised training method, please refer to the content of the first embodiment of the present invention, and the embodiments of the present invention will not be repeated here.

[0155] The sixth embodiment of the present invention also provides a processor for running a program, wherein the program executes an unsupervised training method for a character recognition model during runtime. For details of the unsupervised training method, please refer to the content of the first embodiment of the present invention, which will not be repeated here.

[0156] The seventh embodiment of the present invention provides a specific application embodiment of the unsupervised character recognition method based on the above six embodiments. Please refer to the following content for details.

[0157] In the unsupervised text recognition method provided in this embodiment of the invention, the apparatus mainly includes an image acquisition module 20, an image cleaning module 30, an unsupervised training module 40, and a downstream text recognition scenario application part.

[0158] Figure 10 This is a system block diagram illustrating the overall flow of the unsupervised text recognition method of the present invention. The unsupervised text recognition method includes: acquiring original images from massive image resources on the Internet using an image acquisition module 20; cleaning and filtering the massive original images using an image cleaning module 30; and then training the original images unsupervised using an unsupervised training module 40. Specifically, the image acquisition module 20 uses a text detection model to crop small images that may contain text from the massive Internet images; the image cleaning module 30 uses a text recognition model to filter out noisy small images containing text; and then the images are fed into the subsequent unsupervised training module 40. The unsupervised training module 40 uses unsupervised training techniques to learn text information representations based on image sequence features and language sequence features from massive amounts of unlabeled data. The trained text recognition model can then be used in downstream text recognition scenarios.

[0159] The image acquisition module 20 first uses a search engine to collect a massive amount of images from the Internet, and then uses a text detection model to reason about these images. If the image contains text, it extracts the text image from the image based on the coordinates of the text box.

[0160] Figure 11 A block diagram illustrating the specific execution flow of the image acquisition module 20 is shown. For example... Figure 11 As shown, the image acquisition module 20 includes an internet data collection unit 21, a detection model inference unit 22, and a recognition matting unit 23. The specific details of each unit of the image acquisition module 20 are described below.

[0161] In the Internet data collection unit 21, a large amount of Internet image-text pair data is obtained through engine search. Since the data is automatically obtained from the Internet, it contains a lot of noise, and there are many similar images and privacy images. First, images with too low resolution, unbalanced aspect ratio, similar images, and privacy images are filtered. Then, images are filtered based on their corresponding text, considering language, meaningless text, and text length. Finally, valid Internet images are sent to the detection model inference unit 22 for the next step.

[0162] The detection model inference unit 22 uses a pre-trained text detection model to infer the meaning of the image, retaining images that may contain text and the position information of text boxes. The image matting unit 23, based on the text image and the position information of the text boxes, uses perspective transformation to extract small text images for constructing the original text recognition training set. Perspective transformation is a transformation that projects the original image onto a new view plane, changing the size and shape of the original image. Through perspective transformation, irregular text can be converted into regular text.

[0163] The image cleaning module 30 uses a text recognition model to filter out small images containing noisy text. Figure 12 This is a structural block diagram showing the image cleaning module 30. (See diagram for example.) Figure 12 As shown, the image cleaning module 30 includes an image recognition processing unit 31, an ATT recognition model inference unit 32, a CTC recognition model inference unit 33, a Paddle recognition model inference unit 34, and an image recognition filtering unit 35. The specific details of each unit in the image cleaning module 30 are described below:

[0164] The image processing unit 31 processes the collected internet recognition thumbnails (i.e., the original images) uniformly to facilitate the use of this data by subsequent modules. First, it determines whether the text is vertical or horizontal based on the aspect ratio of the image, and then lays the vertical text upside down to match the aspect ratio of the horizontal text. Then, it scales all the images to the same resolution, ensuring that the aspect ratio of the original image remains unchanged during the scaling process, and fills any insufficient areas with a black background.

[0165] Since the collected Internet recognition images contain a lot of noise, multiple different ATT recognition model inference units 32, CTC recognition model inference units 33, and Paddle recognition model inference units 34 are used to infer the same recognition image to prevent erroneous predictions caused by a single recognition model.

[0166] In the image filtering unit 35, the collected Internet recognition images are filtered based on the recognition results of the ATT recognition model inference unit 32, the CTC recognition model inference unit 33, and the Paddle recognition model inference unit 34. If the prediction results of the three recognition model inference units differ greatly, it indicates that the image contains noise. Therefore, images with noise need to be removed from the constructed dataset to reduce the noise in the image set.

[0167] The unsupervised training module 40 utilizes unsupervised training techniques to learn sequential text information representations from massive amounts of unlabeled data. Figure 13 The main structural block diagram of the unsupervised training module 40 is shown.

[0168] like Figure 13 As shown, the unsupervised training module 40 includes a mask image reconstruction unit 41, a mask image prediction unit 42, a mask text reconstruction unit 43, and a recognition fine-tuning unit 44. The specific details of each unit in the unsupervised training module 40 are described below:

[0169] In the masked image reconstruction unit 41, a portion of the pixels of the original image are masked, and then the masked pixels are reconstructed using the remaining pixels. This process requires reconstructing the entire text based on partial information, allowing the learning of individual character structural information, such as radicals and components. If the entire text is masked, it needs to be reconstructed based on information from the remaining characters, allowing the learning of semantic information from the text context. The detailed steps are as follows:

[0170] Step S411: Mask the original image according to a certain ratio to obtain the masked image. In actual training, an original image can be trained multiple times, and the mask area can be different each time. The masking can be performed according to the predetermined mask ratio.

[0171] Step S412: The masked image is sliced ​​into multiple first strip images along the text reading direction. The sliced ​​first strip images are then fed into the feature extraction module of the neural network model to extract the first image features of each first strip image. Since the first image features of each first strip image only contain partial information about the text, these first image features need to be aggregated. The extracted first image features are then fed into the feature aggregation module of the neural network model to obtain global image features. The extracted global image features contain contextual semantic information about the text. Finally, the aggregated global image features are fed into the pixel prediction module. The pixel prediction module predicts the RGB values ​​of the pixels in the masked region (i.e., pixel prediction values) through the fully connected layers in the neural network model. The pixel prediction values ​​are combined with the mask image to obtain the reconstructed image.

[0172] Step S413: Use the mean squared error loss function to measure the similarity between the original image and the reconstructed image, and then apply the backpropagation algorithm to update the parameters of the feature extraction module, feature aggregation module, and pixel prediction module of the neural network model. This process requires reconstructing the entire text based on partial information, allowing the learning of individual character structural information. The backpropagation algorithm (BP algorithm) is a supervised learning algorithm commonly used to train multilayer perceptrons.

[0173] In the masked image prediction unit 42, a neural network predictor is first randomly initialized, and then fixed. This neural network predictor can also be generated using a prediction neural network model. During the training phase, a portion of the pixels of the original image is masked, and then the neural network predictor is used for supervised training of the prediction results of the original image. This method learns the contextual semantic information of the text. The detailed steps are as follows:

[0174] Step S421: Mask the original image according to a certain ratio to obtain the masked image.

[0175] Step S422: Randomly initialize the neural network predictor, which is a multilayer perceptron, capable of generating prediction results based on the input image and using the prediction results as labels for subsequent training.

[0176] Step S423: Similar to the masked image reconstruction unit 41 described above, the masked image is cut into multiple first strip images along the text reading direction. The cut first strip images are fed into the feature extraction module of the neural network model to extract the first image features of each first strip image. Then, the global image features are obtained through the feature aggregation module. Since the pixel masked area does not provide feature information during the feature extraction process, the obtained global image features are the features of the unmasked strip images located in the unmasked pixel area. The global image features contain the contextual semantic features of the text information in the unmasked strip images. Then, the label prediction module is used to predict the category prediction label of the masked strip images based on the contextual semantic features of the text information in the unmasked strip images.

[0177] Step S424: Based on the original image and the randomly initialized neural network predictor, the original image is cut into multiple second strip images along the text reading direction. For each second strip image, a d-dimensional feature vector is generated offline. First, a codebook containing multiple different category labels is randomly generated (the codebook is k*d in size, meaning it contains the category labels corresponding to k d-dimensional feature vectors). Then, the predicted d-dimensional feature vector is used to find the category label with the highest similarity to the text information in the second strip image to which the feature vector belongs in the codebook. This category label with the highest similarity is used as the true category label for each second strip image. The next step is to update the parameters of the neural network model based on this true category label.

[0178] Step S425: Use mean squared error loss to measure the label similarity between the predicted category label and the true category label of each masked strip image. Then, use backpropagation algorithm to update the feature extraction module, feature aggregation module, and label prediction module. To complete this prediction task, the model needs to have the ability to perform high-dimensional feature prediction based on contextual information. This method can learn the contextual semantic information of text images, improve feature representation ability, and help improve downstream recognition performance.

[0179] In the masked text reconstruction unit 43, a multimodal mask pre-training task is introduced to further improve the performance of downstream text recognition. Partial characters in the corresponding pseudo-labels of the image are masked, and then the remaining characters and the original image information are used to recover the masked characters. This process requires reconstructing the original text based on the text image information and partial text information. Through multimodal pre-training, richer contextual semantic information can be learned. The detailed steps are as follows:

[0180] Step S431: Mask the original text according to a certain ratio to obtain the masked text.

[0181] Step S432: Using the feature extraction and feature aggregation modules of the neural network model learned by the masked image reconstruction unit 41 and the masked image prediction unit 42, the image sequence features of the original image are obtained. These image sequence features include contextual semantic features, which are used to describe the contextual semantic information of the text in the second strip image. Then, the BERT language model is used to extract the language sequence features of the masked text. These language sequence features contain the contextual semantic information of each character. The image sequence features and language sequence features are then fused through the feature aggregation module to obtain global character features that simultaneously contain both image sequence features and language sequence features. Finally, the aggregated global character features are fed into the text prediction module, which predicts the masked characters through a fully connected layer of the neural network to obtain the text prediction structure. The text prediction result is combined with the masked text to obtain the reconstructed text. The text prediction module, pixel prediction module, and label prediction module use different fully connected layers of the neural network to achieve their respective prediction tasks, improving the recognition performance of the entire text recognition model.

[0182] Step S433: Use the cross-entropy loss function to measure the similarity between the original text and the reconstructed text, and then use the backpropagation algorithm to update the parameters of the feature extraction module, feature aggregation module, and text prediction module. Since this process reconstructs the entire text based on the textual information of both the original image and the original text, it can learn both appearance and semantic features simultaneously, thus better assisting downstream text recognition tasks, such as Optical Character Recognition (OCR) tasks.

[0183] In the recognition fine-tuning unit 44, compared to previous recognition models that required a longer training time to achieve good recognition results, the unsupervised pre-trained text recognition model can achieve superior performance with a shorter training time on labeled images. This saves training time on downstream recognition tasks and also achieves better recognition performance.

[0184] This invention introduces an unsupervised method based on mask reconstruction into text recognition tasks. It leverages massive amounts of internet data to learn the structural information of individual characters and the contextual semantic information of the entire text string, thereby improving the robustness and accuracy of downstream text recognition tasks. By introducing an unsupervised method based on mask reconstruction, it is possible to efficiently learn text information representations based on sequence characteristics (image sequence features, language sequence features), and to learn more refined structural information of individual characters (such as radicals and components) as well as the contextual semantic information of the entire text string.

[0185] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. An unsupervised training method for a character recognition model, characterized in that, include: Obtain the original image, which is an image containing text; A mask image is obtained by performing masking processing on a portion of the pixels of the original image. The mask image includes masked areas and unmasked areas of pixels. The masked image is input into a neural network model. The neural network model uses the text information in the unmasked area of ​​the pixel to predict the text information in the masked area of ​​the pixel and obtain the prediction result. The text information is information with a partial structure of a single character, or the text information is information of the partial structure and several complete characters. The partial structure includes radicals and components. Based on the prediction results and the original image, the parameters of the neural network model are updated; The neural network model includes a feature aggregation module and a pixel prediction module. Before inputting the mask image into the neural network model, the method further includes: After the mask image is cut into multiple first strip images, the multiple first strip images are input into the neural network model. The first strip image includes a masked strip image belonging to the masked area of ​​the pixel and an unmasked strip image belonging to the unmasked area of ​​the pixel. The steps of using the neural network model to predict the text information in the masked region of a pixel based on the text information in the unmasked region of the pixel, and obtaining the prediction result, include: The feature aggregation module determines the global image features of the unmasked strip image and inputs the global image features into the pixel prediction module. The global image features include contextual semantic features, which are used to describe the contextual semantic information of the text information in the unmasked strip image. The pixel prediction module predicts the pixel prediction values ​​of the masked strip image based on the global features of the image, and updates the parameters of the neural network model based on the pixel prediction values ​​and the original image.

2. The method according to claim 1, characterized in that, The steps for updating the parameters of the neural network model based on the pixel prediction values ​​and the original image include: Based on the pixel prediction value, the pixels in the masked area of ​​the pixel in the mask image are reconstructed to obtain the reconstructed image; Determine the pixel similarity between the original image and the reconstructed image; The parameters of the neural network model are updated based on the pixel similarity.

3. The method according to claim 1, characterized in that, The neural network model further includes a label prediction module, wherein, before updating the parameters of the neural network model, the method further includes: The global image features of the unmasked strip image are input into the label prediction module, and the global image features include the contextual semantic features; The label prediction module predicts the category label of the text information in the masked strip image based on the contextual semantic features to obtain the category prediction label. The category label is a label used to mark the category to which the text information belongs after classifying the text information according to a preset classification rule. Based on the cutting position and number of cuts of the masked image, the original image is cut into multiple second strip images, and the category true label of the text information of each second strip image is determined. Then, each masked strip image corresponds to a category true label, so that the parameters of the neural network model are updated based on the category prediction label and the category true label of each masked strip image.

4. The method according to claim 3, characterized in that, The step of updating the parameters of the neural network model based on the predicted category label and the true category label of each of the masked strip images includes: Determine the label similarity between the predicted category label and the true category label for each of the masked strip images; The parameters of the neural network model are updated based on the degree of label similarity.

5. The method according to claim 3, characterized in that, The neural network model further includes a feature extraction module, wherein, before determining the global image features of the unmasked strip image through the feature aggregation module, the method further includes: The feature extraction module extracts a first image feature for each of the first strip images, wherein the first image feature is a feature containing the text information of the first strip image; The step of determining the global image features of the unmasked strip image through the feature aggregation module includes: The first image features are input into the feature aggregation module; The feature aggregation module aggregates the text information to determine the contextual semantic information of the text information in each of the first strip images, thereby obtaining the global image features of the unmasked strip image.

6. The method according to claim 5, characterized in that, The neural network model further includes a text prediction module. When performing masking processing on a portion of the pixels of the original image to obtain a masked image, the method further includes: The original text is generated based on the text in the original image, and a portion of the text in the original text is masked to obtain masked text. Before updating the parameters of the neural network model, the method further includes: The masked text is input into the neural network model. The text prediction module predicts the masked text based on the unmasked text in the masked text and obtains the text prediction result. Based on the original text and the text prediction result, the parameters of the neural network model are updated.

7. The method according to claim 6, characterized in that, Before the text prediction module predicts the masked text in the masked text based on the unmasked text in the masked text, the method further includes: The original image is cut into multiple second strip images and input into the neural network model. The feature aggregation module determines the image sequence features of the second strip images. The image sequence features include contextual semantic features, which are used to describe the contextual semantic information of the text information in the second strip images. The language sequence features of each unmasked character in the masked text are extracted using a predetermined language model. The language sequence features are features that contain the contextual semantic information of each character. The steps of using the text prediction module to predict the masked text in the masked text based on the unmasked text in the masked text, and obtaining the text prediction result, include: Obtain the image sequence features and the language sequence features; The feature aggregation module aggregates the image sequence features and the language sequence features to obtain the global character features of each character. Based on the global features of the characters, predict the masked characters in the masked text to obtain the character prediction result.

8. The method according to claim 7, characterized in that, Before determining the image sequence features of the second strip image through the feature aggregation module, the method further includes: The feature extraction module extracts a second image feature for each of the second strip images, the second image feature being a feature containing the text information of the second strip image; The step of determining the image sequence features of the second strip image through the feature aggregation module includes: The second image features are input into the feature aggregation module; The feature aggregation module aggregates the text information of each second strip image to determine the contextual semantic information of the text information in each second strip image, thereby obtaining the image sequence features of the unmasked strip image.

9. The method according to claim 6, characterized in that, The steps for updating the parameters of the neural network model based on the original text and the text prediction results include: Based on the text prediction results, the masked text in the masked text is reconstructed to obtain the reconstructed text; Determine the degree of text similarity between the original text and the reconstructed text; The parameters of the neural network model are updated based on the text similarity.

10. The method according to claim 6, characterized in that, The method further includes: Obtain labeled images containing text that have been labeled with labeled data; The parameters of the neural network model are adjusted based on the labeled image.

11. An unsupervised character recognition method, characterized in that, include: The system receives a text image to be recognized and uses a text recognition model to recognize the text in the text image, wherein the text recognition model is trained by the unsupervised training method of the text recognition model according to any one of claims 1 to 10.

12. An unsupervised training device for a character recognition model, characterized in that, The unsupervised training device for the character recognition model is used to implement the unsupervised training method for the character recognition model according to any one of claims 1 to 10, and the device includes: The acquisition module (11) is used to acquire the original image, wherein the original image is an image containing text; The mask processing module (12) is used to perform mask processing on some pixels of the original image to obtain a mask image, wherein the mask image includes a pixel masked area and a pixel unmasked area; The prediction module (13) is used to input the mask image into the neural network model, and use the neural network model to predict the text information of the masked area of ​​the pixel based on the text information of the unmasked area of ​​the pixel, and obtain the prediction result. The text information is information with a partial structure of a single character, or the text information is information of the partial structure and several complete characters. The partial structure includes radicals and components. The parameter update module (14) is used to update the parameters of the neural network model based on the prediction result and the original image.

13. A computer device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to perform an unsupervised training method for the character recognition model according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform an unsupervised training method for the character recognition model according to any one of claims 1 to 10.

Images