Text recognition model training method, usage method, device, equipment and medium

Abstract

The present disclosure provides a training method, usage method, device, equipment and medium for a text recognition model. It relates to technical fields such as image processing, text recognition, and artificial intelligence. The specific implementation scheme is: obtaining a training data group, wherein the training data group includes training text images and corresponding annotated text information; obtaining a mask probability, wherein the mask probability is used to identify the probability of randomly masking the text encoding features of the annotated text information; training a text recognition model based on the mask probability and the training text images and the annotated text information in the training data group; the text recognition model includes an image encoder, a text encoder and a decoder. The technology disclosed in the present disclosure can effectively improve the accuracy of the trained text recognition model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to technical fields such as image processing, text recognition, and artificial intelligence, and more particularly to a training method, use method, device, equipment, and medium for a text recognition model. Background Art

[0002] Artificial intelligence has a wide range of applications in the field of imaging. Optical character recognition (OCR) is a key branch of artificial intelligence in the imaging field. In OCR, the task of recognizing text character slices is called scene text recognition (STR).

[0003] In existing STR, text information in text images can be recognized by training a text recognition model. Specifically, the text recognition model includes an encoder and a decoder. The encoder is used to extract information from the text image, and the decoder is used to decode the extracted text image information to obtain the text information contained therein. Summary of the Invention

[0004] The present disclosure provides a training method, use method, apparatus, device, and medium for a text recognition model.

[0005] According to one aspect of the present disclosure, a method for training a text recognition model is provided, comprising:

[0006] Obtaining a training data set, wherein the training data set includes training text images and corresponding annotated text information;

[0007] Obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0008] A text recognition model is trained based on the mask probability, the training text images in the training data set, and the annotated text information; the text recognition model includes an image encoder, a text encoder, and a decoder.

[0009] According to another aspect of the present disclosure, a text recognition method is provided, comprising:

[0010] Get the text image to be recognized;

[0011] Obtaining a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0012] Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image. According to another aspect of the present disclosure, a training device for a text recognition model is provided, comprising:

[0013] A data acquisition module is used to acquire a training data set, wherein the training data set includes training text images and corresponding annotated text information;

[0014] A mask acquisition module, configured to acquire a mask probability, wherein the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0015] A training module is used to train a text recognition model based on the mask probability and the training text pictures and the annotated text information in the training data group; the text recognition model includes an image encoder, a text encoder and a decoder.

[0016] According to another aspect of the present disclosure, a text recognition device is provided, comprising:

[0017] Image acquisition module, used to obtain text images to be recognized;

[0018] A mask acquisition module is used to acquire a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0019] The recognition module is used to perform text recognition on the text image based on the text mask feature using a pre-trained text recognition model.

[0020] According to yet another aspect of the present disclosure, there is provided an electronic device, including:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0024] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0025] According to yet another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method of the aspect and any possible implementation manner described above.

[0026] According to the technology disclosed in the present invention, the accuracy of the trained text recognition model can be effectively improved.

[0027] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0029] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0030] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0031] Figure 3 This is a diagram showing the training principle of the text recognition model of this embodiment;

[0032] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0033] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0034] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0035] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0036] Figure 8 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0037] Figure 9 is a schematic diagram according to an eighth embodiment of the present disclosure;

[0038] Figure 10 is a block diagram of an electronic device for implementing the method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0041] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0042] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the related objects are in an "or" relationship.

[0043] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure; Figure 1 As shown, this embodiment provides a text recognition model training method, which may specifically include the following steps:

[0044] S101, obtaining a training data set, wherein the training data set includes training text images and corresponding annotated text information;

[0045] The text recognition model training method of this embodiment is performed by a text recognition model training device, which may be an electronic entity or a software-integrated application for training the text recognition model.

[0046] The text image in this embodiment refers to an image containing text information.

[0047] During the training process, a training data set may include one or more pieces of training data. To improve the training effect, preferably, multiple pieces of training data are included. Each piece of training data may include a training text image and corresponding annotated text information. The text recognition model trained in this embodiment is used to recognize text information in text images, so the annotated text information refers to the text information included in the training text image.

[0048] The training text images in this embodiment may include printed text, handwritten text, artistic text, formulas, and the like.

[0049] S102: Obtain a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding features of the annotated text information;

[0050] S103: Training a text recognition model based on the mask probability and the training text images and annotated text information in the training data set.

[0051] The text recognition model of this embodiment includes an image encoder, a text encoder, and a decoder.

[0052] In this embodiment, the mask probability and the training text images and annotated text information in the training data set participate in the training of the text recognition model together.

[0053] The masking probability in this embodiment is used to randomly mask the text encoding features of the annotated text information during training. This allows the text recognition model to mask the text encoding features during training, providing information support for the text recognition model during decoding, enabling the text recognition model to effectively identify text information in text images. This training method can effectively improve the accuracy of the text recognition model.

[0054] The training method of the text recognition model of this embodiment introduces mask probability to train the text recognition model together with the training text images and annotated text information in the training data group; based on the mask probability, the text recognition model can be effectively trained using the text encoding features of the masked annotated text information during the training process, which can effectively improve the accuracy of the text recognition model in recognizing text images.

[0055] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure; Figure 2 As shown, this embodiment provides a training method for a text recognition model. Figure 1 Based on the technical solutions of the embodiments shown, the technical solutions of the present disclosure are further described in more detail. Figure 2 As shown, the training method of the text recognition model of this embodiment may specifically include the following steps:

[0056] S201, obtaining a training data set, wherein the training data set includes training text images and corresponding annotated text information;

[0057] S202: Obtain a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding features of the annotated text information;

[0058] S203, using the image encoder in the text recognition model to encode the training text image to obtain a first training encoding feature of the training text image;

[0059] Optionally, before step S203 , normalization preprocessing may be performed on the text image to enhance the accuracy of text recognition model training.

[0060] S204, using a text encoder in a text recognition model to encode the annotated text information to obtain a second training encoding feature of the annotated text information;

[0061] S205. Randomly mask the second training coding feature based on the mask probability to obtain a training mask feature;

[0062] S206. Decoding the first training coding feature and the training mask feature using a decoder in the text recognition model to obtain predicted text information;

[0063] S207 : Based on the annotated text information and the predicted text information, adjust the parameters of the image encoder, text encoder, and decoder in the text recognition model.

[0064] For example, the specific implementation of step S207 may include the following steps:

[0065] (1) Obtain the value of the loss function based on the labeled text information and the predicted text information;

[0066] Specifically, the cross entropy between the encoding features of the labeled text information and the encoding features of the predicted text information can be used as the value of the loss function. The encoding features of the labeled text information can refer to the second encoding features obtained by encoding the text encoder. The encoding features of the predicted text information can also be obtained by encoding the predicted text information using the text encoder.

[0067] It should be noted that if the training data set includes only one training data, the cross entropy value of the encoding feature of the labeled text information corresponding to the training data and the encoding feature of the predicted text information can be obtained as the value of the loss function in the above manner. If the training data set includes multiple training data, the average value of the loss function corresponding to the multiple training data can be obtained as the final loss function value, which is recorded as Loss c.

[0068] (2) Based on the value of the loss function Loss c , adjust the parameters of the text encoder, image encoder and decoder in the text recognition model.

[0069] The specific adjustment goal is to move towards the value of the loss function Loss c Adjust in the direction of decreasing.

[0070] Further optionally, in one embodiment of the present disclosure, after step (1), the following steps may also be included:

[0071] (3) Detect the value of the loss function Loss c Is it greater than the preset hyperparameter α? If so, execute step (4); otherwise, if the value of the loss function Loss c If it is less than or equal to the preset hyperparameter α, execute step (5);

[0072] The preset hyperparameter α in this embodiment is a preset value used to balance the value of the loss function Loss c and mask probability.

[0073] (4) Loss based on the value of the loss function c And the preset hyperparameter α, adjust the mask probability, and end.

[0074] (5) Adjust the mask probability to 1 and end.

[0075] For example, the value of the loss function can be based on Loss c And the preset hyperparameter α, use the following formula to adjust the mask probability:

[0076] P=1-(Loss c -α)

[0077] Among them, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

[0078] In this embodiment, the mask probability is adjusted to 1, indicating that during the training process at this time, the value of the loss function is small, and the accuracy of the image encoder and decoder in the text recognition model is already high. At this time, all the text encoding features encoded by the text encoder can be masked, so that the image encoder and decoder can be fully trained more accurately.

[0079] During the training process of this embodiment, the mask probability can be adaptively adjusted, thereby effectively improving the accuracy of the trained text recognition model.

[0080] Steps S203-S207 are as above. Figure 1A specific implementation of step S103 of the illustrated embodiment.

[0081] Optionally, the acquisition of the mask probability in step S202 of this embodiment includes two opportunities:

[0082] During initial training, the mask probability is configured as a random number less than the preset probability threshold, such as 0, which means that the encoding features of the annotated text information do not need to be masked at this time, allowing the decoder in the text recognition model to fully refer to the encoding features of the annotated text information for decoding, so as to improve the accuracy of the decoded and recognized text information.

[0083] During non-initial training, the mask probability adjusted in the previous training step is obtained. For example, the value of the mask probability obtained in the above step (4) during the previous training step is specifically obtained.

[0084] In other words, during training, the mask probability value needs to be updated in real time based on the model's training results. At the beginning of training, the model has not converged and the loss function value is large. At this time, a smaller mask probability can be configured so that the text recognition model's decoder can refer to the text encoding features for decoding and obtain accurate text recognition results. As training progresses, the image encoder and decoder in the text recognition model reach a certain level of accuracy. At this time, the mask probability can be gradually increased, so that the text recognition model's decoder gradually becomes independent of the text encoding features and can also accurately decode.

[0085] In this embodiment, the image encoder can be implemented using densely connected convolutional networks (DenseNet), or can also use residual networks (ResNet), feature pyramid networks (FPNet), or vision transformer networks (ViTNet).

[0086] Since the image encoder uses DenseNet instead of the traditional ViT-encoder, it can maintain the same or even better results while using fewer parameters.

[0087] In this embodiment, the text encoder adopts a transformer network, such as the Bidirectional Encoder Representations from Transformers (BERT) model and various improved models based on BERT.

[0088] In this embodiment, the decoder can be a transform network based on a multi-head self-attention mechanism. Specifically, a 256-layer embedding matrix dimension, an 8-head attention mechanism, a 1024-bit feedforward neural network matrix dimension, and a dropout ratio of 0.3 can be used.

[0089] Figure 3 This is a diagram showing the training principle of the text recognition model of this embodiment. Figure 3 As shown, it can be seen that in the training of the text recognition model of this embodiment, due to the large loss in the early stage, the text encoding features can be controlled to be rarely covered by the mask, that is, the downstream decoder will obtain the complete annotated text content. At this time, the decoder can quickly learn the correct output from the text encoding features. In the medium term, as the accuracy of the model increases, the loss function gradually decreases, and the text encoding features are gradually covered by the mask. At this time, the decoder part cannot obtain the content covered by the mask from the text side, and can only seek supplementation from the image side information, that is, the model weights are gradually transferred from the text side to the image side. In the later stage, when the loss function is less than the preset hyperparameters, the text side information is no longer provided to the decoder. At this time, the training of the text recognition model returns to the normal training of the image encoder and image decoder until the text recognition model is fully converged. In this way, when applying reasoning, text recognition can be performed entirely based on the information on the image side.

[0090] In the training of the text recognition model of this embodiment, masked text encoding features are used to assist the text recognition model in training, so that the text recognition model can converge faster and the accuracy of the trained text recognition model can be effectively improved.

[0091] In the training of the text recognition model of this embodiment, an adaptive masking strategy is adopted, which can effectively reduce the number of hyperparameters and thus effectively reduce the difficulty of model tuning.

[0092] Experimental verification shows that the text recognition model trained by the training method of this embodiment can effectively process text in complex images and is not limited to text images in a single direction, thereby coping with diverse and complex text scenarios.

[0093] Moreover, by introducing text features during the training process, the text recognition model's ability to extract images can be effectively enhanced, thereby effectively improving the generalization ability of the text recognition model in different application scenarios.

[0094] The text recognition model of this embodiment avoids the use of a large language model, reducing the need for complex data and high-cost training.

[0095] The text recognition model trained in this embodiment can be applied in the field of text recognition in complex images, and can effectively improve the recognition accuracy. For example, it can be applied to formula recognition and signature recognition.

[0096] Figure 4 is a schematic diagram according to the third embodiment of the present disclosure; Figure 4 As shown, this embodiment provides a text recognition method, which may specifically include the following steps:

[0097] S401, obtaining a text image to be recognized;

[0098] S402: Obtain a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0099] S403: Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image.

[0100] The text recognition method of this embodiment is performed by a text recognition device, which can be an electronic entity, an intelligent entity, or a software-integrated application. When used, a text image is input, where the text image is a picture containing text information, and the text recognition device can recognize the text information in the text image.

[0101] The text recognition method of this embodiment is to Figure 1-Figure 3 Application of the text recognition model trained by the illustrated embodiment.

[0102] In order to align features with those in the training phase, the text recognition model of this embodiment needs to be pre-configured with a text mask feature containing blank text information when it is applied, which means that all texts are masked.

[0103] The text recognition method of this embodiment refers to the text mask features to align the inference process of the text recognition model with the training process features, and thus the text recognition model can be used to accurately and efficiently perform text recognition on text images.

[0104] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure; Figure 5 As shown, this embodiment provides a text recognition method. Figure 4 Based on the technical solutions of the embodiments shown, the technical solutions of the present disclosure are further described in more detail. Figure 5 As shown, the text recognition method of this embodiment may specifically include the following steps:

[0105] S501, obtaining a text image to be recognized;

[0106] S502: Obtain a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0107] For example, the text mask feature of this embodiment can be represented by a matrix whose elements are all zero. During the file recognition process, this feature is only used to align with the features in the training process to effectively ensure the accuracy of text recognition and has no other significance.

[0108] S503, using the image encoder in the text recognition model to obtain encoding features of the text image;

[0109] S504: Concatenate the encoding features of the text image and the pre-configured text mask features to obtain concatenated features;

[0110] S505: Using the image decoder in the text recognition model, decode the splicing features to obtain text information of the text image.

[0111] Steps S503-S505 of this embodiment are as described above. Figure 4 A specific implementation method of S403 of the illustrated embodiment, through which the encoded features of the text image and the preconfigured text mask features are spliced, and then the spliced ​​features are decoded by the image decoder, so that the features in the reasoning process and the features in the training process can be aligned, effectively ensuring the recognition accuracy of the text recognition model.

[0112] The text recognition model of this embodiment is the above Figure 1-Figure 3 The text recognition model trained in the illustrated embodiment can be further described in detail with reference to the description of the above-mentioned related embodiments, which will not be repeated here.

[0113] The text recognition method of this embodiment can effectively improve the accuracy and efficiency of text recognition by adopting the text recognition model trained in the above embodiment.

[0114] Figure 6 is a schematic diagram according to the fifth embodiment of the present disclosure; Figure 5 As shown, this embodiment provides a text recognition model training device 600, including:

[0115] The data acquisition module 601 is used to acquire a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0116] A mask acquisition module 602 is used to acquire a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0117] The training module 603 is used to train a text recognition model based on the mask probability and the training text images and the annotated text information in the training data group; the text recognition model includes an image encoder, a text encoder and a decoder.

[0118] The text recognition model training device 600 of this embodiment realizes the implementation principle and technical effect of text recognition model training by adopting the above-mentioned modules, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0119] Figure 7 is a schematic diagram according to the sixth embodiment of the present disclosure; Figure 7 As shown, this embodiment provides a text recognition model training device 700. Figure 6 Based on the embodiment shown, the technical solution of the present disclosure is further described in more detail. Figure 7 As shown, the text recognition model training device 700 of this embodiment includes the above Figure 6 Modules with the same name and function are shown as follows: data acquisition module 701 , mask acquisition module 702 and training module 703 .

[0120] like Figure 7 As shown, in the text recognition model training device 700 of this embodiment, the training module 703 includes:

[0121] The picture encoding unit 7031 is used to encode the training text picture using the picture encoder in the text recognition model to obtain a first training encoding feature of the training text picture;

[0122] A text encoding unit 7032 is configured to encode the annotated text information using a text encoder in the text recognition model to obtain a second training encoding feature of the annotated text information;

[0123] a masking unit 7033, configured to randomly mask the second training coding feature based on the masking probability to obtain a training masked feature;

[0124] A decoding unit 7034 is configured to decode the first training encoding feature and the training mask feature using a decoder in the text recognition model to obtain predicted text information;

[0125] The parameter adjustment unit 7035 is used to adjust the parameters of the image encoder, text encoder and decoder in the text recognition model based on the annotated text information and the predicted text information.

[0126] Further optionally, in one embodiment of the present disclosure, the parameter adjustment unit 7035 is configured to:

[0127] Obtaining a value of a loss function based on the annotated text information and the predicted text information;

[0128] Based on the value of the loss function, parameters of the text encoder, image encoder and decoder in the text recognition model are adjusted.

[0129] Further optionally, as Figure 7 As shown, in one embodiment of the present disclosure, the text recognition model training device 700 further includes:

[0130] The mask adjustment module 704 is configured to adjust the mask probability based on the value of the loss function and preset hyperparameters.

[0131] Further optionally, in one embodiment of the present disclosure, the mask adjustment module 704 is configured to:

[0132] Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula:

[0133] P=1-(Loss c -α)

[0134] Wherein, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

[0135] Further optionally, as Figure 7 As shown, in one embodiment of the present disclosure, the training module 703 further includes:

[0136] The determining unit 7036 is used to determine whether the value of the loss function is greater than the preset hyperparameter.

[0137] Further optionally, in one embodiment of the present disclosure, the mask adjustment module 704 is configured to:

[0138] If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

[0139] Further optionally, in one embodiment of the present disclosure, the mask acquisition module 702 is configured to: during initial training, configure the mask probability to be a random number that is less than a preset probability threshold.

[0140] The text recognition model training device 700 of this embodiment realizes the implementation principle and technical effect of text recognition model training by adopting the above-mentioned modules, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0141] Figure 8 is a schematic diagram according to the seventh embodiment of the present disclosure; Figure 8 As shown, this embodiment provides a text recognition device 800, including:

[0142] Image acquisition module 801, used to acquire text images to be recognized;

[0143] The mask acquisition module 802 is used to acquire a pre-configured text mask feature, where the text mask feature is a pre-configured feature containing blank text information;

[0144] The recognition module 803 is configured to perform text recognition on the text image based on the text mask feature and using a pre-trained text recognition model.

[0145] The text recognition device 800 of this embodiment realizes text recognition by adopting the above modules, and its implementation principle and technical effect are the same as those of the above-mentioned related method embodiments. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0146] Figure 9 is a schematic diagram according to the eighth embodiment of the present disclosure; Figure 9 As shown, this embodiment provides a text recognition device 900. Figure 8 Based on the technical solutions of the embodiments shown, the technical solutions of the present disclosure are further described in more detail. Figure 9 As shown, this embodiment provides a text recognition device 900 including the above Figure 8 Modules with the same name and function are shown as follows: image acquisition module 901 , mask acquisition module 902 , and recognition module 903 .

[0147] like Figure 9 As shown, the identification module 903 includes:

[0148] The text encoding unit 9031 is used to obtain encoding features of the text image using the image encoder in the text recognition model;

[0149] The user splices the encoding feature of the text image and the pre-configured text mask feature to obtain a spliced ​​feature in the splicing unit 9032;

[0150] The decoding unit 9033 is configured to decode the splicing features using the image decoder in the text recognition model to obtain text information of the text image.

[0151] Further optionally, in one embodiment of the present disclosure, the text mask feature is represented by a matrix whose elements are all zero.

[0152] The text recognition device 900 of this embodiment realizes text recognition by adopting the above modules, and its implementation principle and technical effect are the same as those of the above-mentioned related method embodiments. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0153] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0154] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0155] Figure 10 A schematic block diagram of an example electronic device 1000 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0156] like Figure 10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. Various programs and data required for the operation of the device 1000 can also be stored in the RAM 1003. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0157] Various components in device 1000 are connected to I / O interface 1005, including an input unit 1006, such as a keyboard, mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a magnetic disk, optical disk, etc.; and a communication unit 1009, such as a network card, modem, wireless communication transceiver, etc. The communication unit 1009 allows device 1000 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0158] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the above-mentioned methods of the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the above method of the present disclosure in any other appropriate manner (for example, by means of firmware).

[0159] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0161] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0163] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0164] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0165] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0166] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A text recognition model training method, comprising: Obtaining a training data set, wherein the training data set includes training text images and corresponding annotated text information; Obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information; Using the image encoder in the text recognition model to encode the training text image to obtain a first training encoding feature of the training text image; Using a text encoder in the text recognition model to encode the annotated text information to obtain a second training encoding feature of the annotated text information; Randomly masking the second training coding feature based on the mask probability to obtain a training mask feature; Based on the first training encoding feature and the training mask feature, decoding is performed using a decoder in the text recognition model to obtain predicted text information; Obtaining a value of a loss function based on the annotated text information and the predicted text information; Adjusting parameters of a text encoder, an image encoder, and a decoder in the text recognition model based on the value of the loss function; The mask probability is adjusted based on the value of the loss function and preset hyperparameters.

2. The method according to claim 1, wherein Adjusting the mask probability based on the value of the loss function and preset hyperparameters includes: Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula: in, represents the mask probability; represents the value of the loss function; are the preset hyperparameters.

3. The method according to claim 1, wherein Before adjusting the mask probability based on the value of the loss function and the preset hyperparameters, the method further includes: Determine whether the value of the loss function is greater than the preset hyperparameter.

4. The method according to claim 1, wherein Adjusting the mask probability based on the value of the loss function and preset hyperparameters further includes: If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

5. The method according to any one of claims 1 to 4, wherein: Obtaining the mask probability includes: during initial training, configuring the mask probability to be a random number that is less than a preset probability threshold.

6. A text recognition method comprising: Get the text image to be recognized; Obtaining a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information; Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image; the text recognition model is a text recognition model trained using the method described in any one of claims 1 to 5 above.

7. The method according to claim 6, wherein: Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image, including: Using the image encoder in the text recognition model to obtain encoding features of the text image; Splicing the encoding feature of the text image and the preconfigured text mask feature to obtain a splicing feature; The image decoder in the text recognition model is used to decode the splicing features to obtain text information of the text image.

8. The method according to claim 6 or 7, wherein: The text mask feature is represented by a matrix whose elements are all zero.

9. A training device for a text recognition model, comprising: A data acquisition module is used to acquire a training data set, wherein the training data set includes training text images and corresponding annotated text information; A mask acquisition module, configured to acquire a mask probability, wherein the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information; Training modules include: A picture encoding unit, configured to encode the training text picture using the picture encoder in the text recognition model to obtain a first training encoding feature of the training text picture; A text encoding unit, configured to encode the annotated text information using a text encoder in the text recognition model to obtain a second training encoding feature of the annotated text information; a masking unit, configured to randomly mask the second training coding feature based on the masking probability to obtain a training masked feature; a decoding unit, configured to decode the first training encoding feature and the training mask feature using a decoder in the text recognition model to obtain predicted text information; Parameter adjustment unit, used for: Obtaining a value of a loss function based on the annotated text information and the predicted text information; Adjusting parameters of a text encoder, an image encoder, and a decoder in the text recognition model based on the value of the loss function; A mask adjustment module is used to adjust the mask probability based on the value of the loss function and preset hyperparameters.

10. The device according to claim 9, wherein The mask adjustment module is used to: Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula: in, represents the mask probability; represents the value of the loss function; are the preset hyperparameters.

11. The device according to claim 9, wherein The training module further includes: A determining unit is used to determine whether the value of the loss function is greater than the preset hyperparameter.

12. The device according to claim 9, wherein The mask adjustment module is used to: If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

13. The device according to any one of claims 9 to 12, wherein: The mask acquisition module is used to: during initial training, configure the mask probability to be a random number that is less than a preset probability threshold.

14. A text recognition device comprising: Image acquisition module, used to obtain text images to be recognized; A mask acquisition module is used to acquire a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information; A recognition module is used to perform text recognition on the text image based on the text mask feature using a pre-trained text recognition model; the text recognition model is a text recognition model trained using the device described in any one of claims 9 to 13 above.

15. The device according to claim 14, wherein The identification module includes: A text encoding unit, configured to obtain encoding features of the text image using the image encoder in the text recognition model; A splicing unit, in which a user splices the encoding feature of the text image and the preconfigured text mask feature to obtain a splicing feature; A decoding unit is used to decode the splicing features using the image decoder in the text recognition model to obtain text information of the text image.

16. The device according to claim 14 or 15, wherein The text mask feature is represented by a matrix whose elements are all zero.

17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5 or 6 to 8.

18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5 or 6-8.

19. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5 or 6 to 8.

Images