Text recognition model training method and device, text recognition model using method and device, equipment and medium

Abstract

The invention provides a text recognition model training method and device, a text recognition model using method and device, equipment and a medium. The method relates to the technical fields of image processing, text recognition, artificial intelligence and the like. According to the specific implementation scheme, a training data set is obtained, wherein the training data set comprises training text pictures and corresponding annotation text information; obtaining a mask probability, wherein the mask probability is used for identifying the probability of randomly masking the text coding feature of the annotated text information; training a text recognition model based on the mask probability, the training text picture in the training data set and the annotated text information; the text recognition model comprises a picture encoder, a text encoder and a decoder. According to the technology disclosed by the invention, the accuracy of the text recognition model obtained by training can be effectively improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to technical fields such as image processing, text recognition, and artificial intelligence, and more particularly to a training method, a use method, a device, equipment, and a medium for a text recognition model. Background Art

[0002] Artificial intelligence has a wide range of application scenarios in the field of images. Optical Character Recognition (OCR) is an important branch of artificial intelligence in the field of images. In OCR, the task of recognizing text character slices is called scene text recognition (STR).

[0003] In the existing STR, the text information in the text image can be recognized by training the text recognition model. Specifically, the text recognition model includes an encoder and a decoder, the encoder is used to extract the information of the text image, and the decoder is used to decode based on the extracted information of the text image to obtain the text information included therein. Summary of the invention

[0004] The present invention provides a training method, a use method, an apparatus, a device and a medium for a text recognition model.

[0005] According to one aspect of the present disclosure, a method for training a text recognition model is provided, comprising:

[0006] Obtaining a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0007] Obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0008] Based on the mask probability, the training text image in the training data set, and the annotated text information, a text recognition model is trained; the text recognition model includes a picture encoder, a text encoder, and a decoder.

[0009] According to another aspect of the present disclosure, there is provided a text recognition method, comprising:

[0010] Get the text image to be recognized;

[0011] Obtaining a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0012] Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image. According to another aspect of the present disclosure, a training device for a text recognition model is provided, comprising:

[0013] A data acquisition module, used to acquire a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0014] A mask acquisition module, used to acquire a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0015] A training module is used to train a text recognition model based on the mask probability and the training text pictures and the annotated text information in the training data group; the text recognition model includes a picture encoder, a text encoder and a decoder.

[0016] According to another aspect of the present disclosure, there is provided a text recognition device, comprising:

[0017] An image acquisition module is used to acquire text images to be recognized;

[0018] A mask acquisition module, used to acquire a preconfigured text mask feature, wherein the text mask feature is a preconfigured feature containing blank text information;

[0019] The recognition module is used to perform text recognition on the text image based on the text mask feature using a pre-trained text recognition model.

[0020] According to yet another aspect of the present disclosure, there is provided an electronic device, including:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0024] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0025] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and the method of any possible implementation manner.

[0026] According to the technology disclosed in the present invention, the accuracy of the trained text recognition model can be effectively improved.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0029] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0030] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0031] Figure 3 is a training principle diagram of the text recognition model of this embodiment;

[0032] Figure 4 is a schematic diagram according to a third embodiment of the present disclosure;

[0033] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0034] Figure 6 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0035] Figure 7 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0036] Figure 8 is a schematic diagram according to a seventh embodiment of the present disclosure;

[0037] Fig. 9 is a schematic diagram according to an eighth embodiment of the present disclosure;

[0038] Fig.10 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0039] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0040] Obviously, the described embodiments are only part of the embodiments of the present disclosure, but not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure.

[0041] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0042] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0043] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure; Figure 1 As shown, this embodiment provides a text recognition model training method, which may specifically include the following steps:

[0044] S101, obtaining a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0045] The execution subject of the text recognition model training method of this embodiment is a text recognition model training device, which can be an electronic entity or a software integrated application for training the text recognition model.

[0046] The text image in this embodiment refers to an image containing text information.

[0047] During the training process, a training data group may include one or more training data. In order to improve the training effect, preferably, multiple training data are included. Each training data may include a training text image and corresponding annotated text information. The text recognition model trained in this embodiment is used to recognize text information in a text image, so the annotated text information refers to the text information included in the training text image.

[0048] The training text image in this embodiment may include printed text, handwritten text, artistic text, formulas, and the like.

[0049] S102, obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding features of the annotated text information;

[0050] S103: Training a text recognition model based on the mask probability and the training text images and annotated text information in the training data set.

[0051] The text recognition model of this embodiment includes an image encoder, a text encoder and a decoder.

[0052] In this embodiment, the mask probability and the training text images and annotated text information in the training data set participate in the training of the text recognition model together.

[0053] The mask probability of this embodiment is used to randomly mask the text encoding features of the annotated text information during the training process, so that the text recognition model can mask the text encoding features during the training process to provide information support for the text recognition model when decoding, so that the text recognition model can effectively recognize the text information in the text image. Through this training method, the accuracy of the text recognition model can be effectively improved.

[0054] The training method of the text recognition model of the present embodiment introduces mask probability to train the text recognition model together with the training text images and annotated text information in the training data group; based on the mask probability, the text recognition model can be effectively trained using the text encoding features of the masked annotated text information during the training process, which can effectively improve the accuracy of the text recognition model in recognizing text images.

[0055] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure; Figure 2 As shown, this embodiment provides a method for training a text recognition model. Figure 1 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 2 As shown, the training method of the text recognition model of this embodiment may specifically include the following steps:

[0056] S201, obtaining a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0057] S202, obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding features of the annotated text information;

[0058] S203, using the picture encoder in the text recognition model to encode the training text picture to obtain a first training encoding feature of the training text picture;

[0059] Optionally, before step S203, normalization preprocessing may be performed on the text image to enhance the accuracy of text recognition model training.

[0060] S204, using a text encoder in a text recognition model to encode the annotated text information to obtain a second training encoding feature of the annotated text information;

[0061] S205, randomly masking the second training coding feature based on the mask probability to obtain a training mask feature;

[0062] S206, decoding using a decoder in a text recognition model based on the first training coding feature and the training mask feature to obtain predicted text information;

[0063] S207: Based on the annotated text information and the predicted text information, adjust parameters of the image encoder, text encoder, and decoder in the text recognition model.

[0064] For example, when step S207 is specifically implemented, it may include the following steps:

[0065] (1) Obtaining the value of the loss function based on the annotated text information and the predicted text information;

[0066] Specifically, the cross entropy value of the coding feature of the annotated text information and the coding feature of the predicted text information can be taken as the value of the loss function. The coding feature of the annotated text information can refer to the second coding feature encoded by the text encoder. The coding feature of the predicted text information can also be obtained by encoding the predicted text information using the text encoder.

[0067] It should be noted that if the training data set includes only one training data, the cross entropy value of the encoding feature of the annotated text information corresponding to the training data and the encoding feature of the predicted text information can be obtained in the above manner as the value of the loss function. If the training data set includes multiple training data, the average value of the loss function corresponding to the multiple training data can be obtained as the final loss function value, recorded as Loss c.

[0068] (2) Based on the value of the loss function Loss c , adjust the parameters of the text encoder, image encoder and decoder in the text recognition model.

[0069] The specific adjustment goal is to move towards the value of the loss function Loss c Adjust in the direction of decreasing.

[0070] Further optionally, in an embodiment of the present disclosure, after step (1), the following steps may also be included:

[0071] (3) Detect the value of the loss function Loss c Is it greater than the preset hyperparameter α? If so, execute step (4); otherwise, if the value of the loss function Loss c is less than or equal to the preset hyperparameter α, execute step (5);

[0072] The preset hyperparameter α in this embodiment is a preset value used to balance the value of the loss function Loss c and mask probability.

[0073] (4) Based on the value of the loss function Loss c And the preset hyperparameter α, adjust the mask probability, and end.

[0074] (5) Adjust the mask probability to 1 and end.

[0075] For example, we can use the loss function to calculate the value of Loss c And the preset hyperparameter α, use the following formula to adjust the mask probability:

[0076] P=1-(Loss c -α)

[0077] Among them, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

[0078] In this embodiment, the mask probability is adjusted to 1, which means that during the training process, the value of the loss function is small, and the accuracy of the image encoder and decoder in the text recognition model is already high. At this time, all the text encoding features encoded by the text encoder can be masked to allow the image encoder and decoder to be fully trained more accurately.

[0079] During the training process of this embodiment, the mask probability can be adaptively adjusted, thereby effectively improving the accuracy of the trained text recognition model.

[0080] Steps S203-S207 are as above. Figure 1A specific implementation of step S103 of the illustrated embodiment.

[0081] Optionally, the acquisition of the mask probability in step S202 of this embodiment includes two occasions:

[0082] During the initial training, the mask probability is configured to be a random number less than the preset probability threshold, such as 0, which means that the encoding features of the annotated text information do not need to be masked at this time, so that the decoder in the text recognition model can fully refer to the encoding features of the annotated text information for decoding, so as to improve the accuracy of the decoded and recognized text information.

[0083] During non-initial training, the mask probability adjusted in the previous training step is obtained. For example, the value of the mask probability obtained in the above step (4) during the previous training step is specifically obtained.

[0084] That is to say, during the training process, the value of the mask probability needs to be updated in real time according to the training effect of the model. At the beginning of training, the model has not converged and the value of the loss function is large. At this time, a smaller mask probability can be configured so that the decoder of the text recognition model can refer to the text encoding features for decoding to obtain accurate text recognition results. As the training progresses, the image encoder and decoder in the text recognition model have a certain degree of accuracy. At this time, the mask probability can be gradually increased so that the decoder of the text recognition model gradually does not rely on the text encoding features and can also accurately decode.

[0085] In this embodiment, the image encoder can be implemented using densely connected convolutional networks (DenseNet), or a residual network (ResNet), a feature pyramid network (FPNet), or a vision transformer network (ViTNet).

[0086] Since the image encoder uses DenseNet instead of the traditional ViT-encoder, it can maintain the same or even better effect with fewer parameters.

[0087] In this embodiment, the text encoder adopts a transformer network, such as a Bidirectional Encoder Representations from Transformers (BERT) model, and various improved models based on BERT.

[0088] In this embodiment, the decoder can be a transform network based on a multi-head self-attention mechanism. Specifically, a network with 256 layers of embedding matrix dimensions, 8 heads of attention mechanism, 1024 feedforward neural network matrix dimensions, and a zeroing ratio (dropout) of 0.3 can be used.

[0089] Figure 3 is a training principle diagram of the text recognition model of this embodiment, such as Figure 3 As shown, it can be known that in the training of the text recognition model of this embodiment, due to the large loss in the early stage, the text encoding features can be controlled to be rarely covered by the mask, that is, the downstream decoder will obtain the complete annotated text content. At this time, the decoder can quickly learn the correct output from the text encoding features. In the medium term, as the accuracy of the model increases, the loss function gradually decreases, and the text encoding features are gradually covered by the mask. At this time, the decoder part cannot obtain the content covered by the mask from the text side, and can only seek supplementation from the image side information, that is, the model weights are gradually transferred from the text side to the image side. In the later stage, when the loss function is less than the preset hyperparameter, the text side information is no longer provided to the decoder. At this time, the training of the text recognition model regresses to the normal training of the image encoder and image decoder until the text recognition model converges completely. In this way, when applying reasoning, text recognition can be performed entirely based on the information on the image side.

[0090] In the training of the text recognition model of this embodiment, masked text encoding features are used to assist the text recognition model in training, so that the text recognition model can converge faster and the accuracy of the trained text recognition model can be effectively improved.

[0091] In the training of the text recognition model of this embodiment, an adaptive masking strategy is adopted, which can effectively reduce the amount of hyperparameters and thus effectively reduce the difficulty of model tuning.

[0092] It has been experimentally verified that the text recognition model trained by the training method of this embodiment can effectively process text in complex images and is not limited to text images in a single direction, thereby coping with diverse and complex text scenes.

[0093] Moreover, by introducing text features during the training process, the image extraction capability of the text recognition model can be effectively enhanced, thereby effectively improving the generalization capability of the text recognition model in different application scenarios.

[0094] The text recognition model of this embodiment avoids the use of a huge language model, reducing the need for complex data and high-cost training.

[0095] The text recognition model trained in this embodiment can be applied in the field of text recognition of complex images, and can effectively improve the recognition accuracy, for example, it can be applied to formula recognition and signature recognition.

[0096] Figure 4 is a schematic diagram according to the third embodiment of the present disclosure; Figure 4 As shown, this embodiment provides a text recognition method, which may specifically include the following steps:

[0097] S401, obtaining a text image to be recognized;

[0098] S402, obtaining a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0099] S403: Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image.

[0100] The text recognition method of this embodiment is performed by a text recognition device, which can be an electronic entity, an intelligent entity, or a software integrated application. When used, a text picture is input, and the text picture is a picture containing text information. At this time, the text recognition device can recognize the text information in the text picture.

[0101] The text recognition method of this embodiment is to Figure 1-Figure 3 Application of the text recognition model trained by the illustrated embodiment.

[0102] In order to align features with those in the training phase, the text recognition model of this embodiment needs to pre-configure text mask features containing blank text information when it is applied, which means that all texts are masked.

[0103] The text recognition method of this embodiment, with reference to the text mask features, can align the inference process of the text recognition model with the training process features, and thus the text recognition model can be used to accurately and efficiently perform text recognition on text images.

[0104] Figure 5 is a schematic diagram according to a fourth embodiment of the present disclosure; Figure 5 As shown, this embodiment provides a text recognition method. Figure 4 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 5 As shown, the text recognition method of this embodiment may specifically include the following steps:

[0105] S501, obtaining a text image to be recognized;

[0106] S502: Obtain a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0107] For example, the text mask feature of this embodiment can be represented by a matrix whose elements are all zero. In the process of file recognition, this feature is only used to align with the features in the training process to effectively ensure the accuracy of text recognition, and has no other meaning.

[0108] S503, using the picture encoder in the text recognition model to obtain the encoding features of the text picture;

[0109] S504, concatenating the encoding feature of the text image and the preconfigured text mask feature to obtain a concatenated feature;

[0110] S505: Using the image decoder in the text recognition model, decode the splicing features to obtain text information of the text image.

[0111] Steps S503-S505 of this embodiment are as described above. Figure 4 A specific implementation method of S403 of the illustrated embodiment, through which the encoded features of the text image and the preconfigured text mask features are spliced, and then the spliced ​​features are decoded by the image decoder, so that the features in the reasoning process and the features in the training process can be aligned, effectively ensuring the recognition accuracy of the text recognition model.

[0112] The text recognition model in this embodiment is the above Figure 1-Figure 3 The text recognition model trained in the illustrated embodiment can also be described in detail with reference to the records of the above-mentioned related embodiments, which will not be described again here.

[0113] The text recognition method of this embodiment can effectively improve the accuracy and efficiency of text recognition by adopting the text recognition model trained in the above embodiment.

[0114] Figure 6 is a schematic diagram according to the fifth embodiment of the present disclosure; Figure 5 As shown, this embodiment provides a text recognition model training device 600, including:

[0115] The data acquisition module 601 is used to acquire a training data set, wherein the training data set includes a training text image and corresponding annotated text information;

[0116] A mask acquisition module 602 is used to acquire a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information;

[0117] The training module 603 is used to train a text recognition model based on the mask probability and the training text images and the annotated text information in the training data group; the text recognition model includes an image encoder, a text encoder and a decoder.

[0118] The text recognition model training device 600 of this embodiment implements the implementation principle and technical effect of text recognition model training by adopting the above-mentioned modules, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0119] Figure 7 is a schematic diagram according to the sixth embodiment of the present disclosure; Figure 7 As shown, this embodiment provides a text recognition model training device 700, in the above Figure 6 Based on the illustrated embodiment, the technical solution of the present disclosure is further described in more detail. Figure 7 As shown, the text recognition model training device 700 of this embodiment includes the above Figure 6 Modules with the same name and function are shown as follows: a data acquisition module 701 , a mask acquisition module 702 , and a training module 703 .

[0120] like Figure 7 As shown, in the text recognition model training device 700 of this embodiment, the training module 703 includes:

[0121] The picture encoding unit 7031 is used to encode the training text picture by using the picture encoder in the text recognition model to obtain a first training encoding feature of the training text picture;

[0122] A text encoding unit 7032 is used to encode the annotated text information using a text encoder in the text recognition model to obtain a second training encoding feature of the annotated text information;

[0123] A masking unit 7033, configured to randomly mask the second training coding feature based on the masking probability to obtain a training mask feature;

[0124] A decoding unit 7034 is used to decode the first training coding feature and the training mask feature using a decoder in the text recognition model to obtain predicted text information;

[0125] The parameter adjustment unit 7035 is used to adjust the parameters of the picture encoder, text encoder and decoder in the text recognition model based on the annotated text information and the predicted text information.

[0126] Further optionally, in one embodiment of the present disclosure, the parameter adjustment unit 7035 is used to:

[0127] Based on the annotated text information and the predicted text information, obtaining a value of a loss function;

[0128] Based on the value of the loss function, parameters of the text encoder, image encoder and decoder in the text recognition model are adjusted.

[0129] Further optionally, if Figure 7 As shown, in one embodiment of the present disclosure, the training device 700 for the text recognition model further includes:

[0130] The mask adjustment module 704 is used to adjust the mask probability based on the value of the loss function and the preset hyperparameters.

[0131] Further optionally, in one embodiment of the present disclosure, the mask adjustment module 704 is used to:

[0132] Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula:

[0133] P=1-(Loss c -α)

[0134] Wherein, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

[0135] Further optionally, if Figure 7 As shown, in one embodiment of the present disclosure, the training module 703 further includes:

[0136] The determination unit 7036 is used to determine whether the value of the loss function is greater than the preset hyperparameter.

[0137] Further optionally, in one embodiment of the present disclosure, the mask adjustment module 704 is used to:

[0138] If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

[0139] Further optionally, in one embodiment of the present disclosure, the mask acquisition module 702 is used to: during initial training, configure the mask probability to be a random number that is less than a preset probability threshold.

[0140] The text recognition model training device 700 of this embodiment implements the implementation principle and technical effect of text recognition model training by adopting the above-mentioned modules, which is the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the records of the above-mentioned related method embodiments, which will not be repeated here.

[0141] Figure 8 is a schematic diagram according to the seventh embodiment of the present disclosure; Figure 8 As shown, this embodiment provides a text recognition device 800, including:

[0142] The image acquisition module 801 is used to acquire the text image to be recognized;

[0143] The mask acquisition module 802 is used to acquire a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information;

[0144] The recognition module 803 is used to perform text recognition on the text image based on the text mask feature using a pre-trained text recognition model.

[0145] The text recognition device 800 of this embodiment realizes the implementation principle and technical effect of text recognition by adopting the above modules, which is the same as the implementation of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0146] Fig. 9 is a schematic diagram according to an eighth embodiment of the present disclosure; Fig. 9 As shown, this embodiment provides a text recognition device 900, in the above Figure 8 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Fig. 9 As shown, this embodiment provides a text recognition device 900 including the above Figure 8 Modules with the same name and function are shown as follows: image acquisition module 901 , mask acquisition module 902 , and recognition module 903 .

[0147] like Fig. 9 As shown, the identification module 903 includes:

[0148] A text encoding unit 9031 is used to obtain encoding features of the text image using the image encoder in the text recognition model;

[0149] A concatenation unit 9032, in which a user concatenates the encoding feature of the text image with the preconfigured text mask feature to obtain a concatenated feature;

[0150] The decoding unit 9033 is used to decode the splicing features using the image decoder in the text recognition model to obtain text information of the text image.

[0151] Further optionally, in one embodiment of the present disclosure, the text mask feature is represented by a matrix whose elements are all zero.

[0152] The text recognition device 900 of this embodiment realizes the implementation principle and technical effect of text recognition by adopting the above modules, which is the same as the implementation of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0153] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0154] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0155] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0156] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0157] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0158] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the above-mentioned methods of the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute the above method of the present disclosure in any other appropriate manner (for example, by means of firmware).

[0159] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0160] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0161] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0162] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0163] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0164] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0165] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0166] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A training method for a text recognition model, comprising: Obtaining a training data set, wherein the training data set includes a training text image and corresponding annotated text information; Obtaining a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information; Based on the mask probability, the training text image in the training data set, and the annotated text information, a text recognition model is trained; the text recognition model includes a picture encoder, a text encoder, and a decoder.

2. The method according to claim 1, wherein: Training a text recognition model based on the mask probability and the training text image and the annotated text information in the training data set includes: Using the picture encoder in the text recognition model to encode the training text picture to obtain a first training encoding feature of the training text picture; Using a text encoder in the text recognition model to encode the annotated text information to obtain a second training encoding feature of the annotated text information; Randomly masking the second training coding feature based on the mask probability to obtain a training mask feature; Based on the first training coding feature and the training mask feature, decoding is performed using a decoder in the text recognition model to obtain predicted text information; Based on the annotated text information and the predicted text information, parameters of the picture encoder, text encoder and decoder in the text recognition model are adjusted.

3. The method according to claim 2, wherein: Based on the annotated text information and the predicted text information, adjusting parameters of the picture encoder, the text encoder and the decoder in the text recognition model includes: Based on the annotated text information and the predicted text information, obtaining a value of a loss function; Based on the value of the loss function, parameters of the text encoder, image encoder and decoder in the text recognition model are adjusted.

4. The method according to claim 3, wherein: After obtaining the value of the loss function based on the annotated text information and the predicted text information, the method further includes: The mask probability is adjusted based on the value of the loss function and a preset hyperparameter.

5. The method according to claim 4, wherein: Adjusting the mask probability based on the value of the loss function and a preset hyperparameter includes: Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula: P=1-(Loss c -α) Wherein, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

6. The method according to claim 4, wherein: Based on the value of the loss function and the preset hyperparameters, before adjusting the mask probability, the method further includes: Determine whether the value of the loss function is greater than the preset hyperparameter.

7. The method according to claim 4, wherein: Adjusting the mask probability based on the value of the loss function and the preset hyperparameters also includes: If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

8. The method according to any one of claims 1 to 7, wherein: Obtaining the mask probability includes: during initial training, configuring the mask probability to be a random number that is less than a preset probability threshold.

9. A text recognition method, comprising: Get the text image to be recognized; Obtaining a preconfigured text mask feature, where the text mask feature is a preconfigured feature containing blank text information; Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image.

10. The method according to claim 9, wherein: Based on the text mask feature, a pre-trained text recognition model is used to perform text recognition on the text image, including: Using the picture encoder in the text recognition model to obtain the encoding features of the text picture; Splicing the encoding feature of the text image and the preconfigured text mask feature to obtain a spliced ​​feature; The image decoder in the text recognition model is used to decode the splicing features to obtain text information of the text image.

11. The method according to claim 9 or 10, wherein: The text mask feature is represented by a matrix whose elements are all zero.

12. A training device for a text recognition model, comprising: A data acquisition module, used to acquire a training data set, wherein the training data set includes a training text image and corresponding annotated text information; A mask acquisition module, used to acquire a mask probability, where the mask probability is used to identify a probability of randomly masking the text encoding feature of the annotated text information; A training module is used to train a text recognition model based on the mask probability and the training text pictures and the annotated text information in the training data group; the text recognition model includes a picture encoder, a text encoder and a decoder.

13. The device according to claim 12, wherein: The training module comprises: A picture encoding unit, used to encode the training text picture using the picture encoder in the text recognition model to obtain a first training encoding feature of the training text picture; A text encoding unit, used to encode the annotated text information using a text encoder in the text recognition model to obtain a second training encoding feature of the annotated text information; A masking unit, configured to randomly mask the second training coding feature based on the masking probability to obtain a training masked feature; A decoding unit, configured to decode the first training coding feature and the training mask feature using a decoder in the text recognition model to obtain predicted text information; A parameter adjustment unit is used to adjust the parameters of the picture encoder, text encoder and decoder in the text recognition model based on the annotated text information and the predicted text information.

14. The device according to claim 13, wherein: The parameter adjustment unit is used to: Based on the annotated text information and the predicted text information, obtaining a value of a loss function; Based on the value of the loss function, parameters of the text encoder, image encoder and decoder in the text recognition model are adjusted.

15. The device according to claim 14, wherein: The device also includes: A mask adjustment module is used to adjust the mask probability based on the value of the loss function and a preset hyperparameter.

16. The device according to claim 15, wherein: The mask adjustment module is used to: Based on the value of the loss function and the preset hyperparameters, the mask probability is adjusted using the following formula: P=1-(Loss c -α) Wherein, P represents the mask probability; Loss c Represents the value of the loss function; α is a preset hyperparameter.

17. The device according to claim 15, wherein: The training module further includes: A determination unit is used to determine whether the value of the loss function is greater than the preset hyperparameter.

18. The device according to claim 15, wherein: The mask adjustment module is used to: If the value of the loss function is less than or equal to the preset hyperparameter, the mask probability is adjusted to 1.

19. The device according to any one of claims 12 to 18, wherein: The mask acquisition module is used to: during initial training, configure the mask probability to be a random number that is less than a preset probability threshold.

20. A text recognition device, comprising: An image acquisition module is used to acquire text images to be recognized; A mask acquisition module, used to acquire a preconfigured text mask feature, wherein the text mask feature is a preconfigured feature containing blank text information; The recognition module is used to perform text recognition on the text image based on the text mask feature using a pre-trained text recognition model.

21. The device according to claim 20, wherein: The identification module comprises: A text encoding unit, used to obtain encoding features of the text image using the image encoder in the text recognition model; A splicing unit, in which a user splices the encoding feature of the text image with the preconfigured text mask feature to obtain a splicing feature; A decoding unit is used to decode the splicing features using the image decoder in the text recognition model to obtain text information of the text image.

22. The device according to claim 20 or 21, wherein: The text mask feature is represented by a matrix whose elements are all zero.

23. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8 or 9-11.

24. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8 or 9-11.

25. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1-8 or 9-11.

Images