Text recognition model training method, recognition method, related equipment and storage medium

Abstract

The present disclosure relates to a text recognition model training method, a recognition method, related equipment and a storage medium, the method comprising obtaining an initial text feature extraction model and a first sample data set, the first sample data set comprising unlabeled text line image samples belonging to different languages; using the first sample data set, unsupervised training is performed on the initial text feature extraction model to obtain a target text feature extraction model; an initial text recognition model is constructed based on the target text feature extraction model; a second sample data set is obtained, the second sample data set comprises labeled text line image samples belonging to the target language; the initial text recognition model is trained using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language. The disclosed embodiment can be used to quickly transfer learning to obtain a target text recognition model, and the text recognition accuracy of the target text recognition model can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a text recognition model training method, a recognition method, related equipment and a storage medium. Background Art

[0002] With the popularity of deep learning technology, OCR (Optimal Character Recognition) methods based on deep learning are increasingly being adopted because of their excellent recognition performance. One of the mainstream frameworks is CRNN (Convolutional Recurrent Neural Network) + CTC (Connectionist Temporal Classification), which normalizes the text line image to a specific size input, such as a fixed height of 32, and extracts the image features through a convolutional neural network, and inputs the features into an encoder for encoding, such as an encoder based on LSTM (Long Short-Term Memory), to obtain a temporal feature sequence, which is then input into a CTC decoder for decoding, and finally the recognition result of the text line image is obtained.

[0003] Since different languages ​​have different character sets, for example, English has 52 uppercase and lowercase letters, Chinese has 3755 simplified first-level commonly used Chinese characters, 5401 traditional first-level commonly used Chinese characters, etc., when there are multiple languages, the character set will become very large, resulting in the inability of the model to converge and train an OCR model that recognizes all languages. Therefore, it is necessary to train an OCR model for each language separately, and each model only classifies the characters contained in the language.

[0004] When training OCR models for multiple languages, it is necessary to collect labeled samples for each language and train the model separately. This is not only costly, but the recognition accuracy of each language is also directly related to the amount of labeled data collected. For some small languages ​​that cannot collect a large amount of data, the trained model will be less effective. Summary of the invention

[0005] The present disclosure provides a training method, device, electronic device and storage medium for a text recognition model, so as to at least solve the problems in the related art of high cost of collecting and annotating samples and poor effect of OCR models for minority languages. The technical solutions of the present disclosure are as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for training a text recognition model is provided, the method comprising:

[0007] Acquire an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages;

[0008] Using the first sample data set, performing unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model;

[0009] Constructing an initial text recognition model, wherein the initial text recognition model includes a text feature extraction layer, and the text feature extraction layer is determined according to the target text feature extraction model;

[0010] Acquire a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language;

[0011] The initial text recognition model is trained using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language.

[0012] Optionally, the initial text recognition model is constructed, the initial text recognition model includes a text feature extraction layer, and the text feature extraction layer is determined according to the target text feature extraction model and includes:

[0013] Determine the target language family to which the target language belongs;

[0014] Acquire a third sample data set, wherein the third sample data set includes unlabeled text line image samples in different languages ​​under the target language family;

[0015] Using the third sample data set to perform unsupervised training on the target text feature extraction model to obtain an updated target text feature extraction model;

[0016] According to the updated target text feature extraction model, the text feature extraction layer of the initial text recognition model is constructed.

[0017] Optionally, the using the second sample data set to train the initial text recognition model to obtain a target text recognition model for recognizing text belonging to the target language includes:

[0018] When the text line image samples in the second sample data set meet the preset sample conditions, obtaining the corpus text corresponding to the target language;

[0019] generating, according to the corpus text, a text line image sample containing the corpus text, to obtain a supplementary sample data set of the second sample data set;

[0020] The initial text recognition model is trained using the second sample data set and the supplementary sample data set.

[0021] Optionally, the acquiring of the corpus text corresponding to the target language includes:

[0022] Acquire text in the second sample data set whose occurrence frequency is less than or equal to a preset frequency;

[0023] A corpus text corresponding to the target language is obtained, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

[0024] Optionally, the training of the initial text recognition model using the second sample data set and the supplementary sample data set includes:

[0025] Using the second sample data set, training the initial text recognition model at a first learning rate;

[0026] The initial text recognition model trained using the second sample data set is trained using the supplementary sample data set at a second learning rate, wherein the first learning rate is greater than the second learning rate.

[0027] Optionally, the using the first sample data set to perform unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model includes:

[0028] Performing data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples;

[0029] The first text line enhanced image sample and the second text line enhanced image sample are used, and an unsupervised algorithm based on contrast loss is adopted to perform unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model.

[0030] Optionally, the initial text recognition model further includes a text feature encoder and a text feature decoder, and the text line image samples in the second sample data set include text content description labels;

[0031] The step of training the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language includes:

[0032] Using the text feature extraction layer to extract text features from the text line image samples in the second sample data set to obtain corresponding text features;

[0033] Processing the text features using the feature encoder and the feature decoder to obtain corresponding text recognition results;

[0034] Determining a target loss function of the initial text recognition model based on the text recognition result and the text content description label;

[0035] Based on the target loss function, the model parameters of the initial text recognition model are adjusted to obtain a target text recognition model for recognizing the target language.

[0036] According to a second aspect of an embodiment of the present disclosure, a text recognition method is provided, the method comprising:

[0037] Obtain an image to be recognized;

[0038] Determine the target language of the image to be recognized, and obtain a target text recognition model corresponding to the target language;

[0039] Normalizing the image to be recognized to obtain a text line image corresponding to the image to be recognized;

[0040] The target text recognition model is used to detect the text line image corresponding to the image to be recognized to obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using any of the methods described in the first aspect of the embodiments of the present disclosure.

[0041] According to a third aspect of an embodiment of the present disclosure, a training device for a text recognition model is provided, the device comprising:

[0042] A first training data acquisition module is configured to execute acquisition of an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages;

[0043] a target text feature extraction model acquisition module, configured to perform unsupervised training on the initial text feature extraction model using the first sample data set to obtain a target text feature extraction model;

[0044] An initial text recognition model construction module is configured to execute construction of an initial text recognition model, wherein the initial text recognition model includes a text feature extraction layer, and the text feature extraction layer is determined according to the target text feature extraction model;

[0045] A second training data acquisition module is configured to acquire a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language;

[0046] The target text recognition model acquisition module is configured to train the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language.

[0047] Optionally, the initial text recognition model construction module includes:

[0048] A target language determination unit, configured to determine the target language to which the target language belongs;

[0049] A third sample data set acquisition unit is configured to acquire a third sample data set, wherein the third sample data set includes unlabeled text line image samples in different languages ​​under the target language family;

[0050] a target text feature extraction model updating unit, configured to perform unsupervised training on the target text feature extraction model using the third sample data set to obtain an updated target text feature extraction model;

[0051] The text feature extraction layer construction unit is configured to execute the text feature extraction layer of the initial text recognition model according to the updated target text feature extraction model.

[0052] Optionally, the target text recognition model acquisition module includes:

[0053] A corpus text acquisition unit, configured to acquire a corpus text corresponding to the target language when a text line image sample in the second sample data set meets a preset sample condition;

[0054] A supplementary sample data set acquisition unit is configured to generate a text line image sample containing the corpus text according to the corpus text, so as to obtain a supplementary sample data set for the second sample data set;

[0055] The initial text recognition model training unit is configured to train the initial text recognition model using the second sample data set and the supplementary sample data set.

[0056] Optionally, the corpus text acquisition unit includes:

[0057] A character acquisition subunit is configured to acquire characters in the second sample data set whose occurrence frequency is less than or equal to a preset frequency;

[0058] The corpus text acquisition subunit is configured to execute acquisition of corpus text corresponding to the target language, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

[0059] Optionally, the initial text recognition model training unit includes:

[0060] A first initial text recognition model training subunit is configured to train the initial text recognition model using the second sample data set at a first learning rate;

[0061] The second initial text recognition model training subunit is configured to use the supplementary sample data set to train the initial text recognition model trained using the second sample data set at a second learning rate, wherein the first learning rate is greater than the second learning rate.

[0062] Optionally, the target text feature extraction model acquisition module includes:

[0063] An image sample processing unit is configured to perform data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples;

[0064] The unsupervised training unit is configured to perform unsupervised training on the initial text feature extraction model using the first text line enhanced image sample and the second text line enhanced image sample, adopting an unsupervised algorithm based on contrast loss, to obtain a target text feature extraction model.

[0065] Optionally, the initial text recognition model further includes a text feature encoder and a text feature decoder, and the text line image samples in the second sample data set include text content description labels;

[0066] The target text recognition model acquisition module includes:

[0067] a text feature acquisition unit, configured to perform text feature extraction on the text line image samples in the second sample data set using the text feature extraction layer to obtain corresponding text features;

[0068] A text recognition result acquisition unit is configured to process the text features using the feature encoder and the feature decoder to obtain a corresponding text recognition result;

[0069] A target loss function determination unit is configured to determine a target loss function of the initial text recognition model based on the text recognition result and the text content description label;

[0070] The model parameter adjustment unit is configured to adjust the model parameters of the initial text recognition model based on the target loss function to obtain a target text recognition model for recognizing the target language.

[0071] According to a fourth aspect of an embodiment of the present disclosure, a text recognition device is provided, the device comprising:

[0072] The module for acquiring an image to be recognized is configured to acquire an image to be recognized;

[0073] A target text recognition model acquisition module is configured to determine the target language to which the image to be recognized belongs and acquire a target text recognition model corresponding to the target language;

[0074] The to-be-recognized image processing module is configured to perform normalization processing on the to-be-recognized image to obtain a text line image corresponding to the to-be-recognized image;

[0075] The image detection module is configured to execute detection on the text line image corresponding to the image to be recognized using the target text recognition model to obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using the training method of the text recognition model described in any one of the first aspects of the embodiments of the present disclosure.

[0076] According to a fifth aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a method as described in any one of the above-mentioned first aspects.

[0077] According to the sixth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any method described in the first aspect of the embodiment of the present disclosure.

[0078] According to a seventh aspect of an embodiment of the present disclosure, there is provided a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the methods described in the first aspect of the embodiment of the present disclosure.

[0079] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0080] By using unlabeled text line image sample data in various languages ​​to perform unsupervised training on the initial text feature extraction model, the common features of different languages ​​can be learned, and then the feature extraction layer of the initial text recognition model of the target language is constructed using the trained target text feature extraction model, and the initial text recognition model is supervised trained using labeled text line image samples of the target language. In this way, when there are not many labeled samples of a small language, the feature expression ability learned from other languages ​​in the feature extraction layer can be used to quickly transfer learning to obtain a target text recognition model that recognizes the target language text. In addition, the target text recognition model of each language can learn the information contained in the samples of other languages ​​through self-supervision, thereby improving the accuracy of the target text recognition model in recognizing the corresponding target language text.

[0081] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0082] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0083] Figure 1 is a flowchart of a method for training a text recognition model according to an exemplary embodiment;

[0084] Figure 2 is a flow chart showing a method of performing unsupervised training on the initial text feature extraction model using the first sample data set to obtain a target text feature extraction model according to an exemplary embodiment;

[0085] Figure 3 is a schematic structural diagram of an initial text recognition model according to an exemplary embodiment;

[0086] Figure 4 is a schematic diagram of a process of constructing an initial text recognition model according to an exemplary embodiment;

[0087] Figure 5 is a flowchart of a method for training a text recognition model according to an exemplary embodiment;

[0088] Figure 6 is a flow chart showing a method of training the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language according to an exemplary embodiment;

[0089] Figure 7 is a schematic diagram of a process of obtaining a corpus text corresponding to the target language according to an exemplary embodiment;

[0090] Figure 8 is a schematic diagram of a process of training the initial text recognition model using the second sample data set and the supplementary sample data set according to an exemplary embodiment;

[0091] Fig. 9 is a flowchart of a text recognition method according to an exemplary embodiment;

[0092] Fig.10 is a block diagram of a training device for a text recognition model according to an exemplary embodiment;

[0093] Fig.11 is a block diagram of a text recognition device according to an exemplary embodiment;

[0094] Fig.12 It is a block diagram of an electronic device for training a text recognition model or for text recognition according to an exemplary embodiment. DETAILED DESCRIPTION

[0095] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0096] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0097] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties.

[0098] In some application scenarios, it is necessary to recognize text in multiple languages, such as Chinese, English, Portuguese, Spanish, Arabic, Thai, Indonesian, Urdu, etc.

[0099] If we follow the current mainstream method, recognizing text in multiple languages ​​requires training multiple OCR models. That is, for each language, we need to collect an equal amount of annotated samples (such as 200,000 annotated samples), and then establish a character set for each language and train their respective OCR models.

[0100] In real-world scenarios, some languages ​​have a large amount of annotated data, and subsequent data collection and annotation are relatively simple, such as Chinese and English. However, it is difficult to obtain annotated data for other languages, such as Arabic and Urdu. As a result, it is relatively easy to train OCR models for Chinese and English, but it is very difficult to train OCR models for minority languages. Annotated data for different languages ​​cannot be used interchangeably, for example, English annotated data is not helpful for Arabic training.

[0101] In view of this, the present embodiment proposes a text recognition model training method, a text recognition method, a device, an electronic device, and a storage medium.

[0102] Figure 1 is a flowchart of a method for training a text recognition model according to an exemplary embodiment. Figure 1 As shown, the training method includes the following steps:

[0103] Step S101: obtaining an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages;

[0104] In this embodiment, the initial text feature extraction model can learn common visual features across languages ​​through the first sample data set, such as stroke features, character features, and writing order features of text. The initial text feature extraction model can be a convolutional neural network model.

[0105] The first sample data set is an unsupervised sample training set, which may include tens of millions of real text line image samples from different languages, for example, real text line image samples from all available languages ​​such as Chinese, English, Portuguese, Spanish, Arabic, Thai, Indonesian, Urdu, etc.

[0106] The text line image samples in the first sample data set can directly use the text area results output by the general text detection module without manual transcription and annotation. Each image sample is a horizontal text line image sample, and each horizontal text line image sample can be normalized to a text line image sample with a fixed height and a variable length. For example, each text line image sample can be normalized to a size of 32*X, where the height is fixed at 32 points and the width X can be variable, that is, the width can be variable.

[0107] Step S103: using the first sample data set, performing unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model;

[0108] In this embodiment, if Figure 2 As shown, the use of the first sample data set to perform unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model may include:

[0109] Step S201: performing data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples;

[0110] In this embodiment, data enhancement processing can be performed on the text line image samples to be processed in the first sample data set to obtain a first text line enhanced image sample and a second text line enhanced image sample, and the data enhancement processing includes color data enhancement, cropping, inversion, translation transformation, rotation transformation, adding Gaussian noise, etc.

[0111] The number of data enhancement processing times can be determined according to demand. For example, for a certain text line image sample to be processed, two random data enhancement processings can be performed on it to obtain a first text line enhanced image sample, and then the text line image sample to be processed can be subjected to two random data enhancement processings to obtain a second text line enhanced image sample.

[0112] It should be noted that the second text line enhancement sample is different from the first text line enhancement sample.

[0113] Step S203: using the first text line enhanced image sample and the second text line enhanced image sample, and adopting an unsupervised algorithm based on contrast loss, unsupervised training is performed on the initial text feature extraction model to obtain a target text feature extraction model.

[0114] In this embodiment, the initial text feature extraction model may include a coding layer, a momentum coding layer, a queue layer and a loss function layer, wherein the coding layer and the momentum coding layer can be synchronized by momentum, and the queue layer includes enhanced sample vectors of each historical text line.

[0115] After obtaining the first text line enhanced image sample and the second text line enhanced image sample, the first text line enhanced image sample is input into the encoding layer to obtain a first text line enhanced image sample vector, and the second text line enhanced image sample is input into the momentum encoding layer to obtain a second text line enhanced image sample vector.

[0116] Then, the first text line enhanced image sample vector and the second text line enhanced image sample vector constitute a positive sample pair, and the second text line enhanced image sample vector and each historical enhanced sample vector in the queue layer constitute a negative sample pair, which are input into the loss function layer. The loss function of the loss function layer will perform loss calculation based on the input positive sample pair and negative sample pair, and determine whether the loss function layer converges according to the loss calculation result.

[0117] When the loss function layer has not converged, loss back propagation is performed to update the parameters of the encoding layer, and the parameters of the momentum encoding layer are updated according to the momentum formula.

[0118] When the loss function layer has not converged, it means that the updated initial text feature extraction model still needs to continue training. At this time, the next text line image sample in the first sample data set will be selected, and the next text line image sample will be randomly enhanced to obtain the next first text line enhanced image sample and the next second text line enhanced image sample. The next first text line enhanced image sample and the next second text line enhanced image sample will be used to continue training the updated initial text feature extraction model until the loss function layer of the initial text feature extraction model converges to obtain the trained target text feature extraction model.

[0119] During training, the second text line enhanced image sample vector may enter the queue layer, and the historical enhanced sample vector at the head of the queue may be dequeued to maintain the length K of the queue layer.

[0120] In this way, the dictionary can be queued, that is, the dictionary can be turned into a queue of length K, so that the size of the dictionary does not need to be constrained by the size of each batch of pictures, and can be set to any size, which ensures the decoupling of the size of the dictionary from the size of each batch of pictures. At the same time, the momentum update method is adopted to ensure that the model can still converge under the training of large data sets.

[0121] In this embodiment, data enhancement processing is performed on the to-be-processed text line image samples in the first sample data set, and the enhanced image samples are used to perform unsupervised training on the initial text feature extraction model. This not only can fully utilize the relatively low-cost unlabeled image samples for model training, but the unsupervised algorithm based on contrast loss can effectively bring together the enhanced image samples belonging to the same image sample, so that the trained target text feature extraction model can more accurately extract features from the text in the image.

[0122] Step S105: constructing an initial text recognition model, wherein the initial text recognition model includes a text feature extraction layer, and the text feature extraction layer is determined according to the target text feature extraction model;

[0123] In some embodiments, Figure 3 As shown, the structure and parameters of the encoding layer in the target text feature extraction model can be retained, and based on the structure and parameters, a feature encoder (such as a BiLSTM encoder) and a feature decoder (such as a CTC decoder) can be built to obtain the initial text recognition model.

[0124] In other embodiments, Figure 4 and Figure 5As shown, the initial text recognition model is constructed, and the initial text recognition model includes a text feature extraction layer. The text feature extraction layer is determined according to the target text feature extraction model and may also include:

[0125] Step S401: determining the target language family to which the target language belongs;

[0126] In this embodiment, the target language that the trained target text recognition model needs to recognize can be determined first, and then the target language family that the target language belongs to can be determined according to the classification of different languages ​​in various language families. For example, it is determined that the trained target text recognition model is used to recognize Portuguese, and since Portuguese belongs to the Latin language family, the target language family is determined to be the Latin language family.

[0127] Step S403: obtaining a third sample data set, wherein the third sample data set includes unlabeled text line image samples in different languages ​​under the target language family;

[0128] For example, when the target language is Portuguese and the target language family is a Latin language family, the third sample data set includes unlabeled text line image samples of different languages ​​under the Latin language family. Since Portuguese, English and Spanish all belong to the Latin language family, the third sample data set may include unlabeled text line image samples of Portuguese, English and Spanish.

[0129] Step S405: performing unsupervised training on the target text feature extraction model using the third sample data set to obtain an updated target text feature extraction model;

[0130] Specifically, data enhancement processing can be performed on the text line image samples to be processed in the third sample data set to obtain third text line enhanced image samples and fourth text line enhanced image samples; then, the third text line enhanced image samples and the fourth text line enhanced image samples are used to perform unsupervised training on the target text feature extraction model using an unsupervised algorithm based on contrast loss to obtain an updated target text feature extraction model.

[0131] In this embodiment, the learning speed of the model can be controlled by controlling the learning rate when updating the target text feature extraction model. Since the first sample data set includes text line image samples of all available languages, a relatively high learning rate can be set during model training to increase the training speed. Since the target text feature extraction model has converged on the first sample data set, a relatively small learning rate can be used to perform fine-tuning on the third sample data set.

[0132] Step S407: constructing a text feature extraction layer of the initial text recognition model according to the updated target text feature extraction model.

[0133] In this embodiment, the structure and parameters of the encoding layer in the updated target text feature extraction model can be retained, and based on the structure and parameters, a feature encoder (such as a BiLSTM encoder) and a feature decoder (such as a CTC decoder) can be built to obtain the initial text recognition model.

[0134] Since different languages ​​in the same target language family have similar visual features, the initial text recognition model constructed by this embodiment can learn visual features common to the target language from other languages ​​in the same target language family, and this embodiment uses a relatively small learning rate to perform fine-tuning on the third sample data set, so that it can fully learn visual features common to the target language from other languages ​​in the same target language family. Based on this, the target text recognition model trained in this embodiment can more accurately recognize text in the target language than the target text recognition model constructed and trained directly using the target text feature extraction model.

[0135] Step S107: obtaining a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language;

[0136] In this embodiment, each text line image sample in the second sample data set is a horizontal text line image sample, and each text line image sample in the second sample data set includes an image and a corresponding text content description label.

[0137] Specifically, each text line image sample can be normalized to a text line image sample with a fixed height and a variable length. For example, each text line image sample can be normalized to a size of 32*X, where the height is fixed to 32 points and the width X may not be fixed, that is, the width may be variable.

[0138] The text content description tag may be a string of characters, which is used to describe the text content contained in the image.

[0139] The number of text line image samples in the second sample data set is related to the type of the target language. For languages ​​such as Chinese and English, which are easy to collect labeled samples, the second sample data set may include a large number of labeled text line image samples in the corresponding language.

[0140] For languages ​​for which it is not easy to collect annotated samples, such as small languages ​​such as Portuguese, the second sample data set may include a small number of annotated text line image samples in the corresponding language.

[0141] In addition, when training target text recognition models in different languages, a character set needs to be established for each language. The character set contains all the text categories of the language that are expected to be recognized. The length of the character set also determines the final number of categories of the target text recognition model. For example, the character set of the English language needs to contain 52 text categories (26 uppercase letters and 26 lowercase letters), and the final classification categories of the target text recognition model used to recognize the English language are also 52. For another example, there are 3501 simplified Chinese characters in Chinese, so the classification categories of the target text recognition model used to recognize Chinese and English are 3501 Chinese characters + 52 English characters + x symbols.

[0142] Step S109: using the second sample data set to train the initial text recognition model to obtain a target text recognition model for recognizing text belonging to the target language.

[0143] In this embodiment, the initial text recognition model may further include a feature encoder and a feature decoder in addition to the feature extraction layer, and the text line image samples in the second sample data set include text content description labels; the training of the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language may include:

[0144] Step 1: Using the text feature extraction layer to extract text features from the text line image samples in the second sample data set to obtain corresponding text features;

[0145] Step 2: Processing the text features using the feature encoder and the feature decoder to obtain corresponding text recognition results;

[0146] Step 3: Determine the target loss function of the initial text recognition model based on the text recognition result and the text content description label;

[0147] Step 4: Based on the target loss function, adjust the model parameters of the initial text recognition model to obtain a target text recognition model for recognizing text belonging to the target language.

[0148] The target loss function represents the distance between the text recognition result and the text content description label.

[0149] During the training process, when the target loss function converges, the adjustment of the model parameters of the initial text recognition model is stopped, and the initial text recognition model trained to convergence is used as the target text recognition model.

[0150] In this embodiment, by using labeled text line image samples belonging to the target language to perform supervised model training on the initial text recognition model, the target text recognition model can have suitable model parameters, and then the target text recognition model can be used to more accurately recognize text belonging to the target language.

[0151] In practical applications, the common features of different languages ​​can be learned by using unlabeled text line image sample data of various languages ​​to perform unsupervised training on the initial text feature extraction model, and then the feature extraction layer of the initial text recognition model of the target language is constructed using the trained target text feature extraction model, and the initial text recognition model is supervised trained using labeled text line image samples of the target language. In this way, when there are not many labeled samples of a small language, the feature expression ability learned from other languages ​​in the feature extraction layer can be used to quickly transfer learning to obtain a target text recognition model that recognizes the target language. In addition, the target text recognition model of each language can learn the information contained in the samples of other languages ​​through self-supervision, thereby improving the accuracy of the target text recognition model in recognizing the corresponding target language text.

[0152] In some embodiments, Figure 5 and Figure 6 As shown, the training of the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language may also include:

[0153] Step S601: when the text line image samples in the second sample data set meet the preset sample conditions, obtaining the corpus text corresponding to the target language;

[0154] Step S603: generating text line image samples containing the corpus text according to the corpus text, and obtaining a supplementary sample data set of the second sample data set;

[0155] Step S605: using the second sample data set and the supplementary sample data set to train the initial text recognition model.

[0156] In this embodiment, the preset sample condition can be set as the text line image samples in the second sample data set are image samples of a preset minority language (such as Arabic or Urdu, etc.), or can be set as the number of text line image samples in the second sample data set is less than or equal to the preset number of samples. That is, when the text line image samples in the second sample data set are image samples of a preset minority language or when the number of text line image samples in the second sample data set is less than or equal to the preset number of samples, the corpus text corresponding to the target language is obtained.

[0157] The corpus text may refer to language material that actually appears in the actual use of the language.

[0158] Specifically, in step S601, a corpus may be queried according to the target language to obtain corpus text corresponding to the target language, or network information may be acquired using a network information acquisition tool according to the target language to obtain corpus text corresponding to the target language.

[0159] Of course, in this embodiment, other methods may also be used to obtain the corpus text corresponding to the target language, such as combining the above two corpus text acquisition methods to obtain the corpus text.

[0160] In step S603, generating a text line image sample containing the corpus text may include: generating at least two copies of the corpus text, the text fonts of the at least two copies are different, for example, the font of the text in the first copy is bold, and the font of the text in the second copy is regular; then, rendering each copy on a specified background to obtain a text line image corresponding to each copy. For example, the copies can be rendered on different backgrounds through multiple TTF (TrueTypeFont) font files to form a text line image.

[0161] Furthermore, when rendering each copy, a randomization process may be performed so that there are certain differences between the generated text line images. Model training based on such samples can improve the recognition accuracy of the target text recognition model. The randomization process includes at least one of the following: randomly adding noise such as Gaussian noise, salt and pepper noise, etc., randomly rotating such as small angle rotation, and randomly twisting the text line.

[0162] Furthermore, when generating the supplementary sample data set, the rendering process can be controlled, such as controlling the TTF font, the background used during rendering, etc., so that the font and background style of each supplementary sample in the generated supplementary sample data set fits the sample distribution in the second sample data set as much as possible.

[0163] In practical applications, for minority languages ​​where it is difficult to collect annotated samples, the sample data set used for training is usually very small, so it is difficult to train a target text recognition model with satisfactory accuracy. By combining the corpus text of the language to generate a virtual supplementary sample set for training, a target text recognition model with satisfactory accuracy can be quickly iterated when there are few annotated samples in the minority language.

[0164] In some embodiments, Figure 7 As shown, the obtaining of the corpus text corresponding to the target language (ie, step S601) may include:

[0165] Step S701: Acquire characters in the second sample data set whose occurrence frequency is less than or equal to a preset frequency;

[0166] Step S703: Acquire a corpus text corresponding to the target language, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

[0167] In this embodiment, the characters whose appearance frequency is less than or equal to the preset frequency may be rare characters or traditional characters.

[0168] In step S701, the second sample data set can be analyzed by a word frequency analysis tool to obtain the frequency of occurrence of each word in the second sample data set. The preset frequency can be set according to the performance requirements of the target text recognition model. The larger the preset frequency is set, the more corpus texts including rare characters or traditional characters can be obtained.

[0169] In practical applications, by acquiring text whose occurrence frequency is less than or equal to a preset frequency, the trained target text recognition model can have a better recognition effect on rare characters or traditional characters.

[0170] In some embodiments, Figure 8 As shown, the training of the initial text recognition model using the second sample data set and the supplementary sample data set (ie, step S605) may include:

[0171] Step S801: using the second sample data set to train the initial text recognition model at a first learning rate;

[0172] Step S803: using the supplementary sample data set, the initial text recognition model trained using the second sample data set is trained at a second learning rate, wherein the first learning rate is greater than the second learning rate.

[0173] In this embodiment, the second sample data set may be first used to train the initial text recognition model at a larger first learning rate, and then the supplementary sample data set may be used to train the initial text recognition model trained with the second sample data set at a smaller second learning rate.

[0174] In a specific embodiment, when the initial text recognition model is trained, the second sample data set can be used to first train the source domain model at a larger first learning rate (such as a basic learning rate of 0.01), and then the supplementary sample data set can be used to transfer the source domain model to the target domain of the supplementary sample data set at a smaller second learning rate (such as a basic learning rate of 0.001). In this way, it can be ensured that when the number of the supplementary sample data set is much smaller than that of the second sample data set, the trained target text recognition model can still fully learn the information in the supplementary sample data set.

[0175] Furthermore, after step S803, the second sample data set and the supplementary sample data set can also be used to jointly train the model trained using the supplementary sample data set. Specifically, the second sample data set and the supplementary sample data set can be used simultaneously to train the model trained using the supplementary sample data set at a smaller third learning rate (such as a basic learning rate of 0.001). In this way, it can be ensured that the trained model does not destroy the information learned in the second sample data set while learning the information in the supplementary sample data set.

[0176] Through this embodiment, a target text recognition model with higher robustness can be obtained. Compared with the target text recognition model trained using only the second sample data set according to the conventional method, the target text recognition model trained using the embodiment of the present disclosure has obviously better performance, and is superior to the text recognition model trained in the traditional way in terms of robustness and generalization.

[0177] Fig. 9 is a flowchart of a text recognition method according to an exemplary embodiment. Fig. 9 As shown, the method comprises the following steps:

[0178] Step S901: obtaining an image to be identified;

[0179] Step S903: determining the target language to which the image to be recognized belongs, and obtaining a target text recognition model corresponding to the target language;

[0180] Step S905: performing normalization processing on the image to be recognized to obtain a text line image corresponding to the image to be recognized;

[0181] Step S907: Use the target text recognition model to detect the text line image corresponding to the image to be recognized, and obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using the training method of the text recognition model described in any of the above embodiments.

[0182] In step S903, a language recognition model may be called to perform language recognition on the text on the image to be recognized, so as to obtain the target language to which the text on the image to be recognized belongs.

[0183] The language recognition model can be obtained by supervised training an initial language recognition model using a fourth sample data set. The fourth sample data set may include image samples belonging to different languages ​​and with language annotation information. The initial language recognition model may be a CNN model. In this way, the trained language recognition model can be used to recognize the target language of the text on the image to be recognized, and obtain a target text recognition model corresponding to the target language.

[0184] For example, when the text on the image to be recognized is in Portuguese, the language recognition model can be used to recognize that the target language of the text on the image to be recognized is Portuguese, and a target text recognition model for recognizing Portuguese text can be obtained.

[0185] In step S905, the image to be recognized may be normalized into a text line image sample with a fixed height and a variable length. For example, the image to be recognized may be normalized to a size of 32*X, wherein the height is fixed at 32 points and the width X may not be fixed.

[0186] In step S907, the text recognition result may be a string of characters, which is used to describe the text content contained in the image to be recognized.

[0187] In an experiment for this embodiment, the language corresponding to the text on the image to be recognized is Portuguese, and the target text recognition model is a model for recognizing Portuguese text.

[0188] Among them, the training process of the target text recognition model for recognizing Portuguese characters is to use the first sample data set to perform unsupervised training on the initial text feature extraction model to obtain the target text feature extraction model, wherein the first sample data set includes unlabeled text line image samples belonging to English and Portuguese. Since English and Portuguese both belong to the Latin language family, the target text feature extraction model is not updated again using text line image samples of English and Portuguese belonging to the same Latin language family; then, the initial text recognition model is constructed using the target text feature extraction model, and the initial text recognition model is supervisedly trained using the second sample data set, wherein the second sample data set includes labeled text line image samples belonging to Portuguese.

[0189] Through the experiment of this embodiment, it is found that the accuracy of the text recognition model obtained by directly using only the labeled text line image samples belonging to Portuguese for supervised training is 79.2% on the Portuguese test set, while the accuracy of the target text recognition model obtained by using the unsupervised training of this embodiment plus the supervised training is 87.0% (+7.8%) on the Portuguese test set. It can be seen that the text recognition method provided by this embodiment can improve the recognition accuracy of the text in the image to be recognized.

[0190] Fig.10 FIG. 1 is a block diagram of a training device for a text recognition model according to an exemplary embodiment. Fig.10 , the device comprises:

[0191] A first training data acquisition module 1010 is configured to execute acquisition of an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages;

[0192] The target text feature extraction model acquisition module 1020 is configured to perform unsupervised training on the initial text feature extraction model using the first sample data set to obtain a target text feature extraction model;

[0193] An initial text recognition model construction module 1030 is configured to execute construction of an initial text recognition model, wherein the initial text recognition model includes a text feature extraction layer, and the text feature extraction layer is determined according to the target text feature extraction model;

[0194] The second training data acquisition module 1040 is configured to acquire a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language;

[0195] The target text recognition model acquisition module 1050 is configured to train the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language.

[0196] Optionally, the initial text recognition model construction module includes:

[0197] A target language determination unit, configured to determine the target language to which the target language belongs;

[0198] A third sample data set acquisition unit is configured to acquire a third sample data set, wherein the third sample data set includes unlabeled text line image samples in different languages ​​under the target language family;

[0199] a target text feature extraction model updating unit, configured to perform unsupervised training on the target text feature extraction model using the third sample data set to obtain an updated target text feature extraction model;

[0200] The text feature extraction layer construction unit is configured to execute the text feature extraction layer of the initial text recognition model according to the updated target text feature extraction model.

[0201] Optionally, the target text recognition model acquisition module includes:

[0202] A corpus text acquisition unit, configured to acquire a corpus text corresponding to the target language when a text line image sample in the second sample data set meets a preset sample condition;

[0203] A supplementary sample data set acquisition unit is configured to generate a text line image sample containing the corpus text according to the corpus text, so as to obtain a supplementary sample data set for the second sample data set;

[0204] The initial text recognition model training unit is configured to train the initial text recognition model using the second sample data set and the supplementary sample data set.

[0205] Optionally, the corpus text acquisition unit includes:

[0206] A character acquisition subunit is configured to acquire characters in the second sample data set whose occurrence frequency is less than or equal to a preset frequency;

[0207] The corpus text acquisition subunit is configured to execute acquisition of corpus text corresponding to the target language, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

[0208] Optionally, the initial text recognition model training unit includes:

[0209] A first initial text recognition model training subunit is configured to train the initial text recognition model using the second sample data set at a first learning rate;

[0210] The second initial text recognition model training subunit is configured to use the supplementary sample data set to train the initial text recognition model trained using the second sample data set at a second learning rate, wherein the first learning rate is greater than the second learning rate.

[0211] Optionally, the target text feature extraction model acquisition module includes:

[0212] An image sample processing unit is configured to perform data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples;

[0213] The unsupervised training unit is configured to perform unsupervised training on the initial text feature extraction model using the first text line enhanced image sample and the second text line enhanced image sample, adopting an unsupervised algorithm based on contrast loss, to obtain a target text feature extraction model.

[0214] Optionally, the initial text recognition model further includes a text feature encoder and a text feature decoder, and the text line image samples in the second sample data set include text content description labels;

[0215] The target text recognition model acquisition module includes:

[0216] a text feature acquisition unit, configured to perform text feature extraction on the text line image samples in the second sample data set using the text feature extraction layer to obtain corresponding text features;

[0217] A text recognition result acquisition unit is configured to process the text features using the feature encoder and the feature decoder to obtain a corresponding text recognition result;

[0218] A target loss function determination unit is configured to determine a target loss function of the initial text recognition model based on the text recognition result and the text content description label;

[0219] The model parameter adjustment unit is configured to adjust the model parameters of the initial text recognition model based on the target loss function to obtain a target text recognition model for recognizing the target language.

[0220] Fig.11 is a block diagram of a text recognition device according to an exemplary embodiment. Fig.11 , the device comprises:

[0221] The to-be-recognized image acquisition module 1110 is configured to execute acquisition of the to-be-recognized image;

[0222] The target text recognition model acquisition module 1120 is configured to determine the target language to which the image to be recognized belongs, and acquire a target text recognition model corresponding to the target language;

[0223] The image processing module 1130 is configured to perform normalization processing on the image to be recognized to obtain a text line image corresponding to the image to be recognized;

[0224] The image detection module 1140 is configured to perform detection on the text line image corresponding to the image to be recognized using the target text recognition model to obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using any of the training methods of the text recognition models described above.

[0225] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0226] Fig.12 is a block diagram of an electronic device for model training or text recognition according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.12 As shown. The electronic device includes a processor, a memory, a model interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The model interface of the electronic device is used to communicate with an external terminal through a model connection. When the computer program is executed by the processor, a training method for a text recognition model or a text recognition method is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0227] Those skilled in the art will understand that Fig.12 The structure shown in the figure is merely a block diagram of a partial structure related to the scheme of the present disclosure, and does not constitute a limitation on the electronic device to which the scheme of the present disclosure is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0228] In an exemplary embodiment, an electronic device is also provided, including: a processor; a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement a training method or a text recognition method as in the embodiments of the present disclosure.

[0229] In an exemplary embodiment, a computer-readable storage medium is also provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the training method or text recognition method in the embodiment of the present disclosure.

[0230] In an exemplary embodiment, a computer program product including instructions is also provided. When the computer program product is run on a computer, the computer is enabled to execute the training method or the text recognition method in the embodiments of the present disclosure.

[0231] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0232] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0233] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for training a text recognition model, characterized in that: The method comprises: Acquire an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages, and the initial text feature extraction model learns common visual features of different languages ​​through the first sample data set; Using the first sample data set, performing unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model; Performing unsupervised training on the target text feature extraction model using a third sample data set to obtain an updated target text feature extraction model, wherein the third sample data set includes unlabeled text line image samples in different languages ​​under the target language family, and the target text feature extraction model learns visual features common to the target language and other languages ​​under the target language family except the target language through the third sample data set; Constructing an initial text recognition model, wherein a text feature extraction layer of the initial text recognition model is constructed according to the updated target text feature extraction model; Acquire a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language; The initial text recognition model is trained using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language.

2. The training method according to claim 1, characterized in that: Before performing unsupervised training on the target text feature extraction model using the third sample data set to obtain an updated target text feature extraction model, the method further includes: Determine the target language family to which the target language belongs; The third sample data set is obtained.

3. The training method according to claim 1, characterized in that: The step of training the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language includes: When the text line image samples in the second sample data set meet the preset sample conditions, obtaining the corpus text corresponding to the target language; generating, according to the corpus text, a text line image sample containing the corpus text, to obtain a supplementary sample data set of the second sample data set; The initial text recognition model is trained using the second sample data set and the supplementary sample data set.

4. The training method according to claim 3, characterized in that: The acquiring of the corpus text corresponding to the target language comprises: Acquire text in the second sample data set whose occurrence frequency is less than or equal to a preset frequency; A corpus text corresponding to the target language is obtained, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

5. The training method according to claim 3, characterized in that: The training of the initial text recognition model using the second sample data set and the supplementary sample data set includes: Using the second sample data set, training the initial text recognition model at a first learning rate; The initial text recognition model trained using the second sample data set is trained using the supplementary sample data set at a second learning rate, wherein the first learning rate is greater than the second learning rate.

6. The training method according to claim 1, characterized in that: The step of performing unsupervised training on the initial text feature extraction model using the first sample data set to obtain a target text feature extraction model comprises: Performing data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples; The first text line enhanced image sample and the second text line enhanced image sample are used, and an unsupervised algorithm based on contrast loss is adopted to perform unsupervised training on the initial text feature extraction model to obtain a target text feature extraction model.

7. The training method according to claim 1, characterized in that: The initial text recognition model further includes a text feature encoder and a text feature decoder, and the text line image samples in the second sample data set include text content description labels; The step of training the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language includes: Using the text feature extraction layer to extract text features from the text line image samples in the second sample data set to obtain corresponding text features; Processing the text features using the feature encoder and the feature decoder to obtain corresponding text recognition results; Determining a target loss function of the initial text recognition model based on the text recognition result and the text content description label; Based on the target loss function, the model parameters of the initial text recognition model are adjusted to obtain a target text recognition model for recognizing the target language.

8. A method for character recognition, characterized in that: The method comprises: Obtain an image to be recognized; Determine the target language of the image to be recognized, and obtain a target text recognition model corresponding to the target language; Normalizing the image to be recognized to obtain a text line image corresponding to the image to be recognized; The target text recognition model is used to detect the text line image corresponding to the image to be recognized to obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using the training method of the text recognition model described in any one of claims 1-7.

9. A training device for a text recognition model, characterized in that: The device comprises: A first training data acquisition module is configured to execute acquisition of an initial text feature extraction model and a first sample data set, wherein the first sample data set includes unlabeled text line image samples belonging to different languages, and the initial text feature extraction model learns common visual features of different languages ​​through the first sample data set; a target text feature extraction model acquisition module, configured to perform unsupervised training on the initial text feature extraction model using the first sample data set to obtain a target text feature extraction model; An initial text recognition model construction module is configured to execute construction of an initial text recognition model, wherein a text feature extraction layer of the initial text recognition model is constructed according to the updated target text feature extraction model; The initial text recognition model construction module includes: a target text feature extraction model updating unit, configured to perform unsupervised training on the target text feature extraction model using a third sample data set to obtain an updated target text feature extraction model, wherein the third sample data set includes unlabeled text line image samples of different languages ​​in the target language family, and the target text feature extraction model learns the visual features common to the target language and other languages ​​in the target language family except the target language through the third sample data set; A second training data acquisition module is configured to acquire a second sample data set, wherein the second sample data set includes labeled text line image samples belonging to the target language; The target text recognition model acquisition module is configured to train the initial text recognition model using the second sample data set to obtain a target text recognition model for recognizing text belonging to the target language.

10. The training device according to claim 9, characterized in that The initial text recognition model building module also includes: A target language determination unit, configured to determine the target language to which the target language belongs; The third sample data set acquiring unit is configured to acquire the third sample data set.

11. The training device according to claim 9, characterized in that: The target text recognition model acquisition module includes: A corpus text acquisition unit, configured to acquire a corpus text corresponding to the target language when a text line image sample in the second sample data set meets a preset sample condition; A supplementary sample data set acquisition unit is configured to generate a text line image sample containing the corpus text according to the corpus text, so as to obtain a supplementary sample data set for the second sample data set; The initial text recognition model training unit is configured to train the initial text recognition model using the second sample data set and the supplementary sample data set.

12. The training device according to claim 11, characterized in that The corpus text acquisition unit comprises: A character acquisition subunit is configured to acquire characters in the second sample data set whose occurrence frequency is less than or equal to a preset frequency; The corpus text acquisition subunit is configured to execute acquisition of corpus text corresponding to the target language, wherein the corpus text includes characters whose occurrence frequency is less than or equal to the preset frequency.

13. The training device according to claim 11, characterized in that The initial text recognition model training unit comprises: A first initial text recognition model training subunit is configured to train the initial text recognition model using the second sample data set at a first learning rate; The second initial text recognition model training subunit is configured to use the supplementary sample data set to train the initial text recognition model trained using the second sample data set at a second learning rate, wherein the first learning rate is greater than the second learning rate.

14. The training device according to claim 9, characterized in that The target text feature extraction model acquisition module includes: An image sample processing unit is configured to perform data enhancement processing on the text line image samples to be processed in the first sample data set to obtain first text line enhanced image samples and second text line enhanced image samples; The unsupervised training unit is configured to perform unsupervised training on the initial text feature extraction model using the first text line enhanced image sample and the second text line enhanced image sample, adopting an unsupervised algorithm based on contrast loss, to obtain a target text feature extraction model.

15. The training device according to claim 9, characterized in that The initial text recognition model further includes a text feature encoder and a text feature decoder, and the text line image samples in the second sample data set include text content description labels; The target text recognition model acquisition module includes: a text feature acquisition unit, configured to perform text feature extraction on the text line image samples in the second sample data set using the text feature extraction layer to obtain corresponding text features; A text recognition result acquisition unit is configured to process the text features using the feature encoder and the feature decoder to obtain a corresponding text recognition result; A target loss function determination unit is configured to determine a target loss function of the initial text recognition model based on the text recognition result and the text content description label; The model parameter adjustment unit is configured to adjust the model parameters of the initial text recognition model based on the target loss function to obtain a target text recognition model for recognizing the target language.

16. A text recognition device, characterized in that: The device comprises: The module for acquiring an image to be recognized is configured to acquire an image to be recognized; A target text recognition model acquisition module is configured to determine the target language to which the image to be recognized belongs and acquire a target text recognition model corresponding to the target language; The to-be-recognized image processing module is configured to perform normalization processing on the to-be-recognized image to obtain a text line image corresponding to the to-be-recognized image; The image detection module is configured to execute detection of the text line image corresponding to the image to be recognized using the target text recognition model to obtain a text recognition result corresponding to the image to be recognized, wherein the target text recognition model is trained using the training method of the text recognition model described in any one of claims 1-7.

17. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the training method as described in any one of claims 1 to 7 or the text recognition method as described in claim 8.

18. A computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the training method as described in any one of claims 1 to 7 or the text recognition method as described in claim 8.

19. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the training method described in any one of claims 1 to 7 or the text recognition method described in claim 8 is implemented.

Images