Text recognition method and device and storage medium

Abstract

The invention provides a text recognition method and device and a storage medium. The method comprises the steps of obtaining a plurality of first text images; for each first text image, performing text recognition on the first text image by using a plurality of first models obtained by pre-training to obtain a plurality of text recognition results; under the condition that the multiple text recognition results meet a preset condition, the first text image serves as a to-be-processed image, and the preset condition comprises that the multiple text recognition results comprise different text recognition results; based on the multiple text recognition results, multiple first training data are synthesized, a second model used for text recognition is obtained through training by means of a first training set, and the first training set comprises the multiple first training data; and performing text recognition on each to-be-processed image by using the second model. According to the invention, the accuracy of character recognition and the labeling efficiency of the character data set can be improved.

Description

Technical Field

[0001] This invention relates to the fields of deep learning and optical character recognition (OCR), and specifically to a text recognition method, apparatus and storage medium. Background Technology

[0002] Deep learning-based text recognition methods require massive amounts of annotated text data. The quantity and quality of the annotated text directly affect the accuracy of text recognition. Generally, the more annotated text images there are, and the more accurate the annotations, the better the text recognition results.

[0003] The most common annotation method currently is to first predict the text image using a text recognition model, and then manually correct the prediction results. Training a text recognition model requires a large amount of annotated text, while manual annotation requires a lot of manpower, which is very inefficient. Furthermore, manually annotated text has some problems, such as the difficulty in annotating rare characters and special symbols (people may not recognize them, making annotation impossible). In addition, manual annotation is prone to errors when annotating similar characters.

[0004] Therefore, there is an urgent need for a text recognition method that can automatically identify text in text images, thereby improving the efficiency of text annotation and the accuracy of text recognition. Summary of the Invention

[0005] At least one embodiment of this application provides a text recognition method, apparatus, and storage medium that can improve the efficiency of text annotation and the accuracy of text recognition.

[0006] To solve the above-mentioned technical problems, this application is implemented as follows:

[0007] In a first aspect, embodiments of this application provide a text recognition method, including:

[0008] S1, acquire multiple first text images;

[0009] S2, for each first text image, multiple pre-trained first models are used to perform text recognition on the first text image respectively to obtain multiple text recognition results; if the multiple text recognition results meet preset conditions, the first text image is used as the image to be processed, the preset conditions include: the multiple text recognition results include different text recognition results;

[0010] S3, based on the multiple text recognition results, synthesize multiple sets of first training data, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data;

[0011] S4, using the second model, perform text recognition on each image to be processed.

[0012] Optional, also includes:

[0013] If all the text recognition results are the same, the text recognition result shall be taken as the final text recognition result of the first text image.

[0014] Optionally, the preset condition also includes: the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed a preset ratio.

[0015] Optional, also includes:

[0016] If the multiple text recognition results do not meet the preset conditions, the first text recognition result shall be taken as the final text recognition result of the first text image.

[0017] Optionally, based on the multiple text recognition results, multiple sets of first training data are synthesized, including:

[0018] The multiple text recognition results are deduplicated, and the deduplicated text recognition results are used as foreground text. The foreground text and the background image are then combined to obtain a sample image.

[0019] The text recognition results synthesized into the sample image are used as the text annotation information of the sample image to obtain the first training data.

[0020] Optionally, S4 includes:

[0021] Using the second model, text recognition is performed on each image to be processed to obtain the second text recognition result and its confidence level;

[0022] If the confidence level is greater than or equal to the first threshold, the second text recognition result is taken as the final text recognition result of the image to be processed.

[0023] If the confidence level is less than or equal to the second threshold, the image to be processed is indicated as failing to be recognized.

[0024] If the confidence level is less than the first threshold but greater than the second threshold, the image to be processed is added to the first image library.

[0025] Optionally, the above methods also include:

[0026] If the number of images in the first image library is greater than a preset number, steps S2-S4 are executed for the images in the first image library to obtain the final text recognition result of the images in the first image library.

[0027] Optionally, the above methods also include:

[0028] Text annotation data is generated using all the obtained first text images and their final text recognition results.

[0029] Optionally, the above methods also include:

[0030] Obtain multiple sets of second training data, which are training data from a public dataset used for text recognition;

[0031] A pre-selected text sample is used as the foreground text, and the foreground text is combined with the background image to obtain a sample image; the text recognition result synthesized into the sample image is used as the text annotation information of the sample image to obtain the third training data;

[0032] Based on the multiple sets of second training data and / or the multiple sets of third training data, multiple second training sets are generated;

[0033] The plurality of first models are trained using the plurality of second training sets, wherein each second training set trains one first model.

[0034] Optionally, when compositing the foreground text and background image to obtain the sample image, the method further includes:

[0035] The sample image is enhanced by at least one of the following methods: setting the font of the foreground text, deforming the foreground text, deleting pixels in the sample image, adding a preset pattern to the sample image, and adjusting the color of the background image.

[0036] Optionally, the above methods also include:

[0037] Receive configuration information input by the user;

[0038] Based on the configuration information, select the synthesis processing method to be used in the synthesis process.

[0039] Optionally, the configuration information includes the business scenario and / or text recognition accuracy; selecting the synthesis processing method based on the configuration information includes at least one of the following:

[0040] Based on the business scenario, select an image relevant to the business scenario as the background image;

[0041] Based on the text recognition accuracy, determine the number of background images used in the synthesis process and / or the type of enhancement method used in the enhancement process.

[0042] Secondly, embodiments of this application provide a text recognition device, including:

[0043] The information sending and receiving module is used to acquire multiple first text images;

[0044] The text recognition module is used to perform text recognition on each first text image using multiple pre-trained first models to obtain multiple text recognition results; if the multiple text recognition results meet preset conditions, the first text image is used as the image to be processed, and the preset conditions include: the multiple text recognition results include different text recognition results;

[0045] The model generation module is used to synthesize multiple sets of first training data based on the multiple text recognition results, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data.

[0046] The text recognition module is also used to perform text recognition on each image to be processed using the second model.

[0047] According to a third aspect of this application, at least one embodiment provides a text recognition device, including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method as described in any of the first aspects.

[0048] According to a fourth aspect of this application, at least one embodiment provides a computer-readable storage medium storing a program that, when executed by a processor, implements the steps of the method as described in any of the first aspects.

[0049] According to a fifth aspect of this application, at least one embodiment provides a computer program product including computer instructions that, when executed by a processor, implement the steps of the method as described in any of the first aspects.

[0050] Compared with existing technologies, the text recognition method and apparatus provided in this application can automatically and quickly perform text recognition on multiple text images, improving the efficiency of text annotation and the accuracy of text recognition. When applied to text annotation, the above method can improve text annotation efficiency, reduce annotation error rate, improve the quality of text annotation data, and thus improve the accuracy of the text recognition model trained based on the above text annotation data. Attached Figure Description

[0051] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0052] Figure 1 This is a flowchart of a text recognition method according to an embodiment of this application;

[0053] Figure 2 This is an example diagram of a text recognition method according to another embodiment of this application;

[0054] Figure 3 This is a schematic diagram illustrating the generation of the second training set according to an embodiment of this application;

[0055] Figure 4 This is a schematic diagram of the structure of a text recognition device according to an embodiment of this application;

[0056] Figure 5 This is another structural schematic diagram of the text recognition device according to an embodiment of this application;

[0057] Figure 6 This is a schematic diagram of the structure of a text recognition system according to an embodiment of the present invention;

[0058] Figure 7 This is another structural schematic diagram of the text recognition device according to an embodiment of this application;

[0059] Figure 8 The computer described in this application is an exemplary hardware structure.

[0060] Figure 9 This is an exemplary hardware structure of a mobile terminal according to an embodiment of this application;

[0061] Figure 10 This is an example diagram illustrating the workflow of a text recognition system according to an embodiment of this application.

[0062] Figure 11 An example of a user interface displayed on a client in an embodiment of this application. Detailed Implementation

[0063] To make the technical problems, technical solutions, and advantages of this application clearer, a detailed description will be provided below in conjunction with the accompanying drawings and specific embodiments. In the following description, specific details such as particular configurations and components are provided merely to aid in a comprehensive understanding of the embodiments of this application. Therefore, those skilled in the art should understand that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Furthermore, for clarity and brevity, descriptions of known functions and structures have been omitted.

[0064] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus. The word "and / or" in the specification and claims indicates at least one of the connected objects.

[0065] In the various embodiments of this application, it should be understood that the sequence number of each process described below does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0066] The following description provides examples and is not intended to limit the scope, applicability, or configuration set forth in the claims. Changes may be made to the function and arrangement of the elements discussed without departing from the spirit and scope of this disclosure. Various procedures or components may be appropriately omitted, substituted, or added to the examples. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Furthermore, features described with reference to certain examples may be combined in other examples.

[0067] This application provides a text recognition method that can automatically recognize text from multiple text images, improving the efficiency of text annotation and the accuracy of text recognition. The text includes, but is not limited to, language characters from various countries or regions, as well as various symbols. The text recognition method can be applied to either the server side or the terminal side. Figure 1 A flowchart illustrating the text recognition method described in this application is provided. Figure 2 Here is a specific example of applying the above method. For example... Figure 1 As shown, the text recognition method includes the following steps:

[0068] Step S1: Obtain multiple first text images.

[0069] Here, the first text image is the image that needs to be text-recognized, and it typically contains text information. In this embodiment, the user can upload multiple first text images to the server or terminal through a user interface. This embodiment can automatically perform text recognition processing on these text images to obtain text recognition results. (Reference) Figure 2 For example, several first text images are: an image containing the text "peeling fruit shells", an image containing the text "sighing and following", an image containing the text "chilly", and so on.

[0070] Step S2: For each first text image, use multiple pre-trained first models to perform text recognition on the first text image to obtain multiple text recognition results; if the multiple text recognition results meet preset conditions, use the first text image as the image to be processed, the preset conditions include: the multiple text recognition results include different text recognition results.

[0071] Here, for each first text image, the embodiments of this application perform the following processing:

[0072] Each first model is used to perform text recognition on the first text image, obtaining the text recognition result generated by each first model, thus yielding multiple text recognition results. Here, the first model is a pre-trained text recognition model. Then, if the multiple text recognition results meet preset conditions, for example, when the multiple text recognition results include different text recognition results, the first text image is used as the image to be processed. Specifically, the first text image can be added to a set of images to be processed. In this method, the first text image is used as the image to be processed as long as the multiple text recognition results include at least two different text recognition results. If the multiple text recognition results are all the same, then that text recognition result is used as the final text recognition result for the first text image.

[0073] refer to Figure 2 For example, there are N first models, designated as first model 1 to N. These first models are used to perform text recognition on each first text image to obtain the text recognition result. Taking an image containing the text "peeling fruit shells" as an example, the recognition result of first model 1 is "peeling fruit shells", the recognition result of first model 2 is "peeling fruit husks", and so on. Since at least first model 1 and first model 2 will have different recognition results, the image containing the text "peeling fruit shells" is used as the image to be processed.

[0074] This application embodiment can also set other more flexible preset conditions. For example, the preset conditions may include: the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed the preset ratio. In this case, not only is it required that the multiple text recognition results include at least two different text recognition results, but the proportion of each type of text recognition result among the multiple text recognition results is further calculated, and when the first text recognition result with the largest proportion does not exceed the preset ratio, the first text image is used as the image to be processed. However, if the multiple text recognition results do not meet the preset conditions, that is, when the first text recognition result with the largest proportion exceeds the preset ratio, then the first text recognition result is used as the final text recognition result of the first text image.

[0075] Through the above processing, embodiments of this application can obtain one or more images to be processed and at least two corresponding text recognition results.

[0076] Step S3: Based on the multiple text recognition results, synthesize multiple sets of first training data, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data.

[0077] Here, in this embodiment, the multiple text recognition results can first be deduplicated to obtain deduplicated text recognition results, and then multiple sets of first training data can be synthesized based on the deduplicated text recognition results. Specifically, the synthesis method can be as follows: the deduplicated text recognition results are used as foreground text, and the foreground text is synthesized with the background image to obtain a sample image. Then, the text recognition results synthesized in the sample image are used as the text annotation information of the sample image to obtain the first training data. Each set of first training data includes a sample image and its corresponding text annotation information. Of course, multiple sets of first training data can also be synthesized directly based on the multiple text recognition results without performing the above deduplication process; this embodiment does not specifically limit this approach.

[0078] In addition, when synthesizing the foreground text and the background image to obtain the sample image, the embodiments of this application can perform enhancement processing on the sample image, and the enhancement processing includes at least one of the following enhancement methods:

[0079] (1) Set the font of the foreground text, including but not limited to: setting the font type, font color, font size, adding an underline to the font, and making the font bold;

[0080] (2) Transform the foreground text, including but not limited to: stretching the text, shrinking the text, rotating the text, etc.

[0081] (3)Delete pixel points in the sample image, including but not limited to: deleting some pixel points of foreground text, deleting some pixel points of the background image, etc.

[0082] (4)Add a preset pattern to the sample image, including but not limited to: adding a straight line (the direction of the straight line can be vertical, horizontal, inclined, etc.) to the sample image, adding a curve to the sample image, adding a watermark pattern to the sample image, etc.

[0083] (5)Adjust the color of the background image, including but not limited to: changing the color of the background image, and the color can be the same color or a gradient color, etc.

[0084] It should be noted that the background image in the embodiment of the present application can be at least one image uploaded by the user as the background image; it can also be a background image selected according to the business scenario selected by the user. At this time, the selected background image is an image related to the business scenario. For example, in the financial field, the background image can be a receipt image, an invoice image, etc.; in the logistics field, the background image can be a logistics document image, an express bill image, etc.

[0085] In addition, the embodiment of the present application can also set the number and / or image enhancement method of the background images participating in synthesis according to the annotation accuracy selected by the user. For example, when the user selects a higher annotation accuracy, more background images and / or more enhancement methods can be used to generate more first training data; when the user selects a lower annotation accuracy, fewer background images and / or fewer enhancement methods can be used to generate fewer first training data.

[0086] In this way, the embodiment of the present application can construct a first training set based on the synthesized multiple first training data. The first training set includes the multiple first training data and can also include other training data, such as various publicly available training data for text recognition, etc. The embodiment of the present application does not make specific limitations on this. Based on the first training set, the embodiment of the present application can train a second model. The second model can be trained based on a pre-trained language model.

[0087] Reference Figure 2 In the example of, for the text recognition result "peel the fruit shell", a set of first training data is generated; for the text recognition result "peel the fruit husk", a set of first training data is also generated, and so on. In this way, multiple first training data can be obtained, and then the second model can be trained using the first training data.

[0088] Step S4, use the second model to perform text recognition on each image to be processed.

[0089] Here, in the embodiments of the present application, the second model trained in S3 is used to perform text recognition on each image to be processed, and a second text recognition result and its confidence are obtained. For a certain image to be processed:

[0090] If the confidence of the second text recognition result of the image to be processed is greater than or equal to the first threshold, then the second text recognition result is used as the final text recognition result of the image to be processed;

[0091] If the confidence of the second text recognition result of the image to be processed is less than or equal to the second threshold, it is prompted that the recognition of the image to be processed fails. At this time, the image to be processed can be discarded (because its image quality is low and the text recognition effect is poor);

[0092] If the confidence of the second text recognition result of the image to be processed is less than the first threshold and greater than the second threshold, then the image to be processed is added to the first image library. The first threshold is greater than the second threshold.

[0093] Reference Figure 2 In the example of, when using the second model to recognize an image to be processed containing the text content "peel the fruit shell", the obtained text recognition result is "peel the fruit shell" and the confidence is 0.99. Assuming it is greater than or equal to the first threshold, at this time, "peel the fruit shell" is used as the final text recognition result of the above-mentioned image to be processed. Similarly, when recognizing an image to be processed containing the text content "sigh along with", the obtained text recognition result is "peel the fruit shell" and the confidence is 0.98, which is greater than the first threshold. At this time, "sigh along with" is used as the final text recognition result of the above-mentioned image to be processed.

[0094] As can be seen from the above, in the case where the text recognition results of multiple first models for text images vary greatly, the embodiments of the present application further use these greatly different text recognition results to generate training data and train a second model, and then use the second model to re-recognize these text images. Since the second model is trained based on greatly different text recognition results, it can better recognize these text images, thereby improving the accuracy of text recognition. In addition, since automatic recognition is performed through the model, the embodiments of the present application can improve the efficiency of text annotation and the accuracy of character recognition.

[0095] When executing on the multiple first text images Figure 1After the processing shown, a first image library can be obtained. The first image library stores first text images whose confidence levels for text recognition results are between a first threshold and a second threshold. If the number of images in the first image library is greater than a preset number, steps S2-S4 can be executed for the images in the first image library to obtain the final text recognition results for the images in the first image library.

[0096] The text recognition method described in this application can be applied to the field of text annotation. For example, the text recognition method is used to recognize the text image to be annotated, which is the first text image mentioned above. Then, text annotation data is generated using all the obtained first text images and their final text recognition results. It can be seen that when the text recognition method described in this application is applied to text annotation, it can automatically annotate a large number of text images, thereby improving text annotation efficiency, reducing the annotation error rate, improving the quality of text annotation data, and thus improving the accuracy of the text recognition model trained based on the above text annotation data.

[0097] The training of the first model in the embodiments of this application will now be described.

[0098] Prior to step S1 above, the method described in this application embodiment may further include the following steps:

[0099] Step a: Obtain multiple sets of second training data, which are training data from a public dataset used for text recognition.

[0100] Here, the publicly available dataset can be publicly available data provided by various institutions or organizations, such as Wikipedia. Specifically, publicly available text recognition datasets can be collected, and the text data annotation format can be converted to the same format, such as {'image_name','label_txt'}.

[0101] Step b: Use the pre-selected text sample as the foreground text, and synthesize the foreground text with the background image to obtain a sample image; use the text recognition result synthesized into the sample image as the text annotation information of the sample image to obtain the third training data.

[0102] Here, the pre-selected text sample can be chosen according to the specific business scenario of the application. For example, commonly used text in the business scenario can be used as the text sample. This application embodiment does not specifically limit this. Similarly, when compositing the foreground text and the background image to obtain a sample image, this application embodiment can also enhance the sample image. The enhancement processing includes at least one of the following enhancement methods: setting the font of the foreground text, deforming the foreground text, deleting pixels in the sample image, adding a preset pattern to the sample image, adjusting the color of the background image, etc.

[0103] For example, based on business scenarios, a large corpus of business-related text formats is collected. Data synthesis methods are used to synthesize the text corpus into corresponding images. At this time, the synthesized data format is also {'image_name','label_txt'}. The synthesized data is the labeled data. In the data synthesis method, firstly, background images related to the business are collected as the background of the synthesized data. Secondly, various data augmentation methods need to be added, such as randomly adding font styles, deleting image pixels, adding horizontal or vertical lines, and slightly bending the text, etc., to generate simulated image augmentation methods and generate simulated data.

[0104] Step c: Generate multiple second training sets based on the multiple sets of second training data and / or the multiple sets of third training data.

[0105] Here, the multiple sets of second training data and / or the multiple sets of third training data can be randomly divided into multiple second training sets. The specific division method is not specifically limited in this embodiment of the application. Figure 3 An example is provided for generating multiple second training sets, such as second training set 1 to N.

[0106] Step d: Using the multiple second training sets, train the multiple first models, wherein each second training set trains one first model.

[0107] Here, the first model can also be trained on the basis of a pre-trained language model. By using each second training set, a first model can be trained to obtain a first model, thus obtaining multiple first models.

[0108] During the training process of the first and second models described above, this embodiment of the application can receive configuration information input by the user, and then select the synthesis processing method adopted by the synthesis processing according to the configuration information.

[0109] Specifically, the configuration information may include the business scenario and / or text recognition accuracy. For example, the user can input the above configuration information through a UI interface. Then, based on the business scenario, an image related to the business scenario is selected as the background image; and / or, based on the text recognition accuracy, the number of background images used in the synthesis process and / or the types of enhancement methods used in the enhancement process are determined. Generally, when the user selects a higher annotation accuracy, more background images and / or more enhancement methods can be used to generate more first training data / third training data; when the user selects a lower annotation accuracy, fewer background images and / or fewer enhancement methods can be used to generate less first training data / third training data.

[0110] Based on the above methods, this application also provides an apparatus for implementing the above methods. Please refer to [link / reference]. Figure 4 The text recognition device provided in this application includes:

[0111] Information transceiver module 31 is used to acquire multiple first text images;

[0112] The text recognition module 32 is used to perform text recognition on each first text image using multiple pre-trained first models to obtain multiple text recognition results; when the multiple text recognition results meet preset conditions, the first text image is used as the image to be processed, and the preset conditions include: the multiple text recognition results include different text recognition results;

[0113] The model generation module 33 is used to synthesize multiple sets of first training data based on the multiple text recognition results, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data.

[0114] The text recognition module 32 is also used to perform text recognition on each image to be processed using the second model.

[0115] Through the above modules, the embodiments of this application can improve the efficiency of text annotation and the accuracy of text recognition.

[0116] Optionally, the text recognition module 32 is further configured to: when the multiple text recognition results are all the same, use the text recognition result as the final text recognition result of the first text image.

[0117] Optionally, the preset condition also includes: the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed a preset ratio.

[0118] Optionally, the text recognition module 32 is further configured to: when the plurality of text recognition results do not meet the preset conditions, use the first text recognition result as the final text recognition result of the first text image.

[0119] Optionally, the model generation module 33 is further configured to:

[0120] The multiple text recognition results are deduplicated, and the deduplicated text recognition results are used as foreground text. The foreground text and the background image are then combined to obtain a sample image.

[0121] The text recognition results synthesized into the sample image are used as the text annotation information of the sample image to obtain the first training data.

[0122] Optionally, the text recognition module 32 is further configured to:

[0123] Using the second model, text recognition is performed on each image to be processed to obtain the second text recognition result and its confidence level;

[0124] If the confidence level is greater than or equal to the first threshold, the second text recognition result is taken as the final text recognition result of the image to be processed.

[0125] If the confidence level is less than or equal to the second threshold, the image to be processed is indicated as failing to be recognized.

[0126] If the confidence level is less than the first threshold but greater than the second threshold, the image to be processed is added to the first image library.

[0127] Optionally, the above-mentioned device further includes:

[0128] The recognition control module is used to, when the number of images in the first image library is greater than a preset number, call the text recognition module and the model generation module to process the images in the first image library to obtain the final text recognition result of the images in the first image library.

[0129] Optionally, the above-mentioned device further includes:

[0130] The data storage module is used to generate text annotation data using all the obtained first text images and their final text recognition results.

[0131] Optionally, the model generation module 33 is further configured to:

[0132] Multiple sets of second training data are obtained, which are training data from a public dataset used for text recognition; a pre-selected text sample is used as foreground text, and the foreground text is synthesized with a background image to obtain a sample image; the text recognition result synthesized into the sample image is used as the text annotation information of the sample image to obtain third training data; multiple second training sets are generated based on the multiple sets of second training data and / or the multiple sets of third training data; multiple first models are trained using the multiple sets of second training data, wherein each second training set trains one first model.

[0133] Optionally, the model generation module 33 is further configured to enhance the sample image when the foreground text and background image are synthesized to obtain the sample image. The enhancement process includes at least one of the following enhancement methods: setting the font of the foreground text, deforming the foreground text, deleting pixels in the sample image, adding a preset pattern to the sample image, and adjusting the color of the background image.

[0134] Optionally, the information transceiver module 31 is also used to receive configuration information input by the user;

[0135] The model generation module 33 is also used to select the synthesis processing method adopted by the synthesis processing according to the configuration information.

[0136] Optionally, selecting the synthesis processing method based on the configuration information includes at least one of the following:

[0137] Based on the business scenario, select an image relevant to the business scenario as the background image;

[0138] Based on the text recognition accuracy, determine the number of background images used in the synthesis process and / or the type of enhancement method used in the enhancement process.

[0139] It should be noted that the various devices / systems provided in the above embodiments are devices corresponding to the above text recognition method. The implementation methods in each of the above embodiments are applicable to the embodiments of this device and can achieve the same technical effect. The device provided in this application embodiment can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0140] Please refer to Figure 5 This application also provides a hardware structure block diagram of a text recognition device, such as... Figure 5 As shown, the text recognition device 400 includes:

[0141] Processor 402; and

[0142] Memory 404, in which computer program instructions are stored.

[0143] When the computer program instructions are executed by the processor, the processor 402 performs the following steps:

[0144] S1, acquire multiple first text images;

[0145] S2, for each first text image, multiple pre-trained first models are used to perform text recognition on the first text image respectively to obtain multiple text recognition results; if the multiple text recognition results meet preset conditions, the first text image is used as the image to be processed, the preset conditions include: the multiple text recognition results include different text recognition results;

[0146] S3, based on the multiple text recognition results, synthesize multiple sets of first training data, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data;

[0147] S4, using the second model, perform text recognition on each image to be processed.

[0148] It should be noted that the systems provided in the above embodiments are devices corresponding to the above text recognition methods. The implementation methods in each of the above embodiments are applicable to the embodiments of this device and can achieve the same technical effect. The device provided in this application can implement all the method steps implemented in the above method embodiments and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiments and the beneficial effects will not be described in detail.

[0149] Furthermore, such as Figure 5 As shown, the text recognition device 400 also includes a network interface 401, an input device 403, a hard disk 405, and a display device 406.

[0150] The various interfaces and devices described above can be interconnected via a bus architecture. The bus architecture can include any number of interconnecting buses and bridges. Specifically, various circuits representing one or more central processing units (CPUs) and / or graphics processing units (GPUs), as represented by processor 402, and one or more memories, as represented by memory 404, are connected together. The bus architecture can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. It is understood that the bus architecture is used to implement communication between these components. In addition to the data bus, the bus architecture also includes a power bus, a control bus, and a status signal bus, which are well known in the art and will not be described in detail herein.

[0151] The network interface 401 can be connected to a network (such as the Internet, a local area network, etc.), receive data such as first text images and / or configuration information from the network, and save the received data to the hard disk 405.

[0152] The input device 403 can receive various instructions input by the operator and send them to the processor 402 for execution. The input device 403 may include a keyboard or a clicking device (e.g., a mouse, trackball, touchpad, or touchscreen).

[0153] The display device 406 can display the results obtained by the processor 402 executing instructions, such as displaying the model training progress.

[0154] The memory 404 is used to store programs and data necessary for the operation of the operating system, as well as intermediate results and other data during the calculation process of the processor 402.

[0155] It is understood that the memory 404 in the embodiments of this application may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory may be random access memory (RAM), which is used as an external cache. The memory 404 of the apparatus and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0156] In some implementations, memory 404 stores elements such as executable modules or data structures, or subsets thereof, or extended sets thereof: operating system 4041 and application program 4042.

[0157] The operating system 4041 includes various system programs, such as the framework layer, core library layer, and driver layer, used to implement various basic business functions and handle hardware-based tasks. The application program 4042 includes various applications, such as a browser, used to implement various application functions. Programs implementing the methods of this application embodiment can be included in application program 4042.

[0158] The methods disclosed in the above embodiments of this application can be applied to processor 402, or implemented by processor 402. Processor 402 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 402 or by instructions in the form of software. The processor 402 may be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory 404, and processor 402 reads the information in memory 404 and completes the steps of the above method in combination with its hardware.

[0159] It is understood that the embodiments described herein can be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers, microprocessors, other electronic units for performing the functions described herein, or combinations thereof.

[0160] For software implementation, the techniques described herein can be achieved through modules (e.g., procedures, functions, etc.) that perform the functions described herein. The software code can be stored in memory and executed by a processor. The memory can be implemented within the processor or externally.

[0161] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0162] If all the text recognition results are the same, the text recognition result shall be taken as the final text recognition result of the first text image.

[0163] Optionally, the preset condition also includes: the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed a preset ratio.

[0164] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0165] If the multiple text recognition results do not meet the preset conditions, the first text recognition result shall be taken as the final text recognition result of the first text image.

[0166] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0167] The multiple text recognition results are deduplicated, and the deduplicated text recognition results are used as foreground text. The foreground text and the background image are then combined to obtain a sample image.

[0168] The text recognition results synthesized into the sample image are used as the text annotation information of the sample image to obtain the first training data.

[0169] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0170] Using the second model, text recognition is performed on each image to be processed to obtain the second text recognition result and its confidence level;

[0171] If the confidence level is greater than or equal to the first threshold, the second text recognition result is taken as the final text recognition result of the image to be processed.

[0172] If the confidence level is less than or equal to the second threshold, the image to be processed is indicated as failing to be recognized.

[0173] If the confidence level is less than the first threshold but greater than the second threshold, the image to be processed is added to the first image library.

[0174] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0175] If the number of images in the first image library is greater than a preset number, steps S2-S4 are executed for the images in the first image library to obtain the final text recognition result of the images in the first image library.

[0176] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0177] Text annotation data is generated using all the obtained first text images and their final text recognition results.

[0178] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0179] Obtain multiple sets of second training data, which are training data from a public dataset used for text recognition;

[0180] A pre-selected text sample is used as the foreground text, and the foreground text is combined with the background image to obtain a sample image; the text recognition result synthesized into the sample image is used as the text annotation information of the sample image to obtain the third training data;

[0181] Based on the multiple sets of second training data and / or the multiple sets of third training data, multiple second training sets are generated;

[0182] The plurality of first models are trained using the plurality of second training sets, wherein each second training set trains one first model.

[0183] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0184] When compositing the foreground text and the background image to obtain a sample image, the sample image is enhanced. The enhancement process includes at least one of the following enhancement methods: setting the font of the foreground text, deforming the foreground text, deleting pixels in the sample image, adding a preset pattern to the sample image, and adjusting the color of the background image.

[0185] Specifically, when the computer program is executed by the processor 402, it can also perform the following steps:

[0186] Receive configuration information input by the user;

[0187] Based on the configuration information, select the synthesis processing method to be used in the synthesis process.

[0188] Specifically, the configuration information includes the business scenario and / or text recognition accuracy; when the computer program is executed by the processor 402, it can also perform the following steps:

[0189] Based on the business scenario, select an image relevant to the business scenario as the background image;

[0190] Based on the text recognition accuracy, determine the number of background images used in the synthesis process and / or the type of enhancement method used in the enhancement process.

[0191] It should be noted that the device provided in this application embodiment can implement all the method steps implemented in the above method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0192] Figure 6 The schematic diagram of the text recognition system 800 provided in the embodiment of the present invention includes: a client 830 and a text recognition device 820. Figure 7 A specific module diagram of the text recognition device 820 is further provided. For example... Figure 7 As shown, the text recognition device 820 includes an information sending and receiving module 823, a text recognition module 822, a data storage module 821, a model generation module 825, and a user interface control module 824.

[0193] The client 830 can be a personal computer or mobile terminal (such as a smartphone, PAD, etc.), or an application running on the aforementioned terminals. A mobile terminal is a user-operated terminal device. A mobile terminal can be a smartphone, a personal digital assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a wearable device, or a next-generation communication system, such as a terminal device in an NR network or a terminal device in a future evolved Public Land Mobile Network (PLMN) network.

[0194] The text recognition device 820 operates a second model 804 and multiple first models, such as... Figure 6 The numbers 801 to 803 shown represent first model A, first model B, and first model C, respectively. The first and second models are generated by the model generation module 825. The first and second models can be obtained by training a pre-trained language model using a relevant training set. The text recognition device 820 can be located on the same computer device, or it can be a server system composed of multiple computers, in which case the various modules can be located on different computer devices. The model generation module 825 can be located within the text recognition device 820, or it can be located on a different computer device.

[0195] In the text recognition system 800 described above, the client 830 can connect to the text recognition device 820 via a wired and / or wireless network. The functions of both the client 830 and the text recognition device 820 can be distributed across multiple computers.

[0196] The following describes the hardware or software structure of the relevant equipment, devices, or functions.

[0197] <Hardware Structure>

[0198] computer:

[0199] The client 830 and the text recognition device 820 communicate via, for example, having Figure 8 The hardware structure shown is then materialized into a computer. Figure 8 An exemplary hardware structure of a computer according to an embodiment of this application is shown.

[0200] refer to Figure 8 The computer 500 includes an input device 501, a display device 502, an external I / F 503, RAM 504, ROM 505, a CPU 506, a communication I / F 507, an HDD 508, etc., interconnected via a bus B. It is acceptable to form a structure in which the input device 501 and the display device 502 are connected when necessary.

[0201] Input device 501 includes a keyboard, mouse, touchpad, etc., through which users input various operation signals. Display device 502 includes a monitor, etc., to display the processing results obtained by computer 500.

[0202] Communication I / F 507 is an interface configured to enable computer 500 to connect to various networks. Therefore, computer 500 performs data communication via communication I / F 507.

[0203] HDD 508 is an exemplary non-volatile storage device for storing programs and data. The stored data includes the operating system (OS) that serves as the foundational software controlling the entire computer 500, application software (also referred to herein as "applications") that provides various functionalities within the OS, and so on. Computer 500 may use a drive device that uses flash memory (e.g., a solid-state drive (SSD)) as the storage medium instead of HDD 508.

[0204] External I / F 503 is an interface for external devices. These external devices include recording media 503a, etc. In this case, computer 500 reads information from and / or writes information to recording media 503a via external I / F 503. Recording media 503a may be a floppy disk, CD, DVD, SD memory card, USB storage device, etc.

[0205] ROM 505 is a non-volatile semiconductor memory (storage device) that retains programs and / or data even when the power is off. ROM 505 stores programs and data used to execute the Basic Input / Output System (BIOS), OS settings, network settings, etc., at the time of power-on of computer 500. RAM 504 is an example of a volatile semiconductor memory (storage device) for temporary storage of programs and / or data.

[0206] CPU 506 is an algorithmic device that reads programs and / or data from storage devices such as ROM 505 and HDD 508. The read program or read data performs a process, thereby materializing the control or functional capabilities of the entire computer 500.

[0207] The client 830 and the text recognition device 820 communicate via, for example Figure 8 The hardware structure of the computer 500 shown is materialized.

[0208] Mobile terminal:

[0209] For example, the client 830 via, as Figure 9 The hardware structure shown is then materialized. Figure 9 An exemplary hardware structure of the mobile terminal in this embodiment is shown. Figure 9 The mobile terminal 12 shown includes a CPU 601, ROM 602, RAM 603, EEPROM 604, CMOS sensor 605, acceleration and orientation sensor 606, and media driver 608.

[0210] The CPU 601 controls the entire operation of the mobile terminal 12. The ROM 602 stores basic input and output programs. The RAM 603 is used as the working area of ​​the CPU 601. The EEPROM 604 reads or writes data corresponding to the control of the CPU 601. The CMOS sensor 605 captures image data corresponding to the control of the CPU 601 to obtain image data. The acceleration and orientation sensor 606 is an electromagnetic compass, rotary compass, acceleration sensor, etc., that detects the Earth's magnetic field.

[0211] Media drive 608 controls the reading or writing (storage) of data from or sent to recordable medium 607, such as flash memory. Data already stored in recordable medium 607 is read out, or new data is written to recordable medium 607. Recordable medium 607 is freely attachable to or detachable from media drive 608.

[0212] EEPROM 604 stores the operating system (OS) executed by CPU 601, as well as related information necessary for network settings. Applications for performing various processes of the first embodiment are stored in EEPROM 604, recordable medium 607, etc.

[0213] The CMOS sensor 605 is a charge-coupled device that converts light into electrical charges and digitizes an image of an object. The CMOS sensor 605 can be materialized, for example, by a charge-coupled device (CCD) sensor, as long as it can capture an image of the object.

[0214] In addition, the mobile terminal 12 includes an audio input unit 609, an audio output unit 610, an antenna 611, a communication unit 612, a wireless LAN communication unit 613, a wireless communication antenna 614, a wireless communication unit 615, a display 616, a touchpad 617, and a bus 619.

[0215] Audio input unit 609 converts sound into audio signals. Audio output unit 610 converts audio signals into sound. Communication unit 612 uses antenna 611 to communicate with the nearest base station device via wireless communication signals. Wireless LAN communication unit 613 performs wireless LAN communication with the access point in accordance with the IEEE 80411 standard. Wireless communication unit 615 performs wireless communication using wireless communication antenna 614.

[0216] Display 616 is configured to display images, various icons, etc. Display 616 is made of liquid crystal, organic EL, etc. Touchpad 617 is mounted on display 616 and is formed of a pressure-sensitive plate or an electrostatic plate. Touch positions on display 616 are detected by touch with a finger or stylus. Bus 619 is an address bus, data bus, etc., electrically connecting the aforementioned units or components.

[0217] Client 830 includes a dedicated battery 618. Client 830 is powered by battery 618. Audio input unit 609 includes a microphone for inputting sound. Audio output unit 610 includes a loudspeaker for outputting sound.

[0218] For example, client 830 via, as Figure 9 The hardware structure shown is then materialized.

[0219] The following is combined with Figure 10 Taking a text annotation application scenario as an example, an example diagram of the workflow of the aforementioned text recognition system 800 is provided. In this workflow, the text recognition method of this application embodiment is used for automated text annotation, thereby improving annotation efficiency and accuracy. The workflow specifically includes:

[0220] In S901, the user interface control module 824 of the text recognition device 820 instructs the information transceiver module 823 to send user interface data to the client 830 and display the user interface on the display device of the client 830.

[0221] Figure 11 An example of a client-side user interface 831 is provided, through which a user can upload files. Taking a text annotation application scenario as an example, the user can select an image file to be annotated for uploading. Additionally, the user interface 831 also provides options for annotation precision and business scenario. Since this example uses text annotation as an illustration, therefore... Figure 11 The annotation precision used in this example is the text annotation precision (corresponding to the text recognition precision mentioned earlier). Figure 11 The annotation precision options include high, medium, and low, and more or fewer levels can be set as needed. Table 1 provides an example of the number of background images and image enhancement methods corresponding to different annotation precision levels. It can be seen that higher precision usually allows for the use of more types of background images to synthesize training data, and / or the use of more types of image enhancement methods to enhance the image data in the training data, thus potentially requiring a longer annotation processing time. The user interface can prompt the user about the differences between the different precision levels.

[0222] Table 1

[0223] Annotation precision Number of background images Image enhancement methods … high 50 Text bending + adding horizontal and vertical lines + random font + deleting pixels … middle 20 Text curvature + add horizontal and vertical lines … Low 10 Text Curved …

[0224] in addition, Figure 11 The business scenario options include finance, human resources, logistics, etc. Based on the business scenario selected by the user, this embodiment of the application can select a background image related to the business scenario to generate training data for training the first model and the second model. Additionally, the image enhancement method for the sample images can be set according to the business scenario selected by the user. Table 2 provides a correspondence between business scenarios, background images, and image enhancement methods, which can be stored in the data storage module 821.

[0225] Table 2

[0226] Business Scenarios Background image Image enhancement methods … finance Invoice background 1.png, Invoice background 2.png, ... Add horizontal and vertical lines … personnel ID card background.png, Residence permit background.png,… Delete pixel … logistics Courier slip background A.png, Courier slip background B.png,… Text bending + random font … … … … …

[0227] In S902, users set annotation configuration information as needed, such as annotation accuracy and business scenario, and upload multiple sets of data to be annotated (such as multiple images to be annotated). The client 830 sends the data to be annotated and the annotation configuration information set by the user through the user interface to the text recognition device 820.

[0228] In S903, for each piece of data to be labeled, the text recognition module 822 uses multiple first models A, B, C, etc., to predict the text information of the data to be labeled, and instructs the data storage module 821 to save the prediction results to the first prediction result list. Table 3 provides an example of the number of background images and data augmentation methods corresponding to different levels of labeling accuracy. The data ID to be labeled is used to represent the ID of each piece of data to be labeled, and the prediction results of each first model for each piece of data to be labeled are shown in each row of Table 3.

[0229] Table 3

[0230] Data ID to be labeled Prediction results of Model A Prediction results of Model B Prediction results of Model C … 0001 AAAAA AAAaa aaAAA … 0002 BBBBB BBBBB BBBBB … 0003 CCCCC CCCCC ccCCC … 0004 DDDDD DDDDd dDDDD … … … … … …

[0231] In S904, it is determined whether the text information of the prediction results of all the first models is the same. If they are all the same, proceed to S905; otherwise, proceed to S906. Here, it is assumed that all prediction results are required to be the same, that is, the proportion of the prediction result with the largest proportion must be equal to 100%.

[0232] In S905, the text recognition module 822 uses the text information of the prediction result as the annotation information of the data to be labeled, and instructs the data storage module 821 to update the annotation result list.

[0233] For example, taking Table 3 as an example, the prediction results of all the first models for the data to be labeled with ID 0002 are the same, all being "BBBBB". In this case, "BBBBB" can be used as the labeling information for the data to be labeled and saved in the labeling result list. Table 4 provides an example of a labeling result list.

[0234] Table 4

[0235]

[0236]

[0237] In S906, the text recognition module 822 determines images related to the business scenario as candidate images based on the business scenario in the annotation configuration information, and determines the number of background images and the image enhancement method based on the annotation accuracy information in the annotation configuration information. Then, based on the determined number of background images, it selects at least one background image from the candidate images, combines the text information of the prediction result with the background image to form a sample image, and enhances the sample image according to the determined image enhancement method.

[0238] In S907, the data storage module 821 uses text information as annotations for sample images and saves them in the first training set corresponding to the sample images.

[0239] Taking the data to be labeled shown in Table 3 as an example, the prediction results for the data with ID 0001 are not entirely the same. Similarly, the prediction results for the data with ID 0003 and ID 0004 are not entirely the same. For the above data to be labeled, each prediction result is synthesized with a background image to obtain a sample image file, and the prediction result is used as the annotation text for the sample image file, thus obtaining the first training set as shown in Table 5. If the user selects "high" annotation precision, then for each prediction result, the prediction result is synthesized with 50 background images to generate 50 sample image files.

[0240] Table 5

[0241]

[0242]

[0243] In S908, the text recognition module 822 sends the first training set to the model generation module 810 to train the second model.

[0244] In S909, the text recognition module 822 uses the trained second model to predict the text information of the data to be labeled and obtains the confidence information of the prediction. Then, it instructs the data storage module 821 to save the predicted text information and its confidence information to the second prediction result list. Table 6 provides an example of the second prediction result list.

[0245] Table 6

[0246] Data ID to be labeled The text predicted by the second model Confidence 0001 AAAAA 0.99 0003 ccCCC 0.55 0004 dDDDD 0.12 … … …

[0247] In S910, the text recognition module 822 determines the processing result of the data to be predicted based on the confidence level of the prediction result and the setting of the confidence threshold, and instructs the data storage module 821 to update the annotation result list. For example, assuming that the first and second thresholds of the confidence threshold are 0.9 and 0.5 respectively, there are different processing methods as shown in Table 7.

[0248] Table 7

[0249] Confidence Processing results ≥0.90 Label >0.50 and <0.90 Repeat processing ≦≤0.50 Abandon labeling

[0250] The text recognition module 822 will re-execute the processing steps S903 to S910 on the data to be labeled that has been processed with the result of "repeated processing" until all the data to be labeled has been labeled or the labeling is abandoned. Table 8 provides an example of the updated labeling result list.

[0251] Table 8

[0252]

[0253]

[0254] In S911, a method is provided to notify the user that annotation is complete and to provide a way to download the annotated data. For example, an email or text message can be sent to the user carrying a download link. The user can then download the annotated data using the download link. Alternatively, the annotated data can be sent directly to the user.

[0255] In some embodiments of this application, a computer-readable storage medium is also provided, on which a program is stored, which, when executed by a processor, performs the following steps:

[0256] S1, acquire multiple first text images;

[0257] S2, for each first text image, use multiple pre-trained first models to perform text recognition on the first text image respectively to obtain multiple text recognition results; if the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed a preset proportion, the first text image is used as the image to be processed, and the multiple text recognition results are deduplicated.

[0258] S3. Using the text recognition results after deduplication, multiple sets of first training data are synthesized, and a second model for text recognition is trained using the first training set, wherein the first training set includes the multiple sets of first training data.

[0259] S4, using the second model, perform text recognition on each image to be processed.

[0260] When executed by the processor, this program can implement all the above-mentioned text recognition methods and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0261] This application also provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the various processes of the above-described text recognition method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0262] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0263] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0264] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0265] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0266] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0267] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0268] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A text recognition method, characterized in that, Including the following steps: S1, acquire multiple first text images; S2, For each first text image, use multiple pre-trained first models to perform text recognition on the first text image respectively, and obtain multiple text recognition results; If the multiple text recognition results meet the preset conditions, the first text image is taken as the image to be processed. The preset conditions include: the multiple text recognition results include different text recognition results. S3, based on the multiple text recognition results, synthesize multiple sets of first training data, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data; S4, using the second model, perform text recognition on each image to be processed.

2. The method as described in claim 1, characterized in that, Also includes: If all the text recognition results are the same, the text recognition result shall be taken as the final text recognition result of the first text image.

3. The method as described in claim 1, characterized in that, The preset condition also includes: the proportion of the first text recognition result with the largest proportion among the multiple text recognition results does not exceed the preset ratio.

4. The method as described in claim 3, characterized in that, Also includes: If the multiple text recognition results do not meet the preset conditions, the first text recognition result shall be taken as the final text recognition result of the first text image.

5. The method as described in claim 1, characterized in that, Based on the multiple text recognition results, multiple sets of first training data are synthesized, including: The multiple text recognition results are deduplicated, and the deduplicated text recognition results are used as foreground text. The foreground text and the background image are then combined to obtain a sample image. The text recognition results synthesized into the sample image are used as the text annotation information of the sample image to obtain the first training data.

6. The method as described in claim 1, characterized in that, S4 includes: Using the second model, text recognition is performed on each image to be processed to obtain the second text recognition result and its confidence level; If the confidence level is greater than or equal to the first threshold, the second text recognition result is taken as the final text recognition result of the image to be processed. If the confidence level is less than or equal to the second threshold, the image to be processed is indicated as failing to be recognized. If the confidence level is less than the first threshold but greater than the second threshold, the image to be processed is added to the first image library.

7. The method as described in claim 6, characterized in that, Also includes: If the number of images in the first image library is greater than a preset number, steps S2-S4 are executed for the images in the first image library to obtain the final text recognition result of the images in the first image library.

8. The method as described in claim 1, characterized in that, Also includes: Text annotation data is generated using all the obtained first text images and their final text recognition results.

9. The method as described in claim 1, characterized in that, Also includes: Obtain multiple sets of second training data, which are training data from a public dataset used for text recognition; A pre-selected text sample is used as the foreground text, and the foreground text is combined with the background image to obtain a sample image; the text recognition result synthesized into the sample image is used as the text annotation information of the sample image to obtain the third training data; Based on the multiple sets of second training data and / or the multiple sets of third training data, multiple second training sets are generated; The plurality of first models are trained using the plurality of second training sets, wherein each second training set trains one first model.

10. The method as described in claim 5 or 9, characterized in that, When compositing the foreground text and the background image to obtain a sample image, the method further includes: The sample image is enhanced by at least one of the following methods: setting the font of the foreground text, deforming the foreground text, deleting pixels in the sample image, adding a preset pattern to the sample image, and adjusting the color of the background image.

11. The method as described in claim 10, characterized in that, Also includes: Receive configuration information input by the user; Based on the configuration information, select the synthesis processing method to be used in the synthesis process.

12. The method as described in claim 11, characterized in that, The configuration information includes the business scenario and / or text recognition accuracy; the step of selecting the synthesis processing method based on the configuration information includes at least one of the following: Based on the business scenario, select an image relevant to the business scenario as the background image; Based on the text recognition accuracy, determine the number of background images used in the synthesis process and / or the type of enhancement method used in the enhancement process.

13. A text recognition device, characterized in that, include: The information sending and receiving module is used to acquire multiple first text images; The text recognition module is used to perform text recognition on each first text image using multiple pre-trained first models to obtain multiple text recognition results. If the multiple text recognition results meet the preset conditions, the first text image is taken as the image to be processed. The preset conditions include: the multiple text recognition results include different text recognition results. The model generation module is used to synthesize multiple sets of first training data based on the multiple text recognition results, and use the first training set to train a second model for text recognition, wherein the first training set includes the multiple sets of first training data. The text recognition module is also used to perform text recognition on each image to be processed using the second model.

14. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 12.

15. A computer program product, characterized in that, Includes computer instructions that, when executed by a processor, implement the steps of the method as described in any one of claims 1 to 12.

Images