Method and apparatus for generating training samples of rare characters

Abstract

The present disclosure provides a method and device for generating training samples of rare characters, and relates to the technical field of image recognition. The specific implementation of the method comprises: in response to a service request sent by a client, issuing an information verification instruction to the client; obtaining a correction character input by a user through the client according to a verification result of a to-be-verified character returned by the client, and constructing a rare dictionary library by taking the correction character as a rare character; collecting a text corpus corresponding to the rare dictionary library, selecting a background image, a font format and a font color to render each text segment of the text corpus, and generating a training image; taking each text segment as an image label of the training image, and combining the training image and the image label to generate a training sample. The implementation can reduce the labor cost, learning cost, computing resource and development cost, the training data amount is sufficient, the recognition accuracy of the trained model is high, the convenience and efficiency of model training and use are improved, and the expansibility is strong.

Description

Technical Field

[0001] This disclosure relates to the field of image text recognition technology, and in particular to a method and apparatus for generating training samples of rare characters. Background Technology

[0002] Image text recognition technology is used to identify text in images. Commonly used technologies include OCR model recognition and deep learning model recognition.

[0003] In existing image text recognition processes, in order to ensure the accuracy of training data and improve the accuracy of text recognition, OCR models rely on text images from real scenes and manually annotated labels; deep learning models rely on the professional background of developers and sufficient computing resources, as well as quality checks on generated training images to eliminate low-quality data.

[0004] However, with the popularization of computer technology, the application scenarios of image text recognition are becoming more and more widespread, and the number of character categories that need to be recognized is increasing, especially rare characters and similar-looking characters. On the one hand, the training images of OCR models cannot cover all character categories, and the number of training images for different categories of characters varies, which will cause OCR to fail to recognize or have low accuracy in special recognition scenarios. Manual annotation will also consume too much manpower and have a high error rate, resulting in low recognition accuracy. On the other hand, deep learning models are too specialized and lack universality. Moreover, the training of the models consumes a lot of GPU computing resources, and the quality inspection and screening after generation not only requires extremely high manpower but also cannot guarantee a sufficient amount of training data. Summary of the Invention

[0005] In view of this, the present disclosure provides a method and apparatus for generating training samples of rare characters, which can solve the problems of unrecognizable, insufficient training samples, low model recognition accuracy, and excessive consumption of human resources, learning costs and computing resources for error-prone, easily confused, and obscure rare characters.

[0006] To achieve the above objectives, according to one aspect of this disclosure, a method for generating training samples of rare characters is provided, the method comprising:

[0007] In response to a business request sent by a client, an information verification instruction is sent to the client; wherein the information verification instruction includes the character to be verified;

[0008] Based on the verification result of the character to be verified returned by the client, the correction character input by the user through the client is obtained, and the correction character is used as the rare character to construct a rare dictionary library;

[0009] collect a text corpus corresponding to the rare dictionary, select a background image, a font format, and a font color to render each text segment of the text corpus to generate a training image;

[0010] use each text segment as an image label of the training image, and combine the training image and the image label to generate the training sample.

[0011] According to another aspect of the present disclosure, a device for generating a training sample of a rare character is provided, and the device includes:

[0012] The issuing module is configured to issue an information verification instruction to the client in response to a service request sent by the client, and the information verification instruction includes a to-be-verified character.

[0013] The obtaining module is configured to obtain a correction character input by the user through the client according to a verification result of the to-be-verified character returned by the client, and use the correction character as the rare character to construct a rare dictionary.

[0014] The rendering module is configured to collect a text corpus corresponding to the rare dictionary, select a background image, a font format, and a font color to render each text segment of the text corpus to generate a training image.

[0015] The generating module is configured to use each text segment as an image label of the training image, and combine the training image and the image label to generate the training sample.

[0016] According to still another aspect of the present disclosure, an electronic device is provided, and the electronic device includes:

[0017] a processor; and

[0018] a memory storing a program,

[0019] wherein the program includes instructions that, when executed by the processor, cause the processor to perform the method for generating a training sample of a rare character.

[0020] According to still another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, and the computer instructions are used to cause the computer to perform the method for generating a training sample of a rare character.

[0021] One or more technical solutions provided in the embodiments of the present application can realize the technical effects of improving generation efficiency, enriching training data of the recognition model, reducing consumption of computing resources, labor cost and the like, without learning cost, and can be simply and quickly realized in a CPU scenario, and improving accuracy of the text recognition model in recognizing rare characters. BRIEF DESCRIPTION OF DRAWINGS

[0022] In the following description of the example embodiments in conjunction with the accompanying drawings, more details, features and advantages of the present disclosure are disclosed, in the accompanying drawings:

[0023] Figure 1 A flowchart of a method for generating training samples of rare characters is shown according to an example embodiment of the present disclosure;

[0024] Figure 2 A flowchart of a method for acquiring rare characters of a full complement path is shown according to another example embodiment of the present disclosure;

[0025] Figure 3 A flowchart of a method for acquiring rare characters of an experience proofreading path is shown according to still another example embodiment of the present disclosure;

[0026] Figure 4 A flowchart of a method for rendering training images is shown according to an example embodiment of the present disclosure;

[0027] Figure 5 A flowchart of a method for determining a target image text recognition model is shown according to an example embodiment of the present disclosure;

[0028] Figure 6 A schematic block diagram of a device for generating training samples of rare characters is shown according to an example embodiment of the present disclosure;

[0029] Figure 7 A structural block diagram of an example electronic device that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0030] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms, and should not be interpreted as being limited to the embodiments set forth herein, but rather, these embodiments are provided to more thoroughly and completely understand the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for exemplary purposes only, and are not intended to limit the scope of protection of the present disclosure.

[0031] It should be understood that each of the steps recited in the method embodiments of the present disclosure can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present disclosure is not limited in this respect.

[0032] The term “comprises” and variations thereof used in the present disclosure are open-ended, that is, “comprising but not limited to.” The term “based on” is intended to mean “based, at least in part, on” The term “one embodiment” means “at least one embodiment”; the term “another embodiment” means “at least one additional embodiment”; the term “some embodiments” means “at least some embodiments.” Related definitions are given throughout the description below. It should be noted that references made in the present disclosure to “first”, “second”, etc. concepts merely serve to distinguish different apparatuses, modules, or units from each other, and are not intended to limit the order or interdependence of the functions performed by these apparatuses, modules, or units.

[0033] It should be noted that the modification of “one”, “multiple” mentioned in the present disclosure is illustrative rather than limiting, and those skilled in the art should understand that unless explicitly indicated otherwise in the context, it should be understood as “one or more”.

[0034] The names of the messages or information exchanged between the plurality of apparatuses in the embodiments of the present disclosure are only for illustrative purposes, and are not intended to limit the scope of the messages or information.

[0035] OCR: Optical Character Recognition, i.e. optical character recognition, is a process that uses computer vision and artificial intelligence technology to recognize characters in images and convert them into computer-readable text.

[0036] GPU: graphics processing unit, i.e. graphics processor, is a microprocessor used for image and graphics related operations.

[0037] RGB color mode: a color standard that obtains various colors by changing the three color channels of red (R), green (G), and blue (B) and superimposing them on each other.

[0038] In the existing OCR model and deep learning model identification process, due to the diversification of the identification scene, the OCR model needs to identify 10,000-20,000 types of characters, the accuracy of the OCR model depends on the training data of the real scene text image manually labeled, and is limited by the limited character categories covered by the real scene training image (for example, rare characters such as rare characters and similar characters are not completely covered), and the number of training images of various types of characters is uneven, which leads to low recognition accuracy of the OCR model for rare characters while consuming high labor cost. On the one hand, due to the professionalism and complexity of the learning algorithm, the learning cost of the deep learning model is too high, and the model training needs to consume high-cost GPU computing resources, and the adaptability and use convenience of the model are poor; on the other hand, the training image generated by the deep learning model also needs the quality inspection of the developer to determine whether it meets the training requirements, and to eliminate poor quality data, further increasing the labor and development cost, and at the same time, it may cause insufficient training data.

[0039] By the rare character training image generation method of the present disclosure, the labor cost, learning cost, computing resource and development cost of the training image can be reduced, the training image data is sufficient and the model trained based on the generated training image has high precision and high recognition accuracy, the model training and use convenience and efficiency are improved, and the applicability and expansibility are strong.

[0040] The scheme of the present disclosure is described below with reference to the accompanying drawings.

[0041] Figure 1 A flowchart of a rare character training sample generation method according to an exemplary embodiment of the present disclosure is shown as Figure 1 As shown in the figure, the rare character training sample generation method of the present disclosure includes the following steps:

[0042] In the embodiment of the present disclosure, the rare character training sample generation method of the present disclosure can be executed by a business server. Alternatively, the rare character training sample generation method of the present disclosure can be executed by a generation server other than the business server, and only the business server and the generation server need to interact, and the business server provides the rare character to the generation server.

[0043] Step S101, in response to a service request sent by a client, an information verification instruction is issued to the client; wherein the information verification instruction includes a to-be-verified character.

[0044] In the current situation of highly developed text recognition technology, the characters that are often misrecognized in the existing verification process are usually rare characters that are uncommon, easily confused or easily mistaken, such as rare characters. Since there is no training sample, the existing recognition model cannot recognize the rare characters. In the embodiments of the present disclosure, the method for generating training samples of rare characters uses the characters corrected by the user in the business request processing process as one of the paths for obtaining rare characters, constructs a rare dictionary library, and then generates training samples of rare characters. The training samples of rare characters can be improved to improve the recognition accuracy of the recognition model of rare characters.

[0045] Further, the business request can be a credit request, a repayment request, a term selection request, a query request, or any request involving an information verification step. When the information verification step of the business request is performed, the business server issues an information verification instruction to the client, so that the user verifies the to-be-verified characters that need to be verified in the business request. The incorrect characters in the to-be-verified characters are corrected to correct characters, so that the rare characters can be obtained based on the user correction path to construct a rare dictionary library.

[0046] Further, according to different user types, the to-be-verified characters are also different. For example, when the user type is personal, the to-be-verified characters can be name, transliteration, address, ID number, and native place. When the user type is an enterprise, the to-be-verified characters can be name, transliteration, registered address, trademark, official website address, unified social credit code, registration number, organization code, organization structure node, and employee name.

[0047] It should be noted that the "character" in the present disclosure can be a word, a phrase, etc., and is not limited to a single Chinese character or a letter. The "character" is only for the purpose of uniformity throughout the text to facilitate understanding of the scheme. Therefore, the rare dictionary library includes various characters such as Chinese characters, letters, words, phrases, etc., and is not limited to a single Chinese character.

[0048] Further, since the user uses various types of languages, the method for generating training samples of rare characters in the present disclosure is not limited to common language scenarios such as Chinese and English, but can also be applied to less commonly used language scenarios such as Korean, Japanese, and Arabic. It is especially suitable for minority language scenarios such as Zhuang, Xibe, Dai, Kyrgyz, Yi, Naxi, Miao, Lisu, and Wa.

[0049] In step S102, according to the verification result of the to-be-verified characters returned by the client, the correction characters input by the user through the client are obtained, and the correction characters are used as the rare characters to construct a rare dictionary library.

[0050] In the embodiments of the present disclosure, after the information verification instruction is issued, the service server determines whether the user corrects the information to be verified in the information verification step according to the verification result returned by the client in response to the information verification instruction, and takes the character corrected by the user as the rare character. The verification result includes corrected input and no corrected input. When the verification result is corrected input, the corrected character input by the user is taken as the rare character.

[0051] Further, before issuing the information verification instruction to the client, the service server first issues an information upload instruction to the client, and the user uploads the information image of various information through the client. The service server uses the experience recognition model that has been trained in history to recognize the information image, and obtains the text character in the information image as the aforementioned to-be-verified character. The experience recognition model includes an experience dictionary library, which can accurately recognize common characters in the information image and cannot accurately recognize rare characters in the information image. The result of recognizing the rare characters is still a common character. Therefore, all the text characters recognized by the experience recognition model are taken as the to-be-verified characters, and the user can correct the rare characters that cannot be accurately recognized as one of the source paths of the rare characters in the present disclosure.

[0052] Further, the rare characters are obtained by using the obtained rare characters to construct a rare dictionary library. In addition to the user correction path, the rare character acquisition path of the rare dictionary library also includes a complete search path and an experience correction path, as shown in the following table. Figure 2 The rare character acquisition method of the complete search path of another embodiment of the present disclosure includes the following steps:

[0053] Step S201, download a standard address library; wherein the standard address library includes a plurality of standard characters.

[0054] In the embodiments of the present disclosure, since there are many rare characters such as rare characters and similar characters in the address, in order to ensure the integrity and completeness of the rare characters in the rare dictionary library, the rare characters are obtained from the standard address library to supplement the rare dictionary library, so as to construct a complete rare dictionary library. The standard address library is obtained from a standard address website, for example, the standard address website is a public administrative division address website. The characters such as “dong”, “ge”, “lao”, and “mao” in the addresses such as “Ludong Village”, “Haokou Miao and Gelao Ethnic Township”, and “Maowa Village” are all rare characters.

[0055] Further, the standard address library stores the corresponding relationship between the character identifier and the standard character.

[0056] Step S202, a first standard character in the standard address library is selected as a current character, and the current character is compared with each character in the experience dictionary library and / or the rare dictionary library of the experience recognition model respectively.

[0057] In the embodiments of the present disclosure, each standard character in the standard address library is compared with each character in the experience dictionary library one by one, and it is determined whether each standard character exists in the experience dictionary library. If the standard character does not exist in the experience dictionary library, it indicates that the standard character is a rare character, and the rare character is added to the rare dictionary library. If the standard character exists in the experience dictionary library, it indicates that the standard character is a common character, and the next character is continued to be judged.

[0058] Meanwhile, each standard character in the standard address library is compared with each character in the rare dictionary library based on the user correction path that has been constructed, and it is determined whether each standard character exists in the rare dictionary library. If the standard character does not exist in the rare dictionary library, it indicates that the standard character is a rare character, and the rare character is added to the rare dictionary library.

[0059] Further, if the standard character exists in the rare dictionary library, the number of times that the standard character appears in the rare dictionary library is judged. If the number of times exceeds a preset number of times threshold, it indicates that the standard character has appeared in the rare dictionary library for many times, and the training data amount of the standard character in the rare dictionary library is sufficient and does not need to be added again, and the next character is continued to be judged.

[0060] Further, the character identifier of the current character is taken as a current identifier.

[0061] Step S203, it is judged whether the current character exists in the experience dictionary library and / or the rare dictionary library. If yes, go to step S204; if no, go to step S205.

[0062] Step S204, it is judged whether the number of times that the current character appears in the rare dictionary library is less than or equal to a preset number of times threshold. If yes, go to step S205; if no, go to step S206.

[0063] In the embodiments of the present disclosure, in the case that the current character only exists in the experience dictionary library, it indicates that the number of times that the current character appears in the rare dictionary library is 0, and the number of times is necessarily less than the number of times threshold, that is, the current character is a rare character.

[0064] In the case that the current character only exists in the rare dictionary library or the current character exists in the experience dictionary library and the rare dictionary library at the same time, the number of times that the current character appears in the rare dictionary library is counted, and it is judged whether the number of times is less than or equal to the number of times threshold. When the number of times is less than or equal to the number of times threshold, it indicates that the current character is a rare character; when the number of times is greater than the number of times threshold, it indicates that the training data amount of the current character in the rare dictionary library is sufficient.

[0065] Further, the frequency threshold value can be selectively set according to actual training requirements, for example, the frequency threshold value is 3.

[0066] In step S205, the current character is added to the rare dictionary library as a rare character, and the frequency of the current character in the rare dictionary library is incremented.

[0067] In the embodiment of the present disclosure, in the case that the current character does not exist in the experience dictionary library and the rare dictionary library, or the current character only exists in the experience dictionary library, or the current character only exists in the rare dictionary library and the frequency is less than or equal to the frequency threshold value, or the current character exists in the experience dictionary library and the rare dictionary library, and the frequency of the current character in the rare dictionary library is less than or equal to the frequency threshold value, it is indicated that the current character is a rare character, the current character is added to the rare dictionary library, and the frequency of the current character in the rare dictionary library is incremented, so as to facilitate subsequent addition judgment.

[0068] In step S206, the current character is updated to a standard character corresponding to a next identifier of the current identifier, and the process goes to step S202.

[0069] In the embodiment of the present disclosure, in the case that the current character exists in the experience dictionary library and the rare dictionary library, and the frequency of the current character in the rare dictionary library is greater than the frequency threshold value, or the current character is added as a rare character, the next character of the current character is taken as a judgment object to continue the judgment.

[0070] In the embodiment of the present disclosure, by using the method for acquiring rare characters of the complete supplement path of the present disclosure, in the case that there are many rare characters in the address, the standard characters in the public and authoritative address website are used as a supplement to expand the rare dictionary library to ensure the integrity of the rare dictionary library, so that the method for generating training samples of rare characters of the present disclosure can be applied to any recognition scene, and the recognition accuracy is high.

[0071] In the embodiment of the present disclosure, as shown in Figure 3 The method for acquiring rare characters of the experience correction path of another embodiment of the present disclosure includes the following steps:

[0072] In the embodiment of the present disclosure, as another supplement of the rare dictionary library, the business server compares the text result recognized by the experience recognition model with the actual text character, calculates the similarity to judge whether the recognized text result is accurate, and takes the character lower than the preset accuracy threshold value as a rare character to join the rare dictionary library, so as to further ensure the integrity and completeness of the rare dictionary library.

[0073] In step S301, historical training data of the experience recognition model is obtained; wherein the historical training data comprises historical labels of historical training images.

[0074] In the embodiments of the present disclosure, the historical training data can be used by the business server in the process of training the experience recognition model, or can be obtained from other public channels of experience recognition model training data, but all are historical training images with historical labels, so as to facilitate subsequent comparison of recognition accuracy.

[0075] In step S302, the historical training images are input into the experience recognition model to obtain current recognition results of the historical training images.

[0076] In the embodiments of the present disclosure, since the training process of the experience recognition model is controlled by a loss function, when the historical training images are input again into the trained experience recognition model, the current recognition results obtained can still be the same as or different from the historical labels, and the historical labels are definitely correct characters. Therefore, the experience correction path of the present disclosure compares the current recognition results with the historical labels to determine the historical labels with recognition errors, and the historical labels with recognition errors also indicate that the experience recognition model still cannot accurately recognize, so the historical labels are identified as rare characters that are easily confused, easily mistaken, or rare and added to the rare dictionary library to further expand the completeness of the rare dictionary library.

[0077] In step S303, the current recognition results are compared with the corresponding historical labels according to image identifiers of the historical training images, and recognition accuracy of the current recognition results is calculated.

[0078] In the embodiments of the present disclosure, the historical labels are accurate characters corresponding to the current recognition results of the historical training images, so the rare characters can be determined according to the similarity comparison results of the current recognition results and the historical labels.

[0079] In step S3031, cosine similarity of the current recognition results and the historical labels is calculated.

[0080] In the embodiments of the present disclosure, the cosine similarity is also called cosine similarity, which refers to estimating the similarity between two characters by calculating the cosine value of the included angle of two vectors. The greater the cosine similarity, the higher the similarity between the current recognition results and the historical labels; the smaller the cosine similarity, the lower the similarity between the current recognition results and the historical labels.

[0081] In step S3032, the minimum edit distance of the current recognition results and the historical labels is calculated.

[0082] In the embodiments of the present disclosure, the minimum edit distance (MED) refers to the number of edit operations required to convert one character into another, also known as Levenshtein distance, and the edit operations include insertion, deletion and replacement, for example, the edit operations of a Chinese character can be insertion, deletion and replacement of strokes. The minimum edit distance refers to the minimum number of edit operations required to convert one character into another, and accordingly, the greater the minimum number of edit operations, the lower the similarity between the two characters; the smaller the minimum number of edit operations, the higher the similarity between the two characters. That is, the greater the minimum edit distance, the lower the similarity between the current recognition result and the historical label; the smaller the minimum edit distance, the higher the similarity between the current recognition result and the historical label.

[0083] Further, if the current recognition result is completely the same as the historical label, the minimum edit distance is 0.

[0084] In step S3033, it is determined whether the cosine similarity is less than or equal to a preset similarity threshold and whether the minimum edit distance is greater than or equal to a preset distance threshold. If yes, go to step S3034; if no, go to step S3035.

[0085] In the embodiments of the present disclosure, by comparing the cosine similarity and the minimum edit distance at the same time, the recognition accuracy of the current recognition result compared with the historical label is determined. The greater the cosine similarity and the smaller the minimum edit distance, the more similar the current recognition result and the historical label, and the higher the recognition accuracy of the current recognition result; the smaller the cosine similarity and the greater the minimum edit distance, the greater the difference between the current recognition result and the historical label, and the lower the recognition accuracy of the current recognition result.

[0086] Further, the similarity threshold and the distance threshold can be selectively set according to actual training needs.

[0087] In step S3034, it is determined that the recognition accuracy of the current recognition result is lower than a preset accuracy threshold.

[0088] In step S3035, it is determined that the recognition accuracy of the current recognition result is higher than a preset accuracy threshold.

[0089] In step S304, the historical label with the recognition accuracy lower than the accuracy threshold is added to the rare character dictionary as a rare character.

[0090] In the embodiments of the present disclosure, the business server adds the historical label lower than the accuracy threshold to the rare character dictionary as a rare character to expand the rare character dictionary and improve the recognition accuracy.

[0091] In the embodiments of the present disclosure, the lower the recognition accuracy, the greater the deviation of the corresponding current recognition result from the historical label. If the accuracy is lower than the threshold, it means that the experience recognition model has a large recognition error deviation. Since the experience dictionary library of the experience recognition model stores common characters, the experience recognition model recognition error is used as a rare character.

[0092] In the embodiments of the present disclosure, by using the method for acquiring rare characters in the experience correction path of the present disclosure, the historical training image is identified again using the experience recognition model, and the current recognition result is compared with the historical label to determine the recognition accuracy of the experience recognition model. The experience recognition model that cannot be accurately recognized is added to the rare dictionary library as a rare character that is prone to error, easy to confuse, and rare. Further, the rare dictionary library is further expanded / supplemented, the completeness of the rare dictionary library is improved, and the recognition scene is expanded.

[0093] In step S103, a text corpus corresponding to the rare dictionary library is collected, and each text segment of the text corpus is rendered by selecting a background image, a font format, and a font color to generate a training image.

[0094] In the embodiments of the present disclosure, after the rare dictionary library is constructed, the training image can be generated using the rare characters. The generation of the training image requires the joint rendering of the background, the font format, and the font color. The renderer can be a text renderer (TextRenderer), or other commonly used renderers for text rendering. According to the actual training needs, the renderer can be selected.

[0095] In the embodiments of the present disclosure, as shown in Figure 4 The rendering method of the training image of the present disclosure includes the following steps:

[0096] In step S401, text data corresponding to the rare dictionary library is collected.

[0097] In the embodiments of the present disclosure, the data source of the text data can be business data or public data. The business data is the text data corrected by the user in the information verification step in response to the business request; and the public data is the text data corresponding to the rare characters obtained from the standard address library.

[0098] Further, the public data can also be text data in various scenes obtained from public sources, especially text data such as addresses and names collected from public sources.

[0099] In step S402, the text data is cut into a plurality of text segments according to a preset text symbol or character length to generate a text corpus.

[0100] In the embodiments of the present disclosure, the text symbol can be various punctuation marks, such as a comma “,”, a colon “:”, a semicolon “;”, and the like. The character length can be set according to actual training needs. After the text data is cut according to the text symbol or the character length, a plurality of text segments can be obtained. Since the text data is collected from the source path of rare characters, that is, business data or public data, the combination of the plurality of text segments can generate a text corpus.

[0101] Further, when the public data is text data in various scenes obtained from a public path, that is, the text data is collected from text data in various scenes, the cut text segments need to be selected, and the text segment combination including each rare character in the rare dictionary is selected as the text corpus.

[0102] Further, the style of the text segment can be a character, a single text line, or multiple lines of text, and the like, which can be cut according to actual training needs.

[0103] Step S403, obtaining a blank background image or an image including a blank region as a background image.

[0104] In the embodiments of the present disclosure, in order to ensure the high quality of the training sample of the rare character and improve the recognition accuracy of the rare character, the background image is selected as a blank background image or an image including a blank region, so as to reduce the interference of other patterns on the training sample. For example, the blank background image can be a landscape, a pure white picture, or the like, and the image including a blank region can be a picture with a cut blank region, a certificate or a business picture with text information erased, or the like.

[0105] Further, in order to ensure the quality of the training sample, the background image should be selected as rich as possible.

[0106] Step S404, selecting various font formats and font colors.

[0107] In the embodiments of the present disclosure, the selected font format and font color should cover as many commonly used font format types and font color types as possible. The font format can be obtained from an open source font, and the font color can be selected from the interval range of the RGB color space according to actual business needs or training needs.

[0108] Step S405, rendering each text segment of the text corpus to the background image according to various font formats and font colors to obtain a rendered image.

[0109] In the embodiments of the present disclosure, each text segment in the text corpus is rendered to the obtained background image according to the selected font format and font color to obtain a rendered image, wherein:

[0110] For the blank background image, each text segment of the text corpus is rendered to an arbitrary position of the background image.

[0111] For the image including a blank area, each text segment of the text corpus is rendered to the blank area of the background image.

[0112] Further, the rendered image can be a character image, a single text line image, or a multi-line text image, etc. corresponding to the style of the text segment.

[0113] Further, the text rendering can be run in a CPU scenario, which is simple, fast, and convenient to use.

[0114] In step S406, the rendered image is post-processed to generate the training image.

[0115] In the embodiments of the present disclosure, in order to improve the quality of the training samples, the rendered image is post-processed to simulate a real recognition scenario and improve the recognition accuracy of rare characters. The post-processing method can be selected according to actual training needs, such as Gaussian noise, salt and pepper noise, blur processing, text bending, transmission transformation, random scaling, random cropping, etc.

[0116] In the embodiments of the present disclosure, the order of steps S401-S402, S403, S404, generation of the text corpus, acquisition of the background image, selection of the font format and color, etc. is not limited by the rendering method of the training image of the present disclosure, and the order between steps S401-S402, S403, S404 can be adjusted at will.

[0117] In the embodiments of the present disclosure, by using the rendering method of the training image of the present disclosure, the text data corresponding to the rare dictionary library source path is cut to generate a text corpus, and a background image, a font format, and a color are selected for rendering, and a training image is obtained after post-processing. The training data for rare characters of the existing recognition model can be compensated, thereby facilitating subsequent recognition model training and use, and ensuring that rare characters in various scenarios can be recognized with high accuracy.

[0118] In step S104, each text segment is taken as an image label of the training image, and the training image and the image label are combined to generate the training sample.

[0119] In the embodiments of the present disclosure, using the text corpus corresponding to the rare dictionary library, each text segment is taken as an image label, and the training image rendered by combining the image label and the text segment corresponding to the image label is obtained to generate a training sample. The experience recognition model is trained to obtain a target image text recognition model that can recognize rare characters with high accuracy.

[0120] In the embodiments of the present disclosure, as shown in Figure 5 The method for determining the target image text recognition model of the present disclosure includes the following steps:

[0121] Step S501, obtaining the training sample.

[0122] In the embodiments of the present disclosure, the experience recognition model or historical training data from other sources can also be mixed into the training sample to further improve the scene adaptability and application accuracy of the target image text recognition model. For example, the text segments eliminated in step S402 are also rendered as training images and added to the training sample to improve the expansibility of the target image text recognition model.

[0123] Step S502, iteratively training the experience recognition model by taking the training image as input and the image label corresponding to the training image as output.

[0124] In the embodiments of the present disclosure, the model algorithm of the experience recognition model can be an OCR algorithm, a deep learning algorithm, or various algorithms suitable for image text recognition, which can be selected according to actual training needs.

[0125] Step S503, determining the target image text recognition model according to the training result of the iterative training.

[0126] In the embodiments of the present disclosure, by using the method for determining the target image text recognition model of the present disclosure, the training sample constructed by the rare dictionary library is used to train the experience recognition model, so as to obtain the target image text recognition model that can recognize rare characters and has extremely high recognition accuracy. In the subsequent use process, the image text is recognized by using the target image text recognition model, which can greatly improve the accuracy and application scene range of image text recognition, ensure the accurate recognition of rare characters, and improve the user experience.

[0127] Further, the use process of the target image text recognition model includes:

[0128] receiving an image recognition request sent by a client; wherein the image recognition request includes an image to be recognized;

[0129] inputting the image to be recognized into the target image text recognition model, and recognizing the image text in the image to be recognized according to the output of the target image text recognition model. Regardless of whether the image to be recognized contains rare characters, the target image text recognition model can accurately recognize the image text therein.

[0130] In the embodiments of the present disclosure, by means of the generation method of the training image of the rare character in the present disclosure, the problem that the rare character in the business is prone to error, confusion and rareness and cannot be recognized is solved. The rare character dictionary library is constructed by acquiring the rare character from different channels, and the training image is generated by using the text corpus corresponding to the channel. The image label corresponding to the training image and the text corpus is used as a training sample. Rich training samples can be generated in batches in the CPU scene, the training sample generation efficiency of the rare character is improved, the recognition model trained by using the training sample can recognize the rare character and has high recognition accuracy, no expensive GPU computing resource is needed, and the method is suitable for a wide range of scenarios and has strong expansibility.

[0131] Figure 6 FIG. 1 is a schematic diagram of main modules of a device for generating a training sample of a rare character according to an embodiment of the present disclosure, as shown in Figure 6 The device for generating a training sample of a rare character according to the present disclosure includes:

[0132] The issuing module 601 is configured to issue an information verification instruction to a client in response to a service request sent by the client, wherein the information verification instruction includes a to-be-verified character.

[0133] The obtaining module 602 is configured to obtain a correction character input by a user through the client according to a verification result of the to-be-verified character returned by the client, and construct a rare character dictionary library by using the correction character as the rare character.

[0134] The rendering module 603 is configured to collect a text corpus corresponding to the rare character dictionary library, and render each text segment of the text corpus by selecting a background image, a font format and a font color to generate a training image.

[0135] The generating module 604 is configured to combine the training image and the image label to generate the training sample by taking each text segment as an image label of the training image.

[0136] The present disclosure also provides an electronic device, including at least one processor and a memory connected with the at least one processor. The memory stores a computer program capable of being executed by the at least one processor, and the computer program is used to make the electronic device execute the method according to the embodiments of the present disclosure when executed by the at least one processor.

[0137] The present disclosure also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program is used to make a computer execute the method according to the embodiments of the present disclosure when executed by a processor of the computer.

[0138] Exemplary embodiments of this disclosure also provide a computer program product, including a computer program, wherein, when executed by a processor of a computer, the computer program is used to cause the computer to perform a method according to an embodiment of this disclosure.

[0139] refer to Figure 7 The present invention describes a structural block diagram of an electronic device 700 that can serve as a server or client of the present disclosure, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0140] like Figure 7 As shown, the electronic device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may also store various programs and data required for the operation of the device 700. The computing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0141] Multiple components in electronic device 700 are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information to electronic device 700. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 704 may include, but is not limited to, disk and optical disk. Communication unit 709 allows electronic device 700 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, wireless communication transceivers and / or chipsets, such as Bluetooth™ devices, WiFi devices, WiMa7 devices, cellular communication devices, and / or the like.

[0142] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above. For example, in some embodiments, Figures 1 to 5 The methods of the present disclosure can be implemented as a computer software program tangibly embodied in a machine readable medium, such as the storage unit 708. In some embodiments, portions or all of the computer program can be loaded and / or installed onto the electronic device 700 via the ROM 702 and / or the communication unit 709. In some embodiments, the computing unit 701 can be configured, by way of firmware or software modules stored in the ROM 702 or storage unit 708, to carry out the methods of the present disclosure. Figures 1 to 5

[0143] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be embodied on the machine, partially on the machine, partially on the machine and partially on a remote machine or entirely on the remote machine or server.

[0144] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0145] ​As used in this disclosure, the terms "machine-readable medium" and "computer- readable medium" refer to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal that can be used to provide machine instructions and / or data to a programmable processor.

[0146] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0147] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0148] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

Claims

1. A method for generating training samples of rare characters, characterized in that, include: In response to a business request sent by a client, an information verification instruction is sent to the client; wherein, the information verification instruction includes a character to be verified, which is obtained by an empirical recognition model; Based on the verification result of the character to be verified returned by the client, the correction character input by the user through the client is obtained, and the correction character is used as the rare character to construct a rare dictionary library; Download the standard address library; wherein the standard address library includes multiple standard characters; The first standard character in the standard address library is selected as the current character, and the current character is compared with each character in the experience dictionary library of the experience recognition model and / or the rare dictionary library. If the current character exists only in the rare dictionary, or if the current character exists in both the experience dictionary and the rare dictionary, count whether the number of times the current character appears in the rare dictionary is less than or equal to a preset threshold. The current character with a frequency less than or equal to the threshold is added as a rare character to the rare dictionary, and the frequency of the rare character corresponding to the current character in the rare dictionary is incremented. Collect a text corpus corresponding to the rare dictionary database, select background image, font format and font color to render each text segment of the text corpus, and generate training images; Each of the text fragments is used as an image label for the training image, and the training image and the image labels are combined to generate the training sample.

2. The generation method as described in claim 1, characterized in that, If the current character exists only in the experience dictionary, or if the current character does not exist in the experience dictionary and / or the rare dictionary, or if the current character appears less than or equal to the frequency threshold in the rare dictionary, the method further includes: The current character is added as a rare character to the rare dictionary, and the occurrence count of the rare character corresponding to the current character in the rare dictionary is incremented.

3. The generation method as described in claim 1, characterized in that, Also includes: Obtain the historical training data of the experience recognition model; wherein, the historical training data includes historical labels of historical training images; The historical training images are input into the empirical recognition model to obtain the current recognition result of the historical training images; Based on the image identifiers of the historical training images, the current recognition result is compared with the corresponding historical labels, the recognition accuracy of the current recognition result is calculated, and historical labels with recognition accuracy lower than the accuracy threshold are added to the rare dictionary as rare characters.

4. The generation method as described in claim 1, characterized in that, The process involves collecting a text corpus corresponding to the rare dictionary database, selecting background images, font formats, and font colors to render various text segments from the corpus, and generating training images, including: Collect text data corresponding to the rare dictionary database, and cut the text data into multiple text segments according to preset text symbols or character lengths to generate a text corpus; Get a blank background image or an image containing blank areas as the background image, and select various font formats and font colors; Each text fragment from the text corpus is rendered onto the background image according to the specified font format and font color to obtain a rendered image; The rendered image is post-processed to generate the training image.

5. The generation method as described in claim 4, characterized in that, The post-processing methods include: Gaussian noise, salt and pepper noise, blurring, text bending, transmission transformation, random scaling or random cropping.

6. The generation method as described in claim 1, characterized in that, Also includes: The empirical recognition model is iteratively trained by taking the training image as input and the image label corresponding to the training image as output. Based on the training results of the iterative training, a target image text recognition model is determined.

7. A device for generating training samples of rare characters, characterized in that, include: The delivery module is used to respond to a business request sent by the client and send an information verification instruction to the client; wherein, the information verification instruction includes a character to be verified, which is obtained by an empirical recognition model; The acquisition module is used to acquire the correction character input by the user through the client based on the verification result of the character to be verified returned by the client, and construct a rare dictionary library using the correction character as the rare character; download a standard address library; wherein the standard address library includes multiple standard characters; select the first standard character in the standard address library as the current character, and compare the current character with each character in the experience dictionary library of the experience recognition model and / or the rare dictionary library; if the current character exists only in the rare dictionary library, or if the current character exists in both the experience dictionary library and the rare dictionary library, count whether the occurrence frequency of the current character in the rare dictionary library is less than or equal to a preset frequency threshold; add the current character with a frequency less than or equal to the frequency threshold as a rare character to the rare dictionary library, and increment the occurrence frequency of the rare character corresponding to the current character in the rare dictionary library; The rendering module is used to collect a text corpus corresponding to the rare dictionary database, select background images, font formats and font colors to render each text segment of the text corpus, and generate training images. The generation module is used to use each of the text fragments as image labels for the training images, and to combine the training images and the image labels to generate the training samples.

8. An electronic device, comprising: processor; as well as Stored program memory, The program includes instructions that, when executed by the processor, cause the processor to perform a method for generating training samples of rare characters according to any one of claims 1-6.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method for generating training samples of rare characters according to any one of claims 1-6.

Images