A method and apparatus for model training

Abstract

This specification discloses a method and apparatus for model training. The method involves acquiring images to be labeled, inputting each image into a text line recognition model to obtain a text line recognition result, and inputting the image into a preset single-character recognition model to obtain a single-character recognition result. Then, based on the single-character recognition result, the text line recognition result is corrected to obtain a corrected text line recognition result. Finally, based on the corrected text line recognition result, the image to be labeled is labeled to obtain a labeled image. The labeled image is then used to train the text line recognition model. This method combines single-character recognition results and text line recognition results, effectively improving the accuracy of data labeling, thereby increasing the accuracy of the text line recognition model.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a method and apparatus for model training. Background Technology

[0002] With the continuous development of electronic and network technologies, text recognition algorithms have made great progress. Currently, training a high-precision text recognition model requires a large amount of finely labeled data for supervision, and preparing a labeled dataset that meets the requirements is extremely costly in terms of both time and manpower.

[0003] In existing technologies, unlabeled data is input into two different character recognition models, which output two recognition results. A discriminator uses one of the recognition results as data annotation, and then uses the annotated data to train the character recognition model. However, the discriminator cannot accurately select the correct recognition result. Therefore, when the recognition results output by the two character recognition models are incorrect, the data annotation will be incorrect, thereby reducing the accuracy of the character recognition model.

[0004] Therefore, improving the accuracy of data annotation, thereby increasing the accuracy of text recognition models, is an urgent problem to be solved. Summary of the Invention

[0005] This specification provides a method, apparatus, storage medium, and electronic device for model training, in order to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This manual provides a method for model training, including:

[0008] Obtain each image to be labeled;

[0009] For each image to be labeled, the image is input into the text line recognition model to be trained to obtain the text line recognition result, and the image to be labeled is input into the preset single character recognition model to obtain the single character recognition result;

[0010] Based on the single character recognition result, the text line recognition result is corrected to obtain the corrected text line recognition result;

[0011] Based on the corrected text line recognition results, the image to be labeled is labeled to obtain the labeled image;

[0012] The text line recognition model to be trained is trained using the annotated image.

[0013] Optionally, the image to be labeled is input into the text line recognition model to be trained to obtain the text line recognition result, which specifically includes:

[0014] The image to be labeled is input into the text line recognition model to be trained, and the text line information contained in the image to be labeled is obtained by the text line recognition model, as well as the position of the text line information in the image to be labeled.

[0015] The position of the text line and the information of the text line are used as the text line recognition result;

[0016] The image to be labeled is input into a preset single-character recognition model to obtain single-character recognition results, specifically including:

[0017] The image to be labeled is input into the single-character recognition model to obtain the information of each single character contained in the image to be labeled as recognized by the single-character recognition model, as well as the position of each single character in the image to be labeled.

[0018] The single-character information and the position of each single-character information in the image to be labeled are used as the single-character recognition result.

[0019] Optionally, before correcting the text line recognition result based on the single character recognition result, the method further includes:

[0020] For each character information in the character recognition result, the overlap degree between the character information and the text line information in the image to be labeled is determined based on the determined character position in the image to be labeled and the text line position.

[0021] If it is determined that the overlap of the region is not less than the set overlap, it is determined that the single character information belongs to the text line information;

[0022] Based on the single-character information belonging to the text line information, determine the string corresponding to the text line information;

[0023] Based on the single-character recognition result, the text line recognition result is corrected, specifically including:

[0024] The text line recognition result is corrected based on the string.

[0025] Optionally, the text line recognition result is corrected based on the string, specifically including:

[0026] If it is determined that the string and the text line information meet the preset conditions, the text line recognition result is corrected according to the string;

[0027] The method further includes:

[0028] If it is determined that the preset conditions are not met between the string and the text line information, the image to be labeled will not be labeled.

[0029] Optionally, determining that the string and the text line information satisfy a preset condition specifically includes:

[0030] The number of individual characters contained in the text line information is determined as the first character count;

[0031] For each character contained in the text line information, if it is determined that the character is the same as the character in the same position in the string, the character is determined to be the first target character;

[0032] The number of characters in the first target character contained in the text line information is determined as the second character count;

[0033] If it is determined that the ratio between the second number of characters and the first number of characters is not less than a set ratio, it is determined that the string and the text line information meet the preset conditions.

[0034] Optionally, the string corresponding to the text line information is determined based on the single-character information belonging to the text line information, specifically including:

[0035] Based on the position of the single character information belonging to the text line information in the image to be labeled, determine the slope of the single character information belonging to the text line information in the image to be labeled.

[0036] Based on the slope of the text, the individual characters belonging to the text line information are sorted to obtain the string.

[0037] Optionally, the text line recognition result further includes: the first accuracy rate corresponding to each single character contained in the text line information, and the single character recognition result further includes: the second accuracy rate corresponding to each recognized single character information;

[0038] Based on the single-character recognition result, the text line recognition result is corrected to obtain the corrected text line recognition result, specifically including:

[0039] For each character information contained in the character recognition result, a character that matches the character position in the image to be labeled is determined from the text line information and is used as the second target character.

[0040] If it is determined that the second accuracy rate corresponding to the single character information is higher than the first accuracy rate corresponding to the second target character, the second target character is replaced using the single character information.

[0041] Optionally, based on the corrected text line recognition result, the image to be labeled is labeled to obtain a labeled image, specifically including:

[0042] If it is determined that the text line recognition result contains at least two text line information, then based on the text line positions of the at least two text line information in the image to be labeled, determine the text line information pairs that are adjacent in position in the image to be labeled;

[0043] If, according to the preset text arrangement order, the position of the last character of the preceding text line in the image to be labeled is determined, and the distance between the position of the first character of the following text line in the image to be labeled does not exceed a set distance, then the corrected preceding text line and the corrected following text line are spliced ​​together to obtain the spliced ​​text line.

[0044] Based on the spliced ​​text line information, the image to be labeled is labeled to obtain the labeled image.

[0045] This specification provides a model training apparatus, comprising:

[0046] The acquisition module is used to acquire each image to be labeled;

[0047] The recognition module is used to input each image to be labeled into the text line recognition model to be trained to obtain the text line recognition result, and to input the image to be labeled into the preset single character recognition model to obtain the single character recognition result.

[0048] The correction module is used to correct the text line recognition result based on the single character recognition result to obtain the corrected text line recognition result;

[0049] The annotation module is used to annotate the image to be annotated based on the corrected text line recognition results, so as to obtain an annotated image;

[0050] The training module is used to train the text line recognition model to be trained using the labeled image.

[0051] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for training the model.

[0052] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method for training the aforementioned model.

[0053] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0054] In the model training method provided in this specification, each image to be labeled is acquired. For each image, it is input into a text line recognition model to be trained to obtain a text line recognition result. The image is also input into a preset single-character recognition model to obtain a single-character recognition result. Then, based on the single-character recognition result, the text line recognition result is corrected to obtain a corrected text line recognition result. Finally, based on the corrected text line recognition result, the image to be labeled is labeled to obtain a labeled image. The labeled image is then used to train the text line recognition model.

[0055] As can be seen from the above method, this method can correct the text line recognition result based on the relatively accurate single character recognition result obtained by the preset single character recognition model, and obtain the corrected text line recognition result. Based on the corrected text line recognition result, the image to be labeled is labeled. Compared with the existing technology, which uses a discriminator to select the recognition results of two text recognition models to label the data, this method can effectively correct the text line recognition result by combining the single character recognition result and the text line recognition result, ensuring the accuracy of data labeling, thereby improving the accuracy of the text line recognition model. Attached Figure Description

[0056] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0057] Figure 1 A schematic flowchart illustrating the model training method provided in the embodiments of this specification;

[0058] Figure 2 A schematic diagram illustrating the method for calculating regional overlap provided in the embodiments of this specification;

[0059] Figure 3A , 3B A schematic diagram illustrating the text line splicing method provided in the embodiments of this specification;

[0060] Figure 4 This is a schematic diagram of the structure of the model training device provided in the embodiments of this specification;

[0061] Figure 5 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this specification. Detailed Implementation

[0062] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0063] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0064] Figure 1 The flowchart of the model training method provided in the embodiments of this specification is shown in the figure, which specifically includes the following steps:

[0065] S100: Obtain each image to be labeled.

[0066] In the embodiments of this specification, the server can acquire each image to be labeled. An image to be labeled can refer to an image without labeling information but containing text information. This image can be obtained through the internet, and then filtered to obtain images containing text information as the images to be labeled. Alternatively, the image to be labeled can be obtained from a preset image database, or it can be manually captured by image acquisition devices such as cameras or video cameras and uploaded to the server specifically for model training.

[0067] It should be noted that the execution entity for training the model can be a server or an electronic device such as a desktop computer. For ease of description, the following explanation will only use a server as the execution entity to describe the model training method provided in this manual.

[0068] S102: For each image to be labeled, input the image to be labeled into the text line recognition model to be trained to obtain the text line recognition result, and input the image to be labeled into the preset single character recognition model to obtain the single character recognition result.

[0069] In the embodiments of this specification, the server can input each image to be labeled into a text line recognition model to be trained, and obtain text line recognition results. The text line recognition results mentioned here may include: the text line information contained in the image to be labeled as identified by the text line recognition model, and the position of the text line information in the image to be labeled. Similarly, the image to be labeled can be input into a preset single-character recognition model to obtain single-character recognition results. The single-character recognition results mentioned here may include: the single-character information contained in the image to be labeled as identified by the single-character recognition model, and the position of each single-character information in the image to be labeled.

[0070] It should be noted that the text line recognition model can contain two sub-models: a text line information model and a text line position model. The text line information model is used to identify the text line information contained in the image to be labeled, and the text line position model is used to identify the position of the text line information in the image to be labeled. Similarly, the single character recognition model can contain two sub-models: a single character information model and a single character position model. The single character information model is used to identify the information of each single character contained in the image to be labeled, and the single character position model is used to identify the position of the single character information in the image to be labeled.

[0071] Among these, the text line position model, single-character information model, and single-character position model are relatively simple. Therefore, they can be trained accurately using a small amount of data through supervised training. However, the text line information model is more complex. Supervised training with a small amount of data is insufficient to obtain a sufficiently accurate model. Therefore, after supervised training with a small amount of data to obtain a preliminary trained text line information model, it is then trained using the model training method provided in this manual. Supervised training can be performed using existing publicly available datasets, such as the ReCTS dataset (Chinese and English street view sign text detection and recognition dataset).

[0072] S104: Based on the single character recognition result, the text line recognition result is corrected to obtain the corrected text line recognition result.

[0073] In the embodiments of this specification, the server can correct the text line recognition results obtained by the text line recognition model to be trained based on the accuracy of the single character recognition results obtained by the pre-trained single character recognition model, thereby obtaining the corrected text line recognition results, making the annotations corresponding to the image to be annotated more accurate.

[0074] Before correcting the text line recognition results based on the individual character recognition results, the server needs to first determine the corresponding text line recognition results. That is, it needs to determine which part of the text line recognition results the individual character information in the individual character recognition results corresponds to, in order to avoid errors in the annotation results. For each individual character information in the individual character recognition results, the server can determine the degree of regional overlap between the individual character information and the text line information in the image to be annotated, based on the determined individual character position and text line position in the image to be annotated.

[0075] Specifically, the character position of the character information in the to-be-annotated image may refer to the coordinate positions of the four vertices of the area covered by the character in the to-be-annotated image (specifically, the area covered by the character can be represented by the text box corresponding to the character in the to-be-annotated image). The text line position in the to-be-annotated image may refer to the coordinate positions of the four vertices of the area covered by the text line in the to-be-annotated image (specifically, the area covered by the text line can be represented by the line text box corresponding to the recognized text line in the to-be-annotated image). Based on the coordinate positions of the four vertices of the area where the character is located and the coordinate positions of the four vertices of the area where the text line is located, the area where the character is located and the area covered by the text line can be obtained, and the percentage value of the overlapping part between the area covered by the character and the area covered by the text line in the area covered by the character can be calculated to obtain the area overlap degree between the area where the character is located and the area where the text line is located. If it is determined that the area overlap degree is not less than the set overlap degree, it is determined that the character information belongs to the text line information, and each character information belonging to the text line information is obtained, and each character information belonging to the text line information is sorted to determine the string corresponding to the text line information. As Figure 2 shown.

[0076] Figure 2 is a schematic diagram of the method for calculating the area overlap degree provided by the embodiment of this specification.

[0077] In Figure 2 , the character recognition results include: X, X, Roast, Fish, Contact, Phone, and the text line recognition results include: XX Roast Fish, Contact Phone. The set overlap degree can be set artificially. Here, it is assumed that the set overlap degree is 75%. The server can obtain the area covered by the character information "Roast" according to the character position of the character information "Roast" ( Figure 2 the black text box corresponding to the character information "Roast" in it). Similarly, the server can obtain the area covered by the text line information "XX Roast Fish" according to the text line position of the text line information "XX Roast Fish" ( Figure 2 the black line text box corresponding to the text line information "XX Roast Fish" in it). It can be clearly seen from Figure 2 that the black text box corresponding to the character information "Roast" is within the black line text corresponding to the text line information "XX Roast Fish", that is, the area overlap degree between the area covered by the character information (Roast) and the area covered by the text line "XX Roast Fish" is 100%, which is higher than the set overlap degree (75%). Therefore, the character information "Roast" belongs to the text line information "XX Roast Fish". By analogy, it can be determined in turn whether the character information: X, X, Roast, Fish, Contact, Phone belongs to the text line information "XX Roast Fish". Then, the server sorts the character information belonging to the text line information "XX Roast Fish" to determine the string corresponding to the text line information.

[0078] If a single character does not belong to any line of text, or if a line of text does not contain any single character information contained in the single character recognition result, then that single character or line of text will be discarded and will not be processed further. For example, if the single character information contained in the recognized single character result differs significantly from the line of text information contained in the recognized line of text result, it indicates that the image quality of the image to be labeled may be poor (e.g., poor image clarity, low brightness, etc.). In order to ensure the training effect of the model, the image to be labeled may not be labeled.

[0079] The server can determine the slope of a single character belonging to a text line in the image to be labeled based on the character's position within that text line. For example, the server can fit a straight line based on the center coordinates of the regions containing each character in the image. Then, it determines the slope of the character belonging to the text line in the image based on the slope of this line. Finally, the server can determine whether the text line is vertical or horizontal based on the slope. If it is vertical, the characters are sorted according to their vertical coordinates to obtain a string. If it is horizontal, the characters are sorted according to their horizontal coordinates to obtain a string.

[0080] In the embodiments of this specification, since the single-character information in the string is obtained from a pre-trained single-character recognition model, the single-character information in the string is more accurate than the text line recognition result. Therefore, the server can correct the text line recognition result based on the string. If it is determined that the string and the text line information meet preset conditions, the server can correct the text line recognition result based on the string; if it is determined that the string and the text line information do not meet preset conditions, the image to be labeled will not be labeled.

[0081] It should be noted that the preset conditions mentioned here can include two types. The first condition can refer to whether the number of individual characters in the text line information is the same as the number of individual characters in the string. If the number of individual characters in the text line information is the same as the number of individual characters in the string, the server can correct the text line recognition result based on the string. If the number of individual characters in the text line information is different from the number of individual characters in the string, the image to be labeled will not be labeled.

[0082] The second condition can refer to whether the number of identical single characters contained in both the text line information and the string meets a certain condition. Specifically, the server can determine the number of single characters contained in the text line information as the first character count. For each single character in the text line information, if it is determined that the single character is the same as a single character in the same position in the string, it is determined that the single character is the first target single character. The number of single characters of the first target single character in the text line information is determined as the second character count. If it is determined that the ratio between the second character count and the first character count is not less than a set ratio, it is determined that the string and the text line information meet the preset condition. In other words, it determines whether the single characters in the same position in the text line information and the string are the same, obtains the number of identical single characters, calculates the number of identical single characters with the number of single characters in the text line information, and if it is determined that the ratio is not less than a set ratio, it is determined that the string and the text line information meet the preset condition.

[0083] It should be noted that if the server determines that the string and the text line information meet any of the above-mentioned conditions, then the server can determine that the string and the text line information meet the preset conditions. Of course, the server can determine that the string and the text line information meet the preset conditions when it determines that the string and the text line information meet both of the above-mentioned conditions at the same time.

[0084] In the embodiments of this specification, the server can correct the text line recognition result based on the string to obtain the corrected text line recognition result. Specifically, the server can determine whether the string and the text line information are the same. If the string and the text line information are the same, the text line information is replaced with the string. The server can also determine whether the single character information in the string is the same as the single character information at the same position in the text line information. If the single character information in the string is different from the single character information at the same position in the text line information, the single character information in the text line information that is different from the string at the same position is replaced with the single character information at the same position in the string.

[0085] Furthermore, the text line recognition result also includes: the first accuracy rate corresponding to each character contained in the text line information, and the single character recognition result also includes: the second accuracy rate corresponding to each recognized single character information. For each single character information contained in the single character recognition result, the server can determine from the text line information a single character whose position in the image to be labeled matches that single character information, and use this single character information as the second target single character. If it is determined that the second accuracy rate corresponding to the single character information is higher than the first accuracy rate corresponding to the second target single character, the second target single character is replaced using that single character information.

[0086] In other words, the server can compare the second accuracy rate corresponding to each single character with the first accuracy rate corresponding to the single character in the same position of the text line information. If the second accuracy rate corresponding to the single character is higher than the first accuracy rate corresponding to the single character in the same position of the text line information, the single character in the same position of the text line information will be replaced with the single character.

[0087] In practical applications, if the server determines that a single character in the string is the same as a line of text, it considers the line of text accurate and sets the first accuracy rate for all characters in the line of text to 100%. If a single character in the string is different from the line of text, the first accuracy rate for the same characters in both the line of text and the string is set to 100%. Then, the server identifies the characters that are different from the string and replaces the characters in different positions in the line of text with their corresponding first accuracy rates, and uses the more accurate single character recognition results obtained from the preset single character recognition model to correct the line of text recognition results, thus obtaining accurate corrected line of text recognition results and improving the training effect of the line of text recognition model.

[0088] S106: Based on the corrected text line recognition results, the image to be labeled is labeled to obtain the labeled image.

[0089] In the embodiments of this specification, the server can annotate the image to be labeled based on the corrected text line recognition results, thereby obtaining an annotated image. It should be noted that in practical applications, the server may recognize multiple text lines from a single image to be labeled. Since the text line recognition model may experience fragmentation during the recognition process—that is, recognizing a longer text line as two shorter text lines—the server needs to concatenate multiple text lines from the same image to be labeled.

[0090] If it is determined that the text line recognition result contains at least two text line information, according to the text line positions of the at least two text line information in the to-be-annotated image, determine the text line information pairs with adjacent positions in the to-be-annotated image. First, according to the text slopes of the text line information, divide the text line information into vertical text line information and horizontal text line information, and then form vertical text line information pairs with adjacent vertical text line information, and form horizontal text line information pairs with adjacent horizontal text line information. Second, if it is determined that the text line information in the same vertical (horizontal) text line information pair is on the same vertical (horizontal) line, and, according to the preset text arrangement order, determine the distance between the position of the last single character of the previous text line information and the position of the first single character of the next text line information in the text line information pair in the to-be-annotated image does not exceed the set distance, splice the corrected previous text line information and the corrected next text line information to obtain the spliced text line information. Finally, according to the spliced text line information, annotate the to-be-annotated image to obtain the annotated image. As Figure 3A 、 3B shown.

[0091] Figure 3A 、 3B is a schematic diagram of the text line splicing method provided by the embodiment of this specification.

[0092] In Figure 3A , the server determines that the text line recognition result in the to-be-annotated image contains three text line information. First, according to the text slopes of the three text line information, determine that the three text line information are horizontal text line information, and further determine that the text line information "Braised Chicken" and "Rice" are on the same horizontal line. Then, according to the preset text arrangement order, determine the position of the last single character "Chicken" in the text line information "Braised Chicken" and the position of the first single character "Rice" in the text line information "Rice". If the distance between the single character "Chicken" and the single character "Rice" does not exceed the set distance, splice the text line information "Braised Chicken" and the text line information "Rice" to obtain the spliced text line information "Braised Chicken Rice", as Figure 3B shown. [[ID=??]]

[0093] The above set distance is set artificially. If the text line information is horizontal text line information, determine the set distance according to the single character height in the text line information. If the text line information is vertical text line information, determine the set distance according to the single character width in the text line information.

[0094] S##: Through the annotated image, perform model training on the to-be-trained text line recognition model.

[0095] It should be noted that there seems to be an error in the line numbering in the original text. The "??" in the line number [[ID=??]] is likely a mistake. I've translated it as best as possible based on the context.In the embodiments of this specification, the server can train a text line recognition model to be trained using labeled images. The text line recognition model to be trained may include two sub-models: a text line information model and a text line position model. The text line position model is mainly used to identify the position of text lines in the labeled image. Through supervised training with a small amount of data, an accurate text line position model can be trained. Therefore, the more complex text line information model can be further trained using labeled images. This not only effectively reduces the human resource cost of model training but also ensures that the text line information model can accurately recognize text lines in images during practical applications.

[0096] As can be seen from the above process, this method can correct the text line recognition results based on the relatively accurate single-character recognition results obtained from the preset single-character recognition model, and then label the image to be labeled. Compared with the existing technology that uses a discriminator to select the recognition results of two text recognition models for data labeling, this method, by combining the single-character recognition results and the text line recognition results, can effectively correct the text line recognition results, ensure the accuracy of data labeling, and thus improve the accuracy of the text line recognition model.

[0097] The above describes a model training method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding model training apparatus, such as... Figure 4 As shown.

[0098] Figure 4 This is a schematic diagram of the structure of the model training device provided in the embodiments of this specification, specifically including:

[0099] Module 400 is used to acquire each image to be labeled;

[0100] The recognition module 402 is used to input each image to be labeled into a text line recognition model to be trained to obtain a text line recognition result, and to input the image to be labeled into a preset single character recognition model to obtain a single character recognition result.

[0101] The correction module 404 is used to correct the text line recognition result based on the single character recognition result to obtain the corrected text line recognition result;

[0102] The annotation module 406 is used to annotate the image to be annotated according to the corrected text line recognition result, so as to obtain the annotated image;

[0103] The training module 408 is used to train the text line recognition model to be trained using the labeled image.

[0104] Optionally, the recognition module 402 is specifically configured to: input the image to be labeled into a text line recognition model to be trained, obtain the text line information contained in the image to be labeled as recognized by the text line recognition model, and the text line position of the text line information in the image to be labeled, and use the text line position and the text line information as the text line recognition result; input the image to be labeled into the single character recognition model, obtain the single character information contained in the image to be labeled as recognized by the single character recognition model, and the single character position of each single character information in the image to be labeled, and use the single character information and the single character position of each single character information in the image to be labeled as the single character recognition result.

[0105] Optionally, the recognition module 402 is specifically used to, for each character information in the character recognition result, determine the degree of regional overlap between the character information and the text line information in the image to be labeled based on the determined character position and the text line position of the character information in the image to be labeled; if the degree of regional overlap is not less than a set degree of overlap, determine that the character information belongs to the text line information; determine the string corresponding to the text line information based on the character information belonging to the text line information; and correct the text line recognition result based on the string.

[0106] Optionally, the correction module 404 is specifically used to: if it is determined that the string and the text line information meet a preset condition, correct the text line recognition result according to the string; if it is determined that the string and the text line information do not meet the preset condition, not label the image to be labeled.

[0107] Optionally, the correction module 404 is specifically used to: determine the number of individual characters contained in the text line information as a first number of characters; for each individual character contained in the text line information, if it is determined that the individual character is the same as the individual character in the same position in the string, determine that the individual character is a first target individual character; determine the number of individual characters of the first target individual character contained in the text line information as a second number of characters; if it is determined that the ratio between the second number of characters and the first number of characters is not less than a set ratio, determine that the string and the text line information meet a preset condition.

[0108] Optionally, the recognition module 402 is specifically used to determine the text slope of the single character information belonging to the text line information in the image to be labeled based on the position of the single character information in the image to be labeled, and sort the single character information belonging to the text line information according to the text slope to obtain the string.

[0109] Optionally, the recognition module 402 is specifically used to: the text line recognition result further includes: a first accuracy rate corresponding to each single character contained in the text line information; the single character recognition result further includes: a second accuracy rate corresponding to each recognized single character information; for each single character information contained in the single character recognition result, a single character that matches the single character position in the image to be labeled is determined from the text line information and used as a second target single character; if it is determined that the second accuracy rate corresponding to the single character information is higher than the first accuracy rate corresponding to the second target single character, the second target single character is replaced by the single character information.

[0110] Optionally, the annotation module 406 is specifically used to: if it is determined that the text line recognition result contains at least two text line information, determine the text line information pairs that are adjacent in the image to be annotated according to the text line positions of the at least two text line information in the image to be annotated; if, according to a preset text arrangement order, the last character of the preceding text line information in the text line information pair is determined to be located in the image to be annotated, and the distance between the last character of the preceding text line information in the text line information pair and the first character of the following text line information in the text line information pair is determined to be located in the image to be annotated does not exceed a set distance; then, the corrected preceding text line information and the corrected following text line information are spliced ​​together to obtain spliced ​​text line information; and the image to be annotated is annotated according to the spliced ​​text line information to obtain an annotated image.

[0111] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided method for training the model.

[0112] This instruction manual also provides Figure 5 The diagram shows the structure of the electronic device. Figure 5 At the hardware level, the equipment used for training this model includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business logic. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functionality. Figure 1The method for training the model described herein. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0113] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0114] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0115] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0116] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0117] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0118] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0119] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0122] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0123] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0124] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0125] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0126] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0127] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0128] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for training a model, characterized in that, include: Obtain each image to be labeled; For each image to be labeled, the image is input into the text line recognition model to be trained to obtain the text line recognition result, and the image to be labeled is input into the preset single character recognition model to obtain the single character recognition result; Based on the single character recognition result, the text line recognition result is corrected to obtain the corrected text line recognition result; based on the corrected text line recognition result, the image to be labeled is labeled to obtain the labeled image; The text line recognition model to be trained is trained using the annotated image. The image to be labeled is input into the text line recognition model to be trained, and the text line recognition results are obtained, including: The image to be labeled is input into the text line recognition model to be trained to obtain the text line information contained in the image to be labeled as recognized by the text line recognition model, and the position of the text line information in the image to be labeled; the text line position and the text line information are used as the text line recognition result. The image to be labeled is input into a preset single-character recognition model to obtain single-character recognition results, specifically including: The image to be labeled is input into the single-character recognition model to obtain the information of each single character contained in the image to be labeled as recognized by the single-character recognition model, as well as the position of each single character in the image to be labeled. The method further includes using the single-character information and the position of each single-character information in the image to be labeled as the single-character recognition result; before correcting the text line recognition result based on the single-character recognition result, the method further includes: For each character information in the character recognition result, the overlap degree between the character information and the text line information in the image to be labeled is determined based on the determined character position in the image to be labeled and the text line position. If it is determined that the overlap of the regions is not less than the set overlap, it is determined that the single character information belongs to the text line information; Based on the single-character information belonging to the text line information, determine the string corresponding to the text line information; Based on the single-character recognition result, the text line recognition result is corrected, specifically including: The text line recognition result is corrected based on the string.

2. The method as described in claim 1, characterized in that, Based on the string, the text line recognition result is corrected, specifically including: If it is determined that the string and the text line information meet the preset conditions, the text line recognition result is corrected according to the string; The method further includes: If it is determined that the preset conditions are not met between the string and the text line information, the image to be labeled will not be labeled.

3. The method as described in claim 2, characterized in that, Determining that the string and the text line information satisfy a preset condition specifically includes: The number of individual characters contained in the text line information is determined as the first character count; For each character contained in the text line information, if it is determined that the character is the same as the character in the same position in the string, the character is determined to be the first target character; The number of characters in the first target character contained in the text line information is determined as the second character count; If it is determined that the ratio between the second number of characters and the first number of characters is not less than a set ratio, it is determined that the string and the text line information meet the preset conditions.

4. The method as described in claim 1, characterized in that, Based on the single-character information belonging to the text line information, the string corresponding to the text line information is determined, specifically including: Based on the position of the single character information belonging to the text line information in the image to be labeled, determine the slope of the single character information belonging to the text line information in the image to be labeled. Based on the slope of the text, the individual characters belonging to the text line information are sorted to obtain the string.

5. The method according to any one of claims 1 to 4, characterized in that, The text line recognition result also includes: the first accuracy rate corresponding to each single character contained in the text line information; the single character recognition result also includes: the second accuracy rate corresponding to each recognized single character information. Based on the single-character recognition result, the text line recognition result is corrected to obtain the corrected text line recognition result, specifically including: For each character information contained in the character recognition result, a character that matches the character position in the image to be labeled is determined from the text line information and is used as the second target character. If it is determined that the second accuracy rate corresponding to the single character information is higher than the first accuracy rate corresponding to the second target character, the second target character is replaced using the single character information.

6. The method as described in claim 3, characterized in that, Based on the corrected text line recognition results, the image to be labeled is labeled to obtain the labeled image, specifically including: If it is determined that the text line recognition result contains at least two text line information, then based on the text line positions of the at least two text line information in the image to be labeled, determine the text line information pairs that are adjacent in position in the image to be labeled; If, according to the preset text arrangement order, the position of the last character of the preceding text line in the image to be labeled is determined, and the distance between the position of the first character of the following text line in the image to be labeled does not exceed a set distance, then the corrected preceding text line and the corrected following text line are spliced ​​together to obtain the spliced ​​text line. Based on the spliced ​​text line information, the image to be labeled is labeled to obtain the labeled image.

7. A device for model training, characterized in that, include: The acquisition module is used to acquire each image to be labeled; The recognition module is used to input each image to be labeled into the text line recognition model to be trained to obtain the text line recognition result, and to input the image to be labeled into the preset single character recognition model to obtain the single character recognition result. The correction module is used to correct the text line recognition result based on the single character recognition result to obtain the corrected text line recognition result; The annotation module is used to annotate the image to be annotated based on the corrected text line recognition results, so as to obtain an annotated image; The training module is used to train the text line recognition model to be trained using the labeled image. The image to be labeled is input into the text line recognition model to be trained, and the text line recognition results are obtained, including: The image to be labeled is input into the text line recognition model to be trained to obtain the text line information contained in the image to be labeled as recognized by the text line recognition model, and the position of the text line information in the image to be labeled; the text line position and the text line information are used as the text line recognition result. The image to be labeled is input into a preset single-character recognition model to obtain single-character recognition results, specifically including: The image to be labeled is input into the single-character recognition model to obtain the information of each single character contained in the image to be labeled as recognized by the single-character recognition model, as well as the position of each single character in the image to be labeled. The single-character information and the position of each single-character information in the image to be labeled are used as the single-character recognition result; before correcting the text line recognition result based on the single-character recognition result, the method further includes: For each character information in the character recognition result, the overlap degree between the character information and the text line information in the image to be labeled is determined based on the determined character position in the image to be labeled and the text line position. If it is determined that the overlap of the regions is not less than the set overlap, it is determined that the single character information belongs to the text line information; Based on the single-character information belonging to the text line information, determine the string corresponding to the text line information; Based on the single-character recognition result, the text line recognition result is corrected, specifically including: The text line recognition result is corrected based on the string.

8. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 6.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 6.

Images