OCR training image generation method and device, computer equipment and storage medium

Abstract

The invention relates to an OCR training image generation method and device, computer equipment and a storage medium. The method comprises the following steps: training an initial image generation model based on a preset first domain image set and a preset second domain image set to obtain a target image generation model; generating an input image base map based on a preset character line size; determining a model input image based on the input image base map and the character corresponding to the character line size; and inputting the model input image into the target image generation model for processing, and outputting an OCR training image. According to the invention, the problem of lack of training images in the OCR model can be solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an OCR training image generation method, device, computer equipment and storage medium. Background Art

[0002] In the digital age, a large number of paper documents and images need to be converted into editable and searchable electronic documents. Currently, OCR models are generally used to convert paper documents and images. When training OCR models, OCR training images obtained through manual scanning and manual annotation are required.

[0003] However, a large number of OCR training images are required for OCR model training. Since OCR training images are obtained through manual scanning and manual annotation, the process of obtaining a large number of OCR training images is not only time-consuming but also costly. In addition, some high-quality paper documents and images used to generate OCR training images have become increasingly difficult to obtain due to privacy or copyright issues. It can be seen that it is currently difficult to obtain sufficient OCR training images for training OCR models. Summary of the invention

[0004] Based on this, it is necessary to provide an OCR training image generation method, apparatus, computer equipment, computer readable storage medium and computer program product to solve the problem of lack of training images in the OCR model in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for generating an OCR training image, comprising:

[0006] Based on the preset first domain image set and second domain image set, the initial image generation model is trained to obtain a target image generation model;

[0007] Generate an input image base map based on a preset character row size;

[0008] Determining a model input image based on the input image base image and characters corresponding to the character row size;

[0009] The model input image is input into the target image generation model for processing, and the OCR training image is output.

[0010] In a second aspect, the present application provides an OCR training image generation device, comprising:

[0011] A model training module, used to train the initial image generation model based on the preset first domain image set and the second domain image set to obtain a target image generation model;

[0012] A base map generation module, used for generating an input image base map based on a preset character row size;

[0013] An input image generation module, for determining a model input image based on an input image base image and characters corresponding to a character row size;

[0014] The training image generation module is used to input the model input image into the target image generation model for processing and output the OCR training image.

[0015] In a third aspect, the present application provides a computer device, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps in the above method when executing the computer program.

[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps in the above method when executed by a processor.

[0017] In a fifth aspect, the present application provides a computer program product, which includes a computer program, and the computer program implements the steps in the above method when executed by a processor.

[0018] The above-mentioned OCR training image generation method, device, computer equipment, computer-readable storage medium and computer program product, based on the generation of the input image base map, processes the input image base map through characters corresponding to the character row size to obtain a model input image, and then inputs the model input image into the target image generation model for processing, and outputs an OCR training image. Through the above implementation, based on the generation of the target image generation model, a model input image for input into the target image generation model can be automatically generated according to the input image base map and the corresponding characters, and then the model input image is processed by the target image generation model to obtain a large number of OCR training images, so that there is no need to obtain OCR training images through manual scanning and manual annotation, and obtaining OCR training images will not be limited by privacy or copyright issues, so the problem of lack of training images in the OCR model can be solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 An application environment diagram of an OCR training image generation method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a flow chart of a method for generating an OCR training image provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a model input image provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of an OCR training image provided in an embodiment of the present application;

[0023] Figure 5 A structural block diagram of an OCR training image generation device provided in an embodiment of the present application;

[0024] Figure 6 An internal structure diagram of a computer device provided in an embodiment of the present application;

[0025] Figure 7 An internal structure diagram of another computer device provided in an embodiment of the present application;

[0026] Figure 8 An internal structure diagram of a computer-readable storage medium provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The OCR training image generation method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through a communication network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0029] like Figure 2 As shown, the present application embodiment provides an OCR training image generation method, which is applied to Figure 1 The terminal 102 or the server 104 in the example is used for explanation. It is understandable that the computer device may include at least one of the terminal and the server. The method comprises the following steps:

[0030] S110 . Based on a preset first domain image set and a second domain image set, an initial image generation model is trained to obtain a target image generation model.

[0031] Wherein, the first domain image set and the second domain image set both include multiple images for training the initial image generation model, and the images in the first domain image set and the second domain image set are images of two different styles; exemplarily, the images in the first domain image set are all images of "horse", and the images in the first domain image set are recorded as first domain images, while the images in the second domain image set are all images of "zebra", and the images in the second domain image set are recorded as second domain images; the initial image generation model is an untrained image generation model; exemplarily, the image generation model is CycleGAN (Cycle Generative Adversarial Network), that is, a cyclic adversarial generation network; the initial image generation model can convert the first domain image into the second domain image, and can also convert the second domain image into the first domain image; the target image generation model is a trained initial image generation model, and the data related to the OCR training image is input into the target image generation model for processing, and the OCR training image is output; wherein the OCR training image is the training data required for training the OCR model.

[0032] Specifically, the preset first domain image set and the second domain image set are used as training data to train an untrained initial image generation model, thereby obtaining a target image generation model.

[0033] S120: Generate an input image base map based on a preset character row size.

[0034] Among them, the character row size is preset according to the size of the character row shown on a carrier in the actual scene, and the character row size includes the character row width W and the character row height H, and the character row consists of a row of characters; exemplarily, in a supermarket shopping scene, the above-mentioned carrier is a shopping receipt, and the size of important character rows in the shopping receipt is obtained. The important character rows may be the character rows where the receipt generation time, receipt code, shopping payment amount, etc. are located; the input image background map is a background map with only a strip frame corresponding to the character row size, the width of the strip frame is the character row width W, and the height of the strip frame is the character row height H. Exemplarily, the area within the strip frame on the input image background map is gray, and the area except the strip frame on the input image background map is black.

[0035] Specifically, according to the size of the character row shown on some carriers in the actual scene, the corresponding character row size is preset according to the size of the character row, and further according to the size of the character row, an input image base map with a strip frame is generated.

[0036] S130. Determine a model input image based on the input image background and characters corresponding to the character row size.

[0037] Among them, the character row size has a corresponding bar frame on the input image base map, and the bar frame is subsequently used to display a certain number of characters. For example, the certain number of characters may be the corresponding characters in the shopping receipt, such as the receipt generation time, the receipt code, the shopping payment amount, etc. The bar frame is subsequently used to display a certain number of characters, that is, the characters corresponding to the character row size; by processing the characters corresponding to the character row size into the bar frame corresponding to the character row size in the input image base map, the input image base map containing the characters can be obtained, and the input image base map containing the characters is recorded as the model input image, and the model input image is as follows Figure 3 As shown, the model input image is subsequently used to be input into the target image generation model for processing to generate an OCR training image.

[0038] Specifically, characters corresponding to the character row size are obtained, and then the characters are processed into a bar frame corresponding to the character row size in the input image base map, thereby obtaining a model input image.

[0039] S140, input the model input image into the target image generation model for processing, and output the OCR training image.

[0040] The OCR training image refers to the training data used to train the OCR model.

[0041] Specifically, a large number of model input images are obtained by repeatedly executing steps S120-S130. Taking one of the model input images as an example, the model input image is input into the target image generation model for processing, and an OCR training image corresponding to the model input image is output. Through the above steps, the target image generation model processes a large number of model input images, thereby obtaining a large number of OCR training images.

[0042] It can be seen that in the embodiment of the present application, on the basis of generating the input image base map, the input image base map is processed by the characters corresponding to the character row size to obtain the model input image, and then the model input image is input into the target image generation model for processing, and the OCR training image is output. Through the above implementation, on the basis of generating the target image generation model, the model input image for input into the target image generation model can be automatically generated according to the input image base map and the corresponding characters, and then the model input image is processed by the target image generation model to obtain a large number of OCR training images, so that there is no need to obtain the OCR training image through manual scanning and manual annotation, and the acquisition of the OCR training image will not be limited by privacy or copyright issues, so the problem of lack of training images in the OCR model can be solved.

[0043] In addition, in the embodiment of the present application, through the implementation of the above steps, a large number of high-quality OCR training images can be generated, thereby providing rich and diverse training data for the training of the OCR model, which helps to improve the recognition accuracy and robustness of the trained OCR model, thereby enabling the OCR model to better adapt to various complex text and image environments; and in generating OCR training images, since there is no need to manually scan and annotate a large number of images or documents, the cost of obtaining OCR training images can be greatly reduced, and the efficiency of obtaining OCR training images can be greatly improved; in addition, the OCR training images obtained through this embodiment can be used to train the corresponding OCR model, and the OCR images generated by the OCR model can be applied to multiple fields, such as document digitization, license plate recognition, bank check processing, etc., thereby meeting the needs of different application scenarios.

[0044] In some embodiments, based on the preset first domain image set and the second domain image set, the initial image generation model is trained to obtain the target image generation model, including:

[0045] S111 . Sampling a preset first domain image set and a preset second domain image set to obtain a sample mean value group.

[0046] Among them, sampling means collecting a certain number of samples (images in the image set) from the image set; the sample mean group includes a first sample mean E1 and a second sample mean E2, the first sample mean E1 is the mean of the samples collected from the first domain image set, and the second sample mean E2 is the mean of the samples collected from the second domain image set.

[0047] Specifically, a certain number of samples are collected from the first domain image set, and the mean of the collected samples is calculated to obtain a first sample mean E1; a certain number of samples are also collected from the second domain image set, and the mean of the collected samples is calculated to obtain a second sample mean E2; the first sample mean E1 and the second sample mean E2 are combined to obtain a sample mean group.

[0048] S112 , respectively inputting the first domain image set and the second domain image set into the initial image generation model for processing, and outputting the first domain image probability and the second domain image probability.

[0049] It should be noted that the initial image generation model includes two sets of generators and discriminators. The generator is used to convert an image input into one domain into an image of another domain, where "domain" can be understood as the style of the image, such as an image in the style of "horse", an image in the style of "zebra", etc.; illustratively, the generator is used to convert an image of "horse" into an image of "zebra"; the discriminator is used to determine the probability that the image converted by the generator is close to the real image. The generator and discriminator of the first group in the initial image generation model are respectively recorded as the first generator G1 and the first discriminator D1, and the generator and discriminator of the second group are respectively recorded as the second generator G2 and the second discriminator D2; the first generator is used to convert the first domain image A in the first domain image set into the corresponding second domain image G1(A). For example, the first generator is used to convert the image of "horse" in the first domain image set into the image of "zebra"; the first discriminator is used to judge the probability that the second domain image converted by the first generator is close to the real image, and the probability is recorded as the first domain image probability D1(G1(A)); the second generator is used to convert the second domain image B in the second domain image set into the corresponding first domain image G2(B). For example, the second generator is used to convert the image of "zebra" in the second domain image set into the image of "horse"; the second discriminator is used to judge the probability that the first domain image converted by the second generator is close to the real image, and the probability is recorded as the second domain image probability D2(G2(B)).

[0050] Specifically, taking a first domain image in the first domain image set as an example, the first domain image is input into the initial image generation model, and the first generator G1 in the initial image generation model converts the first domain image A in the first domain image set into the corresponding second domain image G1(A), and then the first discriminator D1 judges the probability that the second domain image G1(A) is close to the real image to obtain the first domain image probability D1(G1(A)); the second generator G2 in the initial image generation model converts the second domain image B in the second domain image set into the corresponding first domain image G2(B), and then the second discriminator D2 judges the probability that the first domain image G2(B) is close to the real image to obtain the second domain image probability D2(G2(B)).

[0051] S113. Determine an adversarial loss group based on the sample mean group, the first domain image probability, and the second domain image probability.

[0052] Among them, the adversarial loss between the first generator G1 and the first discriminator D1 can be established through the sample mean group and the first domain image probability D1 (G1 (A)), and the loss is recorded as the first adversarial loss L GAN1The smaller the first adversarial loss is, the more the second domain image G1(A) generated by the first generator G1 can deceive the first discriminator D1, so that the first discriminator D1 thinks that the second domain image G1(A) is close to the real second domain image B. The adversarial loss between the second generator G2 and the second discriminator D2 can be established through the sample mean group and the second domain image probability D2(G2(B)), and the loss is recorded as the second adversarial loss L GAN2 The smaller the second adversarial loss is, the more the first domain image G2(B) generated by the second generator G2 can deceive the second discriminator D2, so that the second discriminator D2 thinks that the first domain image G2(B) is close to the real first domain image A. The adversarial loss group includes the first adversarial loss L GAN1 With the second adversarial loss L GAN2 .

[0053] Specifically, the first adversarial loss L is established based on the sample mean group and the first domain image probability D1(G1(A)) GAN1 , and establishes the second adversarial loss L according to the sample mean group and the second domain image probability D2(G2(B)) GAN2 ; Set the first adversarial loss L GAN1 With the second adversarial loss L GAN2 Get the adversarial loss group.

[0054] S114 , inputting the first domain image set and the second domain image set into an initial image generation model for processing, and outputting a first image difference and a second image difference.

[0055] Among them, the first generator G1 in the initial image generation model is used to convert the first domain image A in the first domain image set input therein into the second domain image G1(A), and the second generator G2 can further convert the second domain image G1(A) back into the first domain image G2(G1(A)); the first image difference is the difference between the first domain image A and the first domain image G2(G1(A)); the second generator G2 in the initial image generation model is used to convert the second domain image B in the second domain image set input therein into the first domain image G2(B), and the first generator G1 can further convert the first domain image G2(B) back into the second domain image G1(G2(B)); the second image difference is the difference between the second domain image B and the second domain image G1(G2(B)).

[0056] Specifically, taking a first domain image A in the first domain image as an example, the first domain image A is input into the first generator G1 for conversion to obtain the second domain image G1(A), and then the second domain image G1(A) is input into the second generator G2 for conversion to obtain the first domain image G2(G1(A)), and the difference between the first domain image G2(G1(A)) and the corresponding first domain image A is calculated to obtain the first image difference; also taking a second domain image B in the second domain image as an example, the first domain image B is input into the second generator G2 for conversion to obtain the first domain image G2(B), and then the first domain image G2(B) is input into the first generator G1 for conversion to obtain the second domain image G1(G2(B)), and the difference between the second domain image G1(G2(B)) and the corresponding second domain image B is calculated to obtain the second image difference.

[0057] S115. Determine a cycle consistency loss based on the sample mean group, the first image difference, and the second image difference.

[0058] Among them, the first consistency loss can be constructed according to the difference between the first sample mean E1 in the sample mean group and the first image, and the second consistency loss can be constructed according to the difference between the second sample mean E2 in the sample mean group and the second image; and the cycle consistency loss L can be constructed according to the first consistency loss and the second consistency loss. CYC ; Cycle consistency loss L CYC The smaller it is, the smaller the difference between the first image and the second image is, which means that the first generator G1 and the second generator G2 have stronger ability to reduce the difference between the converted image and the real image.

[0059] Specifically, a first consistency loss is constructed according to the difference between the first sample mean E1 in the sample mean group and the first image, and a second consistency loss is constructed according to the difference between the second sample mean E2 in the sample mean group and the second image; and a cycle consistency loss L is constructed according to the first consistency loss and the second consistency loss. CYC .

[0060] S116. Determine the overall loss based on the adversarial loss group and the cycle consistency loss; train the initial image generation model based on the overall loss to obtain the target image generation model.

[0061] Among them, the overall loss L TOTAL is the first adversarial loss L in the adversarial loss group GAN1 , the second adversarial loss L GAN2 And the cycle consistency loss L CYC The sum of the three; the overall loss L TOTAL Used to train the entire initial image generation model to obtain the target image generation model.

[0062] Specifically, calculate the first adversarial loss L in the adversarial loss group GAN1 , the second adversarial loss L GAN2 And the cycle consistency loss L CYC The sum of the three is the total loss L TOTAL , further, according to the overall loss L TOTAL The initial image generation model is trained to obtain the target image generation model.

[0063] It can be seen that in this embodiment, the first countermeasure loss L GAN1 The first generator G1 and the first discriminator D1 in the initial image generation model can be trained so that the image converted by the first generator G1 can deceive the first discriminator D1, that is, the ability of the first generator G1 to make its converted image close to the real image is improved, and the ability of the first discriminator D1 to distinguish the image converted by the first generator G1 from the real image is also improved; through the second adversarial loss L GAN2 The second generator G2 and the second discriminator D2 in the initial image generation model can be trained so that the image converted by the second generator G2 can deceive the second discriminator D2, that is, the ability of the second generator G2 to make its converted image close to the real image is improved, and the ability of the second discriminator D2 to distinguish the image converted by the second generator G2 from the real image is also improved; through the cycle consistency loss L CYC It is convenient to improve the ability of the first generator G1 and the second generator G2 to make the image after two conversions as consistent as possible with the original image in terms of content; through the overall loss L TOTAL This facilitates the training of the entire initial image generation model, so that the target image generation model can generate OCR training images that are close to the real scene.

[0064] In some embodiments, the sample mean group includes a first sample mean corresponding to the first domain image set and a second sample mean corresponding to the second domain image set; and based on the sample mean group, the first domain image probability and the second domain image probability, determining the adversarial loss group includes:

[0065] S1131. Determine a first adversarial loss based on the first sample mean, the second sample mean, and the first domain image probability.

[0066] Among them, to calculate the first adversarial loss L GAN1, it is necessary to first calculate the product E2*[logD1(G1(A))] of the logarithm of the second sample mean E2 and the first domain image probability D1(G1(A)), where the logarithm is specifically a logarithm with base 2, and the product is recorded as the first product; it is also necessary to calculate the difference (1-D1(G1(A))) between 1 and the first domain image probability D1(G1(A)), and record the difference as the first difference, and then calculate the product E1*[log(1-D1(G1(A)))] of the logarithm of the first sample mean E1 and the first difference, and record the product as the second product; the first adversarial loss L GAN1 is the sum of the first product E2*[logD1(G1(A))] and the second product E1*[log(1-D1(G1(A)))], that is, L GAN1 =E2*[logD1(G1(A))]+E1*[log(1-D1(G1(A)))].

[0067] Specifically, the product of the second sample mean E2 and the logarithm of the first domain image probability D1(G1(A)) is calculated to obtain the first product E2*[logD1(G1(A))]; the difference between 1 and the first domain image probability D1(G1(A)) is also calculated to obtain the first difference (1-D1(G1(A))); then the product of the first sample mean E1 and the logarithm of the first difference is calculated to obtain the second product E1*[log(1-D1(G1(A)))]; finally, the sum of the first product E2*[logD1(G1(A))] and the second product E1*[log(1-D1(G1(A)))] is calculated to obtain the first adversarial loss L GAN1 .

[0068] S1132: Determine a second adversarial loss based on the first sample mean, the second sample mean, and the second domain image probability.

[0069] Among them, to calculate the second adversarial loss L GAN2 , it is necessary to first calculate the product E1*[logD2(G2(B))] of the logarithm of the first sample mean E1 and the second domain image probability D2(G2(B)), where the logarithm is specifically a logarithm with base 2, and the product is recorded as the third product; it is also necessary to calculate the difference (1-D2(G2(B))) between 1 and the second domain image probability D2(G2(B)), and record the difference as the second difference, and then calculate the product E2*[log(1-D2(G2(B)))] of the logarithm of the second sample mean E2 and the second difference, and record the product as the fourth product; the second adversarial loss L GAN2 is the sum of the third product E1*[logD2(G2(B))] and the fourth product E2*[log(1-D2(G2(B)))], that is, L GAN2=E1*[logD2(G2(B))]+E2*[log(1-D2(G2(B)))].

[0070] Specifically, the product of the first sample mean E1 and the logarithm of the second domain image probability D2(G2(B)) is calculated to obtain the third product E1*[logD2(G2(B))]; the difference between 1 and the second domain image probability D2(G2(B)) is also calculated to obtain the second difference (1-D2(G2(B))); then the product of the second sample mean E2 and the logarithm of the second difference is calculated to obtain the fourth product E2*[log(1-D2(G2(B)))]; finally, the sum of the third product E1*[logD2(G2(B))] and the fourth product E2*[log(1-D2(G2(B)))] is calculated to obtain the second adversarial loss L GAN2 .

[0071] S1133. Determine an adversarial loss group based on the first adversarial loss and the second adversarial loss.

[0072] Among them, the adversarial loss group includes the first adversarial loss L GAN1 With the second adversarial loss L GAN2 .

[0073] It can be seen that in this embodiment, the first confrontation loss L can be established GAN1 With the second adversarial loss L GAN2 , where the first adversarial loss L GAN1 It is convenient to train the first generator G1 and the first discriminator D1, so as to improve the ability of the first generator G1 to make its converted image close to the real image, and also improve the ability of the first discriminator D1 to distinguish the converted image from the real image; through the second adversarial loss L GAN2 It is convenient to train the second generator G2 and the second discriminator D2, thereby improving the ability of the second generator G2 to make its converted images close to the real images, and also improving the ability of the second discriminator D2 to distinguish the converted images from the real images.

[0074] In some embodiments, the first domain image set and the second domain image set are input into an initial image generation model for processing, and the first image difference and the second image difference are output, including:

[0075] S1141, input the first domain image set into the first domain generator in the initial image generation model for processing, and output the first domain generated image; input the first domain generated image into the second domain generator in the initial image generation model for processing, and output the second domain return image.

[0076] Among them, the first domain generator is also the above-mentioned first generator G1, the first domain generated image is also the above-mentioned second domain image G1(A), the second domain generator is also the above-mentioned second generator G2, and the second domain return image is also the above-mentioned first domain image G2(G1(A)).

[0077] Specifically, taking a first domain image A in the first domain image set as an example, the first domain image A is input into the first domain generator G1 for conversion to obtain a first domain generated image G1(A). Further, the first domain generated image G1(A) is input into the second domain generator G2 for conversion to obtain a second domain returned image G2(G1(A)).

[0078] S1142: Determine a first image difference based on the first domain image set and the second domain returned image.

[0079] Among them, the first image difference is the difference between the first domain image A in the first domain image set and the corresponding second domain return image G2 (G1(A)). Exemplarily, the difference may be the pixel difference between the same pixel point in the first domain image and the second domain return image.

[0080] Specifically, the difference between the first domain image in the first domain image set and the corresponding second domain return image is calculated, and the difference is recorded as the first image difference.

[0081] S1143, input the second domain image set into the second domain generator in the initial image generation model for processing, and output the second domain generated image; input the second domain generated image into the first domain generator in the initial image generation model for processing, and output the first domain return image.

[0082] Among them, the second domain generator is also the above-mentioned second generator G2, the second domain generated image is also the above-mentioned first domain image G2(B), the first domain generator is also the above-mentioned first generator G1, and the first domain return image is also the above-mentioned second domain image G1(G2(B)).

[0083] Specifically, taking a second domain image B in the second domain image set as an example, the second domain image B is input into the second domain generator G2 for conversion to obtain a second domain generated image G2(B). Further, the second domain generated image G2(B) is input into the first domain generator G1 for conversion to obtain a first domain return image G1(G2(B)).

[0084] S1144: Determine a second image difference based on the second domain image set and the first domain returned image.

[0085] Among them, the second image difference is the difference between the second domain image B in the second domain image set and the corresponding first domain return image G1(G2(B)). Exemplarily, the difference may be the pixel difference between the same pixel point in the second domain image and the corresponding first domain return image.

[0086] Specifically, the difference between the second domain image and the corresponding first domain return image is calculated, and the difference is recorded as the second image difference.

[0087] It can be seen that in this embodiment, a first image difference and a second image difference can be established, wherein the first image difference can facilitate the establishment of a first consistency loss, and the second image difference can facilitate the establishment of a second consistency loss, and the first consistency loss and the second consistency loss can facilitate the subsequent establishment of an overall loss for optimizing the initial image generation model.

[0088] In some embodiments, determining a cycle consistency loss based on the sample mean group, the first image difference, and the second image difference includes:

[0089] S1151 . Determine a first consistency loss and a second consistency loss based on the sample mean group, the first image difference, and the second image difference, respectively.

[0090] The sample mean group includes a first sample mean E1 and a second sample mean E2. In this embodiment, the L1 norm is specifically used to calculate the first image difference and the second image difference. The first image difference can be expressed as ||(A-G2(G1(A)))||1, and the second image difference can be expressed as ||(B-G1(G2(B)))||1, wherein ||.||1 represents the L1 norm, which is used to calculate the difference between the corresponding two images. The first consistency loss The first consistency loss is the product E1*||(A-G2(G1(A)))||1 of the first sample mean E1 and the first image difference ||(A-G2(G1(A)))||1, and the product is recorded as the first consistency loss; the second consistency loss function is the product E2*||(B-G1(G2(B)))||1 of the second sample mean E2 and the second image difference ||(B-G1(G2(B)))||1, and the product is recorded as the second consistency loss.

[0091] Specifically, the product of the first sample mean E1 and the first image difference ||(A-G2(G1(A)))||1 is calculated to obtain the first consistency loss, and the product of the second sample mean E2 and the second image difference ||(B-G1(G2(B)))||1 is calculated to obtain the second consistency loss.

[0092] S1152. Determine a cycle consistency loss based on the first consistency loss and the second consistency loss.

[0093] Among them, the cycle consistency loss L CYC is the sum of the first consistency loss E1*||(A-G2(G1(A)))||1 and the second consistency loss E2*||(B-G1(G2(B)))||1, that is, L CYC =E1*||(A-G2(G1(A)))||1+E2*||(B-G1(G2(B)))||1.

[0094] Specifically, the sum of the first consistency loss E1*||(A-G2(G1(A)))||1 and the second consistency loss E2*||(B-G1(G2(B)))||1 is calculated to obtain the cycle consistency loss L CYC .

[0095] It can be seen that in this embodiment, the cycle consistency loss L can be calculated CYC , through the cycle consistency loss L CYC This facilitates improving the capabilities of the first domain generator G1 and the second domain generator G2 to keep the content of the image after two conversions as consistent as possible with the original image.

[0096] In some embodiments, generating an input image base map based on a preset character row size includes:

[0097] S121. Obtain a preset character row size.

[0098] Among them, the character row size is preset according to the size of the character row shown on a carrier in the actual scene, and the character row size includes the character row width W and the character row height H. The character row consists of a row of characters; exemplarily, in a supermarket shopping scene, the above-mentioned carrier is a shopping receipt, and the size of important character rows in the shopping receipt is obtained. The important character rows may be the character rows where the receipt generation time, receipt code, shopping payment amount, etc. are located.

[0099] Specifically, the size of the character row shown on some carriers in the actual scene is obtained, and the character row size is preset according to the size.

[0100] S122: Generate a corresponding character row frame on a preset base map template based on the character row size to obtain an input image base map.

[0101] Among them, the base map template is a black template, and the template has a corresponding template width and template height. For example, the base map template is as follows: Figure 3 The black template shown in the background; the character row size includes the character row width W and the character row height H. According to the character row size, a corresponding character row frame can be set on the base template. For example, the character row frame is as follows Figure 3 The gray strip box shown; the input image background is a background template with a character row box already set.

[0102] Specifically, a corresponding character row frame is set on a preset base map template according to the character row size, thereby obtaining an input image base map; the size of the character row frame is also the character row size.

[0103] It can be seen that in this embodiment, an input image base map can be generated, thereby preparing for setting a model input image for input into the target image generation model.

[0104] In some embodiments, determining a model input image based on the input image base image and characters corresponding to the character row size includes:

[0105] S131. Obtain the characters corresponding to the character row box.

[0106] The characters corresponding to the character line box can be the receipt generation time, receipt code, shopping payment amount, etc. Figure 3 The characters in the character row box are shown.

[0107] S132: Add the characters to the corresponding character row boxes in the input image base map to obtain a model input image.

[0108] Among them, the acquired characters can be processed into the corresponding character line frame in the input image base map by digital image processing technology, so as to obtain Figure 3 Model input image shown.

[0109] Specifically, the acquired characters are processed into corresponding character row frames in the input image base map by digital image processing technology, thereby obtaining a model input image.

[0110] It can be seen that in this embodiment, the generated input image base map and the preset character generation model input image can be used to generate the model input image, and then the target image generation model can be used to process the model input image to obtain Figure 4 The OCR training image shown.

[0111] It should be understood that, although the steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indications of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0112] Based on the same inventive concept, the embodiment of the present application also provides an OCR training image generation device. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more OCR training image generation device embodiments provided below can refer to the limitations of the OCR training image generation method above, and will not be repeated here.

[0113] like Figure 5 As shown, the embodiment of the present application provides an OCR training image generation device 200, comprising:

[0114] The model training module 210 is used to train the initial image generation model based on the preset first domain image set and the second domain image set to obtain a target image generation model.

[0115] The base map generation module 220 is used to generate an input image base map based on a preset character row size.

[0116] The input image generation module 230 is used to determine the model input image based on the input image background and the characters corresponding to the character row size.

[0117] The training image generation module 240 is used to input the model input image into the target image generation model for processing and output the OCR training image.

[0118] In some embodiments, in terms of training the initial image generation model to obtain the target image generation model based on the preset first domain image set and the second domain image set, the model training module 210 is specifically used to:

[0119] Sampling the preset first domain image set and the second domain image set to obtain a sample mean group;

[0120] The first domain image set and the second domain image set are respectively input into the initial image generation model for processing, and the first domain image probability and the second domain image probability are output;

[0121] Determine an adversarial loss group based on the sample mean group, the first domain image probability, and the second domain image probability;

[0122] Inputting the first domain image set and the second domain image set into the initial image generation model for processing, and outputting a first image difference and a second image difference;

[0123] determining a cycle consistency loss based on the sample mean group, the first image difference, and the second image difference;

[0124] Determine the overall loss based on the adversarial loss group and the cycle consistency loss;

[0125] The initial image generation model is trained based on the overall loss to obtain the target image generation model.

[0126] In some embodiments, the sample mean group includes a first sample mean corresponding to the first domain image set and a second sample mean corresponding to the second domain image set; in determining the adversarial loss group based on the sample mean group, the first domain image probability and the second domain image probability, the model training module 210 is specifically used to:

[0127] Determining a first adversarial loss based on the first sample mean, the second sample mean, and the first domain image probability;

[0128] Determining a second adversarial loss based on the first sample mean, the second sample mean, and the second domain image probability;

[0129] Based on the first adversarial loss and the second adversarial loss, an adversarial loss group is determined.

[0130] In some embodiments, in inputting the first domain image set and the second domain image set into the initial image generation model for processing and outputting the first image difference and the second image difference, the model training module 210 is specifically used to:

[0131] Input the first domain image set into the first domain generator in the initial image generation model for processing, and output the first domain generated image; input the first domain generated image into the second domain generator in the initial image generation model for processing, and output the second domain return image;

[0132] Determining a first image difference based on the first domain image set and the second domain returned image;

[0133] Input the second domain image set into the second domain generator in the initial image generation model for processing, and output the second domain generated image; input the second domain generated image into the first domain generator in the initial image generation model for processing, and output the first domain return image;

[0134] A second image difference is determined based on the second domain image set and the first domain return image.

[0135] In some embodiments, in determining the cycle consistency loss based on the sample mean group, the first image difference and the second image difference, the model training module 210 is specifically used to:

[0136] Determine a first consistency loss and a second consistency loss based on the sample mean group, the first image difference, and the second image difference, respectively;

[0137] A cycle consistency loss is determined based on the first consistency loss and the second consistency loss.

[0138] In some embodiments, in terms of generating an input image base map based on a preset character row size, the base map generating module 220 is specifically used to:

[0139] Get the preset character row size;

[0140] Based on the character row size, a corresponding character row frame is generated on a preset base map template to obtain the input image base map.

[0141] In some embodiments, in determining the model input image based on the input image background and the characters corresponding to the character row size, the input image generation module 230 is specifically used to:

[0142] Get the character corresponding to the character line box;

[0143] Add the characters to the corresponding character row boxes in the input image base map to obtain the model input image.

[0144] Each module in the above OCR training image generation device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0145] In some embodiments, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data related to the generation of OCR training images. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the steps in the above-mentioned OCR training image generation method are implemented.

[0146] In some embodiments, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the steps in the above-mentioned OCR training image generation method are implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen; the input device of the computer device can be a touch layer covering the display screen, or a key, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0147] Those skilled in the art will understand that Figure 6 or Figure 7 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0148] In some embodiments, a computer device is provided. The computer device includes a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the steps in the above method embodiments are implemented.

[0149] In some embodiments, Figure 8 The figure shows an internal structure diagram of a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0150] In some embodiments, a computer program product is provided. The computer program product includes a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0151] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0152] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited thereto.

[0153] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0154] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for generating an OCR training image, characterized in that: include: Based on the preset first domain image set and second domain image set, the initial image generation model is trained to obtain a target image generation model; Generate an input image base map based on a preset character row size; Determining a model input image based on the input image base image and characters corresponding to the character row size; The model input image is input into the target image generation model for processing, and an OCR training image is output.

2. The method according to claim 1, characterized in that: The method of training the initial image generation model based on the preset first domain image set and the second domain image set to obtain the target image generation model includes: Sampling the preset first domain image set and the second domain image set to obtain a sample mean group; Inputting the first domain image set and the second domain image set into an initial image generation model for processing respectively, and outputting a first domain image probability and a second domain image probability; Determining an adversarial loss group based on the sample mean group, the first domain image probability, and the second domain image probability; Inputting the first domain image set and the second domain image set into the initial image generation model for processing, and outputting a first image difference and a second image difference; determining a cycle consistency loss based on the sample mean group, the first image difference, and the second image difference; Determining an overall loss based on the adversarial loss group and the cycle consistency loss; The initial image generation model is trained based on the overall loss to obtain a target image generation model.

3. The method according to claim 2, characterized in that The sample mean group includes a first sample mean corresponding to the first domain image set and a second sample mean corresponding to the second domain image set; and determining the adversarial loss group based on the sample mean group, the first domain image probability and the second domain image probability includes: Determining a first adversarial loss based on the first sample mean, the second sample mean, and the first domain image probability; Determining a second adversarial loss based on the first sample mean, the second sample mean, and the second domain image probability; Based on the first adversarial loss and the second adversarial loss, an adversarial loss group is determined.

4. The method according to claim 2, characterized in that: The step of inputting the first domain image set and the second domain image set into the initial image generation model for processing and outputting a first image difference and a second image difference comprises: Input the first domain image set into the first domain generator in the initial image generation model for processing, and output a first domain generated image; input the first domain generated image into the second domain generator in the initial image generation model for processing, and output a second domain return image; determining a first image difference based on the first domain image set and the second domain returned image; Input the second domain image set into the second domain generator in the initial image generation model for processing, and output the second domain generated image; input the second domain generated image into the first domain generator in the initial image generation model for processing, and output the first domain return image; A second image difference is determined based on the second domain image set and the first domain return image.

5. The method according to claim 2, characterized in that: The determining of the cycle consistency loss based on the sample mean group, the first image difference and the second image difference comprises: Determine a first consistency loss and a second consistency loss based on the sample mean group, the first image difference, and the second image difference, respectively; A cycle consistency loss is determined based on the first consistency loss and the second consistency loss.

6. The method according to claim 1, characterized in that The step of generating an input image base map based on a preset character row size includes: Get the preset character row size; A corresponding character row frame is generated on a preset base map template based on the character row size to obtain an input image base map.

7. The method according to claim 6, characterized in that The step of determining a model input image based on the input image base image and characters corresponding to the character row size comprises: Obtain the character corresponding to the character row box; The characters are added to corresponding character row boxes in the input image base image to obtain a model input image.

8. An OCR training image generation device, characterized in that: include: A model training module, used to train the initial image generation model based on the preset first domain image set and the second domain image set to obtain a target image generation model; A base map generation module, used for generating an input image base map based on a preset character row size; An input image generation module, used for determining a model input image based on the input image base image and characters corresponding to the character row size; The training image generation module is used to input the model input image into the target image generation model for processing and output an OCR training image.

9. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images