Method, device, electronic device and storage medium for recognizing text in image

Abstract

The present application provides a method, device, electronic device and storage medium for identifying text in an image. (A) Construct a recognition model for predicting text in an image based on an image; (B) Construct a first training data set using multiple synthetic samples of first-category fields and multiple synthetic samples of second-category fields; (C) Obtain a first test data set including multiple real samples of first-category fields and multiple real samples of second-category fields; (D) Iteratively update the recognition model based on the first training data set and the first test data set to obtain an initial recognition model; (E) Construct a second training data set; (F) Iteratively update the initial recognition model to obtain a target recognition model; (G) Determine the text in the image to be detected based on the target recognition model and the image to be detected. According to the method and device, different types of field information can be accurately identified, thereby effectively improving the accuracy of the final recognition result.

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, electronic device and storage medium for recognizing text in an image. Background Art

[0002] Nowadays, industries such as banking, finance, insurance, public security, and logistics have to deal with a large amount of ticket information entry work every day. These businesses involve multiple types of tickets such as identity cards, driver's licenses, vehicle registration certificates, bank cards, etc. If they only rely on manual entry, a lot of manpower and material resources are required, and the work efficiency is low. Therefore, image recognition technology is needed to automate operations, thereby reducing manpower and material resources, and greatly improving work efficiency.

[0003] In actual business, the important information that needs to be collected in the ticket is often distributed in fields of different composition forms, and there are obvious differences between these fields. Therefore, how to achieve a high recognition accuracy rate for fields of various composition forms is an urgent problem to be solved. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for recognizing text in an image. By using training samples constructed with two types of fields, a target recognition model is obtained by stage-by-stage training. When recognizing text in an image, the accuracy of the final recognition result can be effectively improved.

[0005] The present application provides a method for recognizing text in an image, the method comprising:

[0006] (A) Building a recognition model for predicting text in an image based on the image;

[0007] (B) obtaining synthetic samples of the first category of fields and synthetic samples of the second category of fields, and constructing a first training data set using multiple synthetic samples of the first category of fields and multiple synthetic samples of the second category of fields;

[0008] (C) obtaining a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields;

[0009] (D) iteratively updating the recognition model based on the first training data set and the first test data set to obtain an initial recognition model;

[0010] (E) constructing a second training data set using the real samples of the plurality of first-category fields and the real samples of the plurality of second-category fields;

[0011] (F) iteratively updating the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model;

[0012] (G) Based on the target recognition model and the image to be detected, determine the text in the image to be detected.

[0013] Furthermore, the method further includes obtaining synthetic samples of the first type of fields and synthetic samples of the second type of fields through the following steps:

[0014] Extract background information and corpus information of sample images;

[0015] Filter out the first category of field information and the second category of field information in the corpus information;

[0016] Fusion of the first category field information and the background information to generate synthetic samples of the first category fields;

[0017] The second-category field information and the background information are integrated to generate synthetic samples of the second-category fields.

[0018] Further, the step (D) comprises:

[0019] (D1) extracting, from the first training data set, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process;

[0020] (D2) training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate a first-order recognition model;

[0021] (D3) training the first-order recognition model based on the extracted synthetic samples of the first category of fields and the extracted synthetic samples of the second category of fields according to a preset first sampling ratio to generate an initial recognition model;

[0022] (D4) performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields;

[0023] (D5) determining a first ratio adjustment parameter based on a first recognition accuracy rate of the first category of fields and a first recognition accuracy rate of the second category of fields;

[0024] (D6) determining whether the first ratio adjustment parameter satisfies a predetermined condition;

[0025] (D7) If the predetermined condition is met and the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted and the process returns to step (D3);

[0026] (D8) If the predetermined condition is not satisfied and the number of training times does not reach the second predetermined number of times, the process returns to step (D4).

[0027] Further, the step (D5) comprises:

[0028] Obtaining weights of the first category of fields and weights of the second category of fields;

[0029] The ratio adjustment parameter is determined based on the first recognition accuracy of the first category of fields, the weight of the first category of fields, the first recognition accuracy of the second category of fields, and the weight of the second category of fields.

[0030] Further, the step (F) comprises:

[0031] (F1) extracting, from the second training data set, according to a preset second sampling ratio, real samples of the first category field and real samples of the second category field used in a training step in a training process, and training the initial recognition model based on the extracted real samples of the first category field and the real samples of the second category field to generate a target recognition model;

[0032] (F2) performing a second test on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields;

[0033] (F3) determining a second ratio adjustment parameter based on the recognition accuracy of the first category of fields and the recognition accuracy of the second category of fields;

[0034] (F4) determining whether the second ratio adjustment parameter satisfies a predetermined condition;

[0035] (F5) if the predetermined condition is met, adjusting the preset second sampling ratio, and when the number of training times does not reach the third predetermined number of times, returning to step (F1);

[0036] (F6) If the predetermined condition is not satisfied and the number of training times does not reach the third predetermined number of times, the process returns to step (F2).

[0037] Furthermore, after generating the target recognition model, the method further includes:

[0038] Determine, based on the first test data set, a third recognition accuracy rate of the target recognition model for the first category of fields and a third recognition accuracy rate of the second category of fields;

[0039] Determine whether the third recognition accuracy rate of the target recognition model for the first category of fields and the third recognition accuracy rate of the second category of fields reach corresponding accuracy thresholds respectively;

[0040] If it is reached, determine the target recognition model as the target recognition model for executing step (G);

[0041] If not reached, update the second training data set, and retrain the initial recognition model based on the updated second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds.

[0042] Furthermore, the first category of fields includes Chinese character fields; and the fields in the second category of synthetic field samples include at least one of numeric fields and English fields.

[0043] The present application also provides a device for recognizing text in an image, the device comprising:

[0044] A recognition model building module, used to build a recognition model for predicting text in an image based on the image;

[0045] A first training set construction module, used to obtain synthetic samples of the first category of fields and synthetic samples of the second category of fields, and construct a first training data set using multiple synthetic samples of the first category of fields and multiple synthetic samples of the second category of fields;

[0046] A test set construction module, used to obtain a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields;

[0047] An initial recognition model determination module is used to iteratively update the recognition model based on the first training data set and the first test data set to obtain an initial recognition model;

[0048] A second training set construction module, used to construct a second training data set using the real samples of the multiple first-category fields and the real samples of the multiple second-category fields;

[0049] A target recognition model determination module is used to iteratively update the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model;

[0050] The detection module is used to determine the text in the image to be detected based on the target recognition model and the image to be detected.

[0051] Furthermore, the first training set construction module is further used to obtain synthetic samples of the first category of fields and synthetic samples of the second category of fields through the following steps:

[0052] Extract background information and corpus information of sample images;

[0053] Filter out the first category of field information and the second category of field information in the corpus information;

[0054] Fusion of the first category field information and the background information to generate synthetic samples of the first category fields;

[0055] The second-category field information and the background information are integrated to generate synthetic samples of the second-category fields.

[0056] Furthermore, the initial recognition model determination module is used to:

[0057] Extracting, from the first training data set, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process;

[0058] Training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate a first-order recognition model;

[0059] According to a preset first sampling ratio, the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model;

[0060] Performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields;

[0061] Determine a first ratio adjustment parameter based on a first recognition accuracy rate of the first category of fields and a first recognition accuracy rate of the second category of fields;

[0062] determining whether the first ratio adjustment parameter satisfies a predetermined condition;

[0063] If the predetermined condition is met, when the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted, and according to the adjusted preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process are extracted from the first training data set, and the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model;

[0064] If the predetermined condition is not met and the number of training times does not reach a second predetermined number of times, a first test is performed on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields.

[0065] Furthermore, the initial recognition model determination module is also used for:

[0066] Obtaining weights of the first category of fields and weights of the second category of fields;

[0067] The ratio adjustment parameter is determined based on the first recognition accuracy of the first category of fields, the weight of the first category of fields, the first recognition accuracy of the second category of fields, and the weight of the second category of fields.

[0068] Furthermore, the target recognition model determination module is used to:

[0069] According to a preset second sampling ratio, extracting real samples of the first category of fields and real samples of the second category of fields used in a training step in a training process from the second training data set, training the initial recognition model based on the extracted real samples of the first category of fields and the real samples of the second category of fields, and generating a target recognition model;

[0070] Performing a second test on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields;

[0071] Determining a second ratio adjustment parameter based on the recognition accuracy of the first category of fields and the recognition accuracy of the second category of fields;

[0072] determining whether the second ratio adjustment parameter satisfies a predetermined condition;

[0073] If the predetermined condition is met, the preset second sampling ratio is adjusted, and when the number of training times does not reach the third predetermined number of times, the real samples of the first category field and the real samples of the second category field used in one training step in one training process are extracted from the second training data set according to the adjusted preset second sampling ratio, and the initial recognition model is trained based on the extracted real samples of the first category field and the real samples of the second category field to generate a target recognition model;

[0074] If the predetermined condition is not met and the number of training times does not reach a third predetermined number of times, a second test is performed on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields.

[0075] Further, after using the target recognition model determination module to generate the target recognition model, the target recognition model determination module is further used to:

[0076] Determine, based on the first test data set, a third recognition accuracy rate of the target recognition model for the first category of fields and a third recognition accuracy rate of the second category of fields;

[0077] Determine whether the third recognition accuracy rate of the target recognition model for the first category of fields and the third recognition accuracy rate of the second category of fields reach corresponding accuracy thresholds respectively;

[0078] If the target recognition model is reached, the target recognition model is determined to be a target recognition model for the text in the image to be detected based on the target recognition model and the image to be detected;

[0079] If not reached, update the second training data set, and retrain the initial recognition model based on the updated second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds.

[0080] Furthermore, the first category of fields includes Chinese character fields; the second category of fields includes at least one of numeric fields and English fields.

[0081] An embodiment of the present application also provides an electronic device, including: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for recognizing text in an image as described above are performed.

[0082] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method for recognizing text in an image as described above are executed.

[0083] The embodiments of the present application provide a method, device, electronic device and storage medium for recognizing text in an image. First, a recognition model for predicting text in an image based on an image is constructed, and a first training data set is constructed using synthetic samples of multiple first-category fields and synthetic samples of multiple second-category fields; and a first test data set including real samples of multiple first-category fields and real samples of multiple second-category fields is obtained; then, based on the first training data set and the first test data set, the recognition model is iteratively updated to obtain an initial recognition model; and a second training data set is constructed using real samples of the multiple first-category fields and real samples of the multiple second-category fields; finally, based on the second training data set and the first test data set, the initial recognition model is iteratively updated to obtain a target recognition model; based on the target recognition model and the image to be detected, the text in the image to be detected is determined. In this way, when the present application recognizes the text in the image to be detected, different types of field information can be accurately identified, thereby effectively improving the accuracy of the final recognition result.

[0084] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0086] Figure 1 A flowchart of a method for recognizing text in an image provided by an embodiment of the present application;

[0087] Figure 2 A flowchart of a method for generating an initial recognition model provided in an embodiment of the present application;

[0088] Figure 3 A flowchart of a method for generating a target recognition model provided in an embodiment of the present application;

[0089] Figure 4 A flowchart of a method for determining a target recognition model provided in an embodiment of the present application;

[0090] Figure 5 A schematic diagram of the structure of a device for recognizing text in an image provided by an embodiment of the present application;

[0091] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0092] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0093] Research has found that nowadays, industries such as banking, finance, insurance, public security, and logistics have to deal with a large amount of ticket information entry work every day. These businesses involve multiple types of tickets such as identity cards, driver's licenses, vehicle registration certificates, bank cards, etc., and the important information in these tickets is often distributed in different types of fields, and there are obvious differences between these fields. Therefore, how to achieve a high recognition accuracy for various types of fields is an urgent problem to be solved.

[0094] Based on this, an embodiment of the present application provides a method for recognizing text in an image. When recognizing text in an image to be detected, information of different types of fields can be accurately identified, thereby effectively improving the accuracy of the final recognition result.

[0095] See also Figure 1 , Figure 1 A flowchart of a method for recognizing text in an image provided by an embodiment of the present application. Figure 1 As shown in , the method for recognizing text in an image provided by an embodiment of the present application includes:

[0096] Step S101, constructing a recognition model for predicting text in an image based on the image.

[0097] In this step, a recognition model for recognizing text in an image is constructed using an existing method. The recognition model includes a general model and an adaptation structure related to the recognition task.

[0098] Step S102, obtaining synthetic samples of the first category field and synthetic samples of the second category field, and constructing a first training data set using multiple synthetic samples of the first category field and multiple synthetic samples of the second category field. Here, as an example, the first category field includes a Chinese character field; the second category field includes at least one of a numeric field and an English field.

[0099] As an example, the method may additionally include the steps of obtaining synthetic samples of the first category of fields and synthetic samples of the second category of fields. Specifically, before step S102, first, background information and corpus information of the sample image are extracted, and secondly, the first category of field information and the second category of field information in the corpus information are screened out, and then, the first category of field information and the background information are fused to generate synthetic samples of the first category of fields, and finally, the second category of field information and the background information are fused to generate synthetic samples of the second category of fields.

[0100] Specifically, a background image and some corpus information of the sample can be obtained, and then multiple first-category field information and multiple second-category field information that meet the requirements can be screened out from the corpus information. The first-category field information and the background image can be fused, blurred, and processed to generate synthetic information of the first-category field, and the second-category field information and the background image can be fused, blurred, and processed to generate synthetic information of the second-category field. The first training data set can be constructed using the generated synthetic samples of multiple first-category fields and the generated synthetic samples of multiple second-category fields. Here, those skilled in the art can generate a certain number of synthetic samples of the first-category fields and a certain number of synthetic samples of the second-category fields according to the actual situation.

[0101] In the following, specific examples are used to explain in detail how to generate synthetic samples of the first type of field and synthetic samples of the second type of field.

[0102] For example, the recognition model constructed is used to identify the driving license as an example to illustrate the fusion process of the synthetic samples of the first type of field and the synthetic samples of the second type of field. Specifically, according to the text marked on the driving license, the content corresponding to the labels such as vehicle type, owner, address, usage nature, brand number, etc. can be divided into the first type of field, and the content corresponding to the labels such as license plate number, vehicle identification code, engine number, registration date, etc. can be divided into the second type of field. First, the background image of the driving license is obtained, and the background image may include multiple labels, and the content corresponding to the label is empty; then, according to the text filling rules corresponding to the labels in the driving license, the image of the text sample is generated from the news corpus. For example: the vehicle identification number is composed of 17 English numbers and the English letters "I", "O", and "Q" will not be used, so the text sample image corresponding to the vehicle identification number generated by the news corpus meets the corresponding text filling rules, so that a large number of text sample images are generated. Then, according to the principle of dividing the field type of the label on the driving license, the text sample image and the background image are fused, blurred, and other operations can be performed to generate a synthetic sample. When generating synthetic samples, a text sample image of the first category of fields is selected from the generated text sample images, and the image contains at least one text content corresponding to the first category of field labels in the driving license, and the text sample is fused with the background, etc., to generate a synthetic sample of the first category of fields; a text sample image of the second category of fields is selected from the generated text sample images, and the image contains at least one text content corresponding to the second category of field labels in the driving license, and the text sample is fused with the background, etc., to generate a synthetic sample of the second category of fields. Among them, the total number of synthetic samples of the first category of fields and synthetic samples of the second category of fields generated by this application is 2.5 million, and the ratio of synthetic samples of the first category of fields to synthetic samples of the second category of fields is 2:23. It should be noted that the total number of synthetic samples of the first category of fields and synthetic samples of the second category of fields and the ratio of synthetic samples of the first category of fields to synthetic samples of the second category of fields can be selected and adjusted for applicability according to influencing conditions such as ticket type.

[0103] Step S103: Acquire a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields.

[0104] In this step, a large number of real images of the same type as the image to be detected are first obtained, and then the obtained real images are classified, cropped, labeled, etc. according to the different types of fields in the real images to generate multiple real samples of the first category of fields and multiple real samples of the second category of fields. The first test data set can be formed using multiple real samples of the first category of fields and multiple real samples of the second category of fields.

[0105] For example, the recognition model constructed is used to identify driving licenses. 1,000 real driving licenses can be randomly selected, and the corresponding labels on the driving licenses and the text information corresponding to the labels can be selected according to the field type classification principle. At least one label and the corresponding text information are selected. After the selection is completed, multiple real samples of the first type of fields and multiple real samples of the second type of fields are respectively formed. The real samples of the first type of fields and the real samples of the second type of fields constitute the first test data set. Here, 1,000 real driving licenses are selected, but the generated first test data set can exceed 1,000.

[0106] Step S104, iteratively updating the recognition model based on the first training data set and the first test data set to obtain an initial recognition model. Here, as an example, the method may additionally include the step of iteratively updating the recognition model based on the first training data set and the first test data set to obtain an initial recognition model. For details, please refer to Figure 2 , Figure 2 Flow chart of the method for generating an initial recognition model provided in the embodiment of the present application. Figure 2 As shown in , the method for generating an initial recognition model provided by an embodiment of the present application includes:

[0107] Step S201: extracting, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process from the first training data set.

[0108] Here, as an example, the preset first sampling ratio can be determined by considering factors such as the ticket type and the field type corresponding to the synthetic sample, and then the number of synthetic samples participating in the training in each training step of each training can be determined based on the computer computing power and the total number of synthetic samples. Then, the number of synthetic samples of the first category of fields and the number of synthetic samples of the second category of fields that need to be extracted in each training step for training can be determined.

[0109] For example, the recognition model constructed is used to identify driving licenses. The preset first sampling ratio is 23:2. As an example, the number of synthetic samples required to participate in training for each training step is set to 256, where the preset first sampling ratio of synthetic samples of the first category of fields and synthetic samples of the second category of fields is 23:2. That is, it is determined that the number of synthetic samples of the first category of fields participating in training is 236, and the number of synthetic samples of the second category of fields participating in training is 20. Here, when calculating, if the number obtained is not an integer, a rounding operation is performed. For the choice of rounding down or rounding up, the applicability is selected according to the target, as long as the total number of synthetic samples required to participate in training for each set training step is guaranteed to remain unchanged.

[0110] Here, it should be understood that the above-mentioned preset first sampling ratio can be any ratio set according to the actual scenario, and is not limited to the above-mentioned ratio, and the present invention does not make any limitation here.

[0111] Step S202: Training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the extracted synthetic samples of the second category of fields to generate a first-order recognition model.

[0112] In this step, the first predetermined number of times is determined based on factors such as the number of synthetic samples of the first category of fields and the synthetic samples of the second category of fields and the difficulty of detection. Then, the first predetermined number of trainings are performed using the synthetic samples of the first category of fields and the synthetic samples of the second category of fields extracted according to a preset first sampling ratio. After the training is completed, a first-order recognition model is generated.

[0113] For example, taking the constructed recognition model for identifying driving licenses as an example, assuming that the first predetermined number of times is 10 times, the recognition model can be trained 10 times using all synthetic samples of the first category of fields and synthetic samples of the second category of fields to generate a first-order recognition model.

[0114] Step S203: According to a preset first sampling ratio, the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model.

[0115] Here, when the training starts, synthetic samples of the first category of fields and synthetic samples of the second category of fields are extracted according to the preset first sampling ratio to participate in the training. The trained model is a first-order recognition model that has been trained before. When a certain number of training times is reached, the training is stopped and an initial recognition model is generated.

[0116] Step S204: performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields.

[0117] In this step, when executing step S203, a test node is set at intervals of a certain training step length, and then each time a test node is reached, the first test data set is used to perform field recognition performance testing on the initial recognition model being trained at this moment, and the first recognition accuracy rate of the initial recognition model for the first category of fields and the first recognition accuracy rate of the second category of fields are obtained.

[0118] Here, it is possible to perform detection only on each test node and then feedback the test result. The interval between two test nodes may be a certain training step length. The interval training step length can be set as appropriate.

[0119] For example, taking the constructed recognition model for identifying driving licenses as an example, the interval training step is set to 1000 training steps, and the first test data set is used to test the initial recognition model trained to this point, and the first recognition accuracy rate L1 of the initial recognition model for the first category of fields and the first recognition accuracy rate L2 of the second category of fields are obtained.

[0120] Step S205, determining a first ratio adjustment parameter based on the first recognition accuracy of the first category field and the first recognition accuracy of the second category field. Here, as an example, the method may additionally include: obtaining the weight of the first category field and the weight of the second category field; determining the first ratio adjustment parameter based on the first recognition accuracy of the first category field, the weight of the first category field, the first recognition accuracy of the second category field, and the weight of the second category field.

[0121] In this step, the weights of the first category fields and the second category fields are first obtained, and then the ratio adjustment parameter can be determined based on the first recognition accuracy of the first category fields and the first recognition accuracy of the second category fields and the corresponding calculation formula.

[0122] Here, the weight W1 of the first type of field and the weight W2 of the second type of field can be set according to the difficulty of identifying the field type, and the proportion adjustment parameter is recorded as A.

[0123] Exemplarily, taking the constructed recognition model for identifying driving licenses as an example, the weight W1 of the first type of field is set to 1.05, the weight W2 of the second type of field is set to 1, and the calculation formula for setting the proportional adjustment parameter A is A=W1*L1-W2*L2.

[0124] Step S206: determine whether the first ratio adjustment parameter meets a predetermined condition.

[0125] In this step, a first ratio adjustment parameter of the synthetic sample is obtained, and the obtained first ratio adjustment parameter is compared with a set parameter in a predetermined condition to determine whether the predetermined condition is met.

[0126] Here, the predetermined conditions can be determined based on parameters such as field type, total number of samples, number of samples involved in each training step, and computer computing power.

[0127] For example, the constructed recognition model is used to identify the driving license. Assume that the predetermined condition is whether the first proportional adjustment parameter A is greater than 0. It is set that when A is less than 0, the predetermined condition is met, and when A is greater than or equal to 0, the predetermined condition is not met.

[0128] If the predetermined condition is met, in step S207, when the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted and the process returns to step S203.

[0129] In this step, when the ratio adjustment parameter meets the predetermined conditions and the number of training times has not reached the second predetermined number of times, the preset first sampling ratio will be adjusted, and the adjusted first sampling ratio will be returned to step S203. Step S203 re-extracts the synthetic samples of the first category of fields and the synthetic samples of the second category of fields based on the adjusted preset first sampling ratio, and performs the training process of the initial recognition model.

[0130] Here, the process of adjusting the preset first sampling ratio is a dynamic adjustment process, and each adjustment is performed on the basis of the last adjustment.

[0131] For example, taking the recognition model constructed for identifying a driving license as an example, assuming that the second predetermined number of times is 10 times, assuming that the adjustment of the preset first sampling ratio in the predetermined condition is to adjust the preset first sampling ratio by increasing the number n of synthetic samples of the first category of fields participating in the training, the specific calculation formula for increasing the number n is: In this example, the detection object is a driving license, and the maximum value of n can be set to 10. It should be noted that the value of n can be adjusted according to the applicability of the actual situation. The first-order recognition model is continuously trained 10 times using synthetic samples of two types of fields according to the adjusted preset first sampling ratio to generate an initial recognition model.

[0132] If the predetermined condition is not met, in step S208, when the number of training times does not reach the second predetermined number of times, the process returns to step S204.

[0133] In this step, when the ratio adjustment parameter does not meet the predetermined conditions, there is no need to adjust the preset first sampling ratio, and when the number of training times does not reach the second predetermined number, return to step S204 and re-use the first test data set to test the initial recognition model.

[0134] Here, return to step S204, re-test, test on each test node, and then feedback the test result. The two test nodes may be separated by a certain training step length. The training step length of the interval can be set according to the applicability.

[0135] Return to reference Figure 1 , step S105, constructing a second training data set using the real samples of the multiple first-category fields and the real samples of the multiple second-category fields.

[0136] In this step, the image acquisition and image processing method for constructing the second training data set is the same as the method for constructing the first test data set. The difference is that the number of real samples of the two types of fields contained in the two data sets may be different and / or the ratio between the real samples of the first type of field and the real samples of the second type of field in the two data sets may be different.

[0137] Step S106, iteratively updating the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model. Here. As an example, the method may additionally include the step of iteratively updating the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model. For details, please refer to Figure 3 , Figure 3 Flow chart of the method for generating a target recognition model provided in the embodiment of the present application. Figure 3 As shown in , the method for generating an initial recognition model provided by an embodiment of the present application includes:

[0138] Step S301, according to a preset second sampling ratio, extract the real samples of the first category field and the real samples of the second category field used in a training step in a training process from the second training data set, train the initial recognition model based on the extracted real samples of the first category field and the real samples of the second category field, and generate a target recognition model.

[0139] Here, as an example, the preset second sampling ratio is determined according to factors such as ticket type and computer computing power, and the number of samples participating in the training in each training step is determined, that is, the number of real samples of the first category field and the real samples of the second category field that need to be extracted for training is determined. Then, the determined real samples of the first category field and the real samples of the second category field are used for training until the training is completed to generate a target recognition model.

[0140] Here, the recognition model constructed for identifying driving licenses is used as an example for explanation, assuming that the number of real samples required for each training step is the same as that in step S201, and assuming that the preset second sampling ratio of the real samples of the first type of field and the real samples of the second type of field is 4:1. The calculation formula for determining the number of real samples of the two types of fields participating in the training is the same as that in step S201, and will not be repeated here.

[0141] Here, it should be understood that the above-mentioned preset second sampling ratio can be any ratio set according to the actual scenario, and is not limited to the above-mentioned ratio, and the present invention does not make any limitation here.

[0142] Step S302: Perform a second test on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields.

[0143] In this step, when training is performed in step S301, a test node is set at intervals of a certain training step length, and then each time a test node is reached, the recognition performance of the initial recognition model being trained at this moment is tested using the first test data set to obtain the second recognition accuracy rate of the first category of fields and the second recognition accuracy rate of the second category of fields for the initial recognition model trained to this moment.

[0144] Here, detection can be performed only on each test node, and then the test result can be fed back. The interval between two test nodes can be a certain amount of training steps, and the interval training step can be set as appropriate.

[0145] Step S303: determining a second ratio adjustment parameter based on the second recognition accuracy of the first category of fields and the second recognition accuracy of the second category of fields.

[0146] In this step, first obtain the weight coefficient corresponding to the first category of fields and the weight coefficient corresponding to the second category of fields, and then calculate the second recognition accuracy of the first category of fields and the second recognition accuracy of the second category of fields based on the initial recognition model trained to this moment, and determine the second proportion adjustment parameter of the real sample.

[0147] Step S304: determine whether the second ratio adjustment parameter meets a predetermined condition.

[0148] In this step, a second ratio adjustment parameter of a real sample is obtained, and the obtained ratio adjustment parameter is compared with a parameter in a preset condition to determine whether the preset condition is met.

[0149] Here, the predetermined condition can be determined according to parameters such as field type, total number of samples, number of samples involved in each training step, and computer computing power. Here, as an example, it can be assumed that the predetermined condition is the same as the predetermined condition for training and generating the initial recognition model in step S206.

[0150] Step S305: if the predetermined condition is met, adjust the preset second sampling ratio, and when the number of training times does not reach the third predetermined number of times, return to step S301.

[0151] In this step, when the second ratio adjustment parameter meets the predetermined conditions and the number of training times has not reached the third predetermined number of times, the preset second sampling ratio will be adjusted, and the adjusted preset second sampling ratio will be returned to step S301. Step S301 will perform the training process of the target recognition model on the real samples of the first category of fields and the real samples of the second category of fields that are re-extracted based on the adjusted second sampling ratio.

[0152] Here, the process of adjusting the preset second sampling ratio is a dynamic adjustment process, and each adjustment is performed on the basis of the preset second sampling ratio after the last adjustment. The third predetermined number of times is set based on the actual situation.

[0153] Step S306: if the predetermined condition is not met and the number of training times does not reach the third predetermined number of times, return to step S302.

[0154] In this step, when the second ratio adjustment parameter does not meet the predetermined conditions, there is no need to adjust the preset second sampling ratio, and when the number of training times does not reach the third predetermined number, return to step S402 and re-use the first test data set to test the initial recognition model in training.

[0155] Here, return to step S302, re-test, test each test node, and then feedback the test result. The two test nodes may be separated by a certain training step length. The training step length of the interval can be set as appropriate.

[0156] As an example, after step S301, the method may additionally include a step of determining the target recognition model. For details, see Figure 4 , Figure 4 Flow chart of the method for determining the target recognition model provided in the embodiment of the present application. Figure 4 As shown in , the determination method of the target recognition model provided in the embodiment of the present application includes:

[0157] Step S401: determining a third recognition accuracy rate of a target recognition model for a first category of fields and a third recognition accuracy rate of a second category of fields based on the first test data set.

[0158] In this step, the trained target recognition model is used to recognize the first category of fields and the second category of fields in the first test data set, and the third recognition accuracy rate of the target recognition model for the first category of fields and the third recognition accuracy rate of the second category of fields are determined.

[0159] Step S402 , determining whether the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach corresponding accuracy thresholds respectively.

[0160] In this step, the third recognition accuracy of the first category of fields using the target recognition model is compared with the preset recognition accuracy of the first category of fields, and the third recognition accuracy of the second category of fields using the target recognition model is compared with the preset recognition accuracy of the second category of fields to determine whether the corresponding accuracy threshold is reached.

[0161] Here, the accuracy threshold may be set in advance, or the recognition accuracy of the first category of fields and the recognition accuracy of the second category of fields generated by the initial recognition model may be compared to determine whether the recognition accuracy has been improved.

[0162] Step S403, if reached, determining the target recognition model is based on the target recognition model and the image to be detected, and determining the target recognition model of the text in the image to be detected.

[0163] In this step, if the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds, the training process is completed, and the target recognition model is the target recognition model that finally determines the text in the image to be detected.

[0164] Step S404, if not reached, update the second training data set, and retrain the initial recognition model based on the updated second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds.

[0165] In this step, if the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields do not reach the preset accuracy threshold, the second training data set is adjusted, and the training process from step S301 to step S306 is repeated based on the adjusted second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy threshold.

[0166] Here, the second training data set may be adjusted by increasing the number of samples in the second training data set or performing equal replacement of samples in the second training data set, thereby achieving the purpose of adjusting the second training data set.

[0167] Return to reference Figure 1 , step S107, determining the text in the image to be detected based on the target recognition model and the image to be detected.

[0168] In this step, the image to be detected is input into the target recognition model, and the target recognition model can extract text information from the image to be detected and feed back the extracted text information to the user.

[0169] Here, the recognized text information can be fed back to the user in a visual manner.

[0170] A method for recognizing text in an image provided by an embodiment of the present application, when recognizing text information in an image to be detected, first, a recognition model for predicting text in an image based on an image is constructed, and a first training data set is constructed using synthetic samples of multiple first-category fields and synthetic samples of multiple second-category fields; and a first test data set including real samples of multiple first-category fields and real samples of multiple second-category fields is obtained; then, based on the first training data set and the first test data set, the recognition model is iteratively updated to obtain an initial recognition model; and a second training data set is constructed using the real samples of the multiple first-category fields and the real samples of the multiple second-category fields; finally, based on the second training data set and the first test data set, the initial recognition model is iteratively updated to obtain a target recognition model; and based on the target recognition model and the image to be detected, the text in the image to be detected is determined.

[0171] In this way, when recognizing text in the image to be detected, the present application first uses a large number of synthetic samples to train an initial recognition model, and then uses a small number of real samples to fine-tune the initial recognition model to generate a target recognition model that meets the requirements. By first training with synthetic samples, the cost of manual labeling is reduced, and then fine-tuning is performed using real samples, so that the finally generated target recognition model has a higher recognition accuracy. Therefore, when recognizing text in the image to be detected, information of different types of fields can be accurately identified, effectively improving the accuracy of the recognition results.

[0172] See also Figure 5 , Figure 5 A schematic diagram of a device for recognizing text in an image provided by an embodiment of the present application is shown in FIG. Figure 5 As shown in , the identification device 500 includes:

[0173] A recognition model building module 510, for building a recognition model for predicting text in an image based on the image;

[0174] A first training set construction module 520, configured to obtain synthetic samples of the first category of fields and synthetic samples of the second category of fields, and construct a first training data set using multiple synthetic samples of the first category of fields and multiple synthetic samples of the second category of fields;

[0175] A test set construction module 530 is used to obtain a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields;

[0176] An initial recognition model determination module 540 is used to iteratively update the recognition model based on the first training data set and the first test data set to obtain an initial recognition model;

[0177] A second training set construction module 550, configured to construct a second training data set using the multiple real samples of the first category fields and the multiple real samples of the second category fields;

[0178] The target recognition model determination module 560 is used to iteratively update the initial recognition model based on the second training data set and the first test data set to obtain the target recognition model;

[0179] The detection module 570 is used to determine the text in the image to be detected based on the target recognition model and the image to be detected.

[0180] Furthermore, the first training set construction module 520 is further configured to obtain synthetic samples of the first category of fields and synthetic samples of the second category of fields through the following steps:

[0181] Extract background information and corpus information of sample images;

[0182] Filter out the first category of field information and the second category of field information in the corpus information;

[0183] Fusion of the first category field information and the background information to generate synthetic samples of the first category fields;

[0184] The second-category field information and the background information are integrated to generate synthetic samples of the second-category fields.

[0185] Furthermore, the initial recognition model determination module 540 is also used for:

[0186] Extracting, from the first training data set, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process;

[0187] Training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate a first-order recognition model;

[0188] According to a preset first sampling ratio, the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model;

[0189] Performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields;

[0190] Determine a first ratio adjustment parameter based on a first recognition accuracy rate of the first category of fields and a first recognition accuracy rate of the second category of fields;

[0191] determining whether the first ratio adjustment parameter satisfies a predetermined condition;

[0192] If the predetermined condition is met, when the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted, and according to the adjusted preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process are extracted from the first training data set, and the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model;

[0193] If the predetermined condition is not met and the number of training times does not reach a second predetermined number of times, a first test is performed on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields.

[0194] Furthermore, the initial recognition model determination module 540 is also used for:

[0195] Obtaining weights of the first category of fields and weights of the second category of fields;

[0196] The ratio adjustment parameter is determined based on the recognition accuracy of the first category of fields, the weight of the first category of fields, the recognition accuracy of the second category of fields, and the weight of the second category of fields.

[0197] Furthermore, the target recognition model determination module 560 is used to:

[0198] According to a preset second sampling ratio, extracting real samples of the first category of fields and real samples of the second category of fields used in a training step in a training process from the second training data set, training the initial recognition model based on the extracted real samples of the first category of fields and the real samples of the second category of fields, and generating a target recognition model;

[0199] Performing a second test on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields;

[0200] Determining a second ratio adjustment parameter based on the recognition accuracy of the first category of fields and the recognition accuracy of the second category of fields;

[0201] determining whether the second ratio adjustment parameter satisfies a predetermined condition;

[0202] If the predetermined condition is met, the preset second sampling ratio is adjusted, and when the number of training times does not reach the third predetermined number of times, the real samples of the first category field and the real samples of the second category field used in one training step in one training process are extracted from the second training data set according to the adjusted preset second sampling ratio, and the initial recognition model is trained based on the extracted real samples of the first category field and the real samples of the second category field to generate a target recognition model;

[0203] If the predetermined condition is not met and the number of training times does not reach a third predetermined number of times, a second test is performed on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields.

[0204] Further, after the target recognition model is generated by using the target recognition model determination module 560, the target recognition model determination module 560 is further used to:

[0205] Determine, based on the first test data set, a third recognition accuracy rate of the target recognition model for the first category of fields and a third recognition accuracy rate of the second category of fields;

[0206] Determine whether the third recognition accuracy rate of the target recognition model for the first category of fields and the third recognition accuracy rate of the second category of fields reach corresponding accuracy thresholds respectively;

[0207] If the target recognition model is reached, the target recognition model is determined to be a target recognition model for the text in the image to be detected based on the target recognition model and the image to be detected;

[0208] If not reached, update the second training data set, and retrain the initial recognition model based on the updated second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds.

[0209] Furthermore, the first category of fields includes Chinese character fields; the second category of fields includes at least one of numeric fields and English fields.

[0210] The embodiment of the present application provides a device for recognizing text in an image. When recognizing text information in an image to be detected, first, a recognition model for predicting text in an image based on an image is constructed, and a first training data set is constructed using synthetic samples of multiple first-category fields and synthetic samples of multiple second-category fields; and a first test data set including real samples of multiple first-category fields and real samples of multiple second-category fields is obtained; then, based on the first training data set and the first test data set, the recognition model is iteratively updated to obtain an initial recognition model; and a second training data set is constructed using real samples of the multiple first-category fields and real samples of the multiple second-category fields; finally, based on the second training data set and the first test data set, the initial recognition model is iteratively updated to obtain a target recognition model; based on the target recognition model and the image to be detected, the text in the image to be detected is determined. In this way, when recognizing text in an image to be detected, the present application can accurately identify information of different types of fields, thereby effectively improving the accuracy of the final recognition result.

[0211] See also Figure 6 , Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 6 As shown in , the electronic device 600 includes a processor 610 , a memory 620 and a bus 630 .

[0212] The memory 620 stores machine-readable instructions executable by the processor 610. When the electronic device 600 is running, the processor 610 communicates with the memory 620 via the bus 630. When the machine-readable instructions are executed by the processor 610, the above-mentioned Figure 1 as well as Figure 4 The steps of the method for recognizing text in an image in the method embodiment shown in the figure, and the specific implementation manner can be referred to the method embodiment, which will not be described in detail here.

[0213] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 as well as Figure 4 The steps of the method for recognizing text in an image in the method embodiment shown in the figure, and the specific implementation manner can be referred to the method embodiment, which will not be described in detail here.

[0214] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0215] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0216] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0217] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0218] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0219] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for recognizing text in an image, characterized in that: The method comprises: (A) Building a recognition model for identifying text in images; (B) obtaining synthetic samples of the first category of fields and synthetic samples of the second category of fields, and constructing a first training data set using multiple synthetic samples of the first category of fields and multiple synthetic samples of the second category of fields; (C) obtaining a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields; (D) iteratively updating the recognition model based on the first training data set and the first test data set to obtain an initial recognition model; (E) constructing a second training data set using the real samples of the plurality of first-category fields and the real samples of the plurality of second-category fields; (F) iteratively updating the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model; (G) determining text in the image to be detected based on the target recognition model and the image to be detected; The step (D) comprises: (D1) extracting, from the first training data set, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process; (D2) training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the extracted synthetic samples of the second category of fields to generate a first-order recognition model; (D3) training the first-order recognition model based on the extracted synthetic samples of the first category of fields and the extracted synthetic samples of the second category of fields according to a preset first sampling ratio to generate an initial recognition model; (D4) performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields; (D5) determining a first ratio adjustment parameter based on the first recognition accuracy rate of the first category of fields and the first recognition accuracy rate of the second category of fields; (D6) determining whether the first ratio adjustment parameter satisfies a predetermined condition; (D7) If the predetermined condition is met and the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted and the process returns to step (D3); (D8) If the predetermined condition is not met and the number of training times does not reach the second predetermined number of times, return to step (D4); The step (D5) comprises: Obtaining weights of the first category of fields and weights of the second category of fields; A first ratio adjustment parameter is determined based on the first recognition accuracy of the first category of fields, the weight of the first category of fields, the first recognition accuracy of the second category of fields, and the weight of the second category of fields.

2. The method according to claim 1, characterized in that The following steps are used to obtain synthetic samples of the first type of field and synthetic samples of the second type of field: Extract background information and corpus information of sample images; Filter out the first category of field information and the second category of field information in the corpus information; Fusion of the first category field information and the background information to generate synthetic samples of the first category fields; The second-category field information and the background information are integrated to generate synthetic samples of the second-category fields.

3. The method according to claim 1, characterized in that The step (F) comprises: (F1) extracting, from the second training data set, according to a preset second sampling ratio, real samples of the first category field and real samples of the second category field used in a training step in a training process, and training the initial recognition model based on the extracted real samples of the first category field and the real samples of the second category field to generate a target recognition model; (F2) performing a second test on the initial recognition model based on the first test data set to obtain a second recognition accuracy rate of the initial recognition model for the first category of fields and a second recognition accuracy rate of the second category of fields; (F3) determining a second ratio adjustment parameter based on the second recognition accuracy rate of the first category of fields and the second recognition accuracy rate of the second category of fields; (F4) determining whether the second ratio adjustment parameter satisfies a predetermined condition; (F5) If the predetermined condition is met, the preset second sampling ratio is adjusted, and when the number of training times does not reach the third predetermined number of times, the execution returns to step (F1); (F6) If the predetermined condition is not satisfied and the number of training times does not reach the third predetermined number of times, the process returns to step (F2).

4. The method according to claim 3, characterized in that After generating the target recognition model, the method further includes: Determine, based on the first test data set, a third recognition accuracy rate of the target recognition model for the first category of fields and a third recognition accuracy rate of the second category of fields; Determine whether the third recognition accuracy rate of the target recognition model for the first category of fields and the third recognition accuracy rate of the second category of fields reach corresponding accuracy thresholds respectively; If it is reached, the target recognition model is determined to be the target recognition model for executing step (G); If not reached, update the second training data set, and retrain the initial recognition model based on the updated second training data set until the third recognition accuracy of the target recognition model for the first category of fields and the third recognition accuracy of the second category of fields reach the corresponding accuracy thresholds.

5. The method according to claim 1, characterized in that The first category of fields includes Chinese character fields; the second category of fields includes at least one of a number field and an English field.

6. A device for recognizing text in an image, characterized in that: The device comprises: A recognition model building module, used to build a recognition model for recognizing text in an image; A first training set construction module, used to obtain synthetic samples of the first category of fields and synthetic samples of the second category of fields, and construct a first training data set using multiple synthetic samples of the first category of fields and multiple synthetic samples of the second category of fields; A test set construction module, used to obtain a first test data set including a plurality of real samples of the first category of fields and a plurality of real samples of the second category of fields; An initial recognition model determination module is used to iteratively update the recognition model based on the first training data set and the first test data set to obtain an initial recognition model; A second training set construction module, used to construct a second training data set using the real samples of the multiple first-category fields and the real samples of the multiple second-category fields; A target recognition model determination module is used to iteratively update the initial recognition model based on the second training data set and the first test data set to obtain a target recognition model; A detection module, used for determining text in the image to be detected based on the target recognition model and the image to be detected; The initial recognition model determination module is used for: Extracting, from the first training data set, according to a preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process; Training the recognition model for a first predetermined number of times based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate a first-order recognition model; According to a preset first sampling ratio, the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model; Performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields; Determine a first ratio adjustment parameter based on a first recognition accuracy rate of the first category of fields and a first recognition accuracy rate of the second category of fields; determining whether the first ratio adjustment parameter satisfies a predetermined condition; If the predetermined condition is met, when the number of training times does not reach the second predetermined number of times, the preset first sampling ratio is adjusted, and according to the adjusted preset first sampling ratio, synthetic samples of the first category of fields and synthetic samples of the second category of fields used in a training step in a training process are extracted from the first training data set, and the first-order recognition model is trained based on the extracted synthetic samples of the first category of fields and the synthetic samples of the second category of fields to generate an initial recognition model; If the predetermined condition is not met, when the number of training times does not reach a second predetermined number of times, performing a first test on the initial recognition model based on the first test data set to obtain a first recognition accuracy rate of the initial recognition model for the first category of fields and a first recognition accuracy rate of the second category of fields; The initial recognition model determination module is also used for: Obtaining weights of the first category of fields and weights of the second category of fields; A first ratio adjustment parameter is determined based on the first recognition accuracy of the first category of fields, the weight of the first category of fields, the first recognition accuracy of the second category of fields, and the weight of the second category of fields.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to perform the method for recognizing text in an image as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for recognizing text in an image as claimed in any one of claims 1 to 5 is executed.

Images