Multi-modal large model training method for certificate OCR (Optical Character Recognition) task

Abstract

The invention discloses a multi-modal large model training method for a certificate OCR (optical character recognition) task. The method comprises the following steps: performing fine adjustment on a multi-modal large model through a small number of labeled samples; carrying out reinforcement learning training on the multi-modal large model through the remaining labeled samples until the recognition precision of the multi-modal large model on the verification set reaches a set standard, and completing the training of the multi-modal large model; the label of the sample label is structured output of the key field in the certificate image. According to the method, a long text task is disassembled into field-level feedback and optimization, relative advantages are generated in combination with intra-group comparison, so that effective learning in a sparse environment is realized, and the method greatly improves the recognition accuracy of a model to complex certificate contents while ensuring the normalization of an output format, and has a good application prospect. The method has the advantages that the method is simple and easy to implement, good generalization ability and reasoning level are shown, the multi-modal large model with excellent performance for the certificate OCR task can be trained under the condition that the manual labeling workload is reduced, and the method has wide application prospects.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of optical character recognition (OCR), and more particularly, the present application relates to a multimodal large model training method for an identification document OCR task. BACKGROUND

[0002] Extracting text information from identification document images using OCR technology (optical character recognition) has a wide range of applications in the industry. However, traditional OCR methods often require a large amount of manually annotated data for supervised training, and the model is prone to errors when encountering complex formats or long texts, making it difficult to generalize efficiently. With the development of multimodal large model technology and the outstanding performance of large models in OCR tasks, using multimodal large models for end-to-end extraction of image text has become a technical trend.

[0003] In the current training of multimodal large models (MLLMs), GRPO training is mainly focused on tasks with relatively simple answer forms, such as solving mathematical problems, where the answer is usually a single number or short text.

[0004] However, in the context of long text, traditional reinforcement learning faces the problem of sparse rewards: a long string of text output by the model often only receives a sparse feedback signal as a whole. Most existing reward models only give a single score feedback for the entire text output, regardless of the length of the text, which makes the learning signal too thin. In particular, in the visual OCR task of multimodal large models, if only the final output is completely correct, the model will have difficulty determining where the mistake is, resulting in low training efficiency. SUMMARY

[0005] The present application provides a multimodal large model training method for an identification document OCR task, aiming to improve at least one of the above problems.

[0006] The present application is implemented as follows: a multimodal large model training method for an identification document OCR task, the method is as follows:

[0007] (1) Fine-tune the multimodal large model with a small amount of labeled samples;

[0008] (2) Perform reinforcement learning training on the remaining labeled samples to the multimodal large model until the recognition accuracy of the multimodal large model on the validation set reaches the set standard, completing the training of the multimodal large model;

[0009] Wherein, the label of sample annotation is the structured output of the key field in the identification document image.

[0010] Further, the reinforcement learning training process of the multimodal large model is as follows:

[0011] (21) input the certificate image in the sample into the fine-tuned multi-modal large model, and the multi-modal large model outputs multiple candidate outputs, each of which contains a formatted key field;

[0012] (22) structurally analyze each candidate output and calculate the reward score of each candidate output;

[0013] (23) calculate the average of the reward scores of all candidate outputs, and take the difference between the reward score of each candidate output and the average of the reward scores as the advantage value, and take the advantage value as part of the loss value of the multi-modal large model.

[0014] Further, the extracted key fields are stored in a data structure of field name-field value pairs.

[0015] Further, the key fields include numerical fields and text fields.

[0016] Further, the reward score of each candidate output is composed of the reward score of the text field and the reward score of the numerical field.

[0017] Further, the reward score of the text field is obtained as follows:

[0018] Select the text field from the candidate output, and extract the standard text field with the same field name from the sample label; calculate the edit distance between the selected text field in the candidate output and the corresponding standard text field, and determine the reward score of the corresponding text field based on the edit distance.

[0019] Further, when there are multiple text fields in the key fields, the corresponding weight values are assigned based on the importance of the text fields, and the products of the reward scores of all text fields and the weight values are summed to form the reward score of the text field.

[0020] Further, the reward score of the numerical field is obtained as follows:

[0021] Compare the numerical field in the candidate output with the corresponding numerical field in the sample label. If they are completely identical, give a positive reward score, and if they are not completely identical, give a negative reward score.

[0022] Further, after step (2), it further includes:

[0023] (3) input the certificate image to be recognized into the trained multi-modal large model, and the multi-modal large model outputs the key fields in the structured certificate image.

[0024] The application realizes a brand-new multi-modal large model training method, fully utilizes fine-grained reward signals to guide multi-modal large model learning, and has the following beneficial technical effects compared with traditional learning methods:

[0025] (1) The reward design of field-by-field comparison and weighted summation enables the multi-modal large model to obtain rich feedback information in long text output tasks. Even if the output is not completely correct, the multi-modal large model can still receive rewards for partially correct fields, and it is clear which fields need to be improved, thereby effectively alleviating the sparse reward problem in long text scenarios.

[0026] (2) The relative reward signal is provided by comparing multiple outputs of the same input. This method is similar to having the model compete with itself. Attempts that perform better than the average will be reinforced, and attempts that are below average will be weakened. This relative reward evaluation not only improves the stability of the learning signal, but also reduces the dependence on precise absolute reward scales, allowing the training to still be effectively optimized when fine-grained annotations are lacking.

[0027] (3) Through formatted instructions and structured parsing, the multi-modal large model is guided to output each key field in a clear segmented form. This is equivalent to explicitly decomposing the OCR task into subtasks, prompting the model to think and extract independently for each field. This approach improves the multi-modal large model's logical reasoning ability and adaptability to complex layouts, significantly improving the accuracy of field extraction.

[0028] (4) The reward function can set different weights and evaluation standards for different fields. For example, key information (name, ID number) can be given a higher weight to ensure that the model prioritizes these fields for correctness. Secondary fields (such as detailed or omitted addresses) can be given moderate tolerance. Through this weight adjustment mechanism, the model's learning goals are more aligned with business needs.

[0029] (5) In the reinforcement learning phase, a large number of unlabeled real ID images can be used based on a small amount of labeled data to obtain feedback through intra-modal self-comparison, thereby reducing the need for manual annotation. The introduction of automatic fine-grained feedback mechanisms can accelerate the acquisition of high-quality data, enabling the multi-modal large model to efficiently utilize unlabeled data for self-improvement, improve its generalization ability for different types of ID, and reduce labor costs.

[0030] In summary, the training method provided by the application introduces new reward design and strategy optimization, enabling the multi-modal large model to achieve high precision and robust recognition in ID OCR tasks, effectively addressing the problem of sparse rewards in long text reinforcement learning. While reducing human intervention, the multi-modal large model's understanding and reasoning ability for complex documents are significantly improved, demonstrating significant practical application value. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 A multi-modal large model training method flowchart for an identification document OCR task is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0032] The specific embodiments of the present application will be further described below with reference to the accompanying drawings, and by describing the embodiments, to help those skilled in the art have a more complete, accurate and in-depth understanding of the inventive concept and technical solutions of the present application.

[0033] The present application proposes a multi-modal large model GRPO training method for an identification document OCR task. Through the methods of formatted instruction output, partial fine-tuning, structured field parsing, field weighting reward function, and group relative strategy optimization, the recognition accuracy and reasoning ability of the model in various identification document OCR tasks are significantly improved, and the demand for manual annotation data is reduced.

[0034] Figure 1 A multi-modal large model training method flowchart for an identification document OCR task is provided for an embodiment of the present application. The method is as follows:

[0035] (1) Fine-tune the multi-modal large model with a small amount of labeled samples;

[0036] (2) Train the multi-modal large model through reinforcement learning with the remaining labeled samples until the recognition accuracy of the multi-modal large model on the validation set reaches the set standard, and complete the training of the multi-modal large model;

[0037] The label of the sample annotation is the structured output of the key field in the identification document image, which can be called a label. By predefining a template, the key field is filled into the corresponding position of the template to form the label of the identification document image. The structured output of the key field in the label is beneficial to subsequent field parsing and extraction.

[0038] In an embodiment of the present application, the training process of the multi-modal large model is divided into two stages, which are as follows:

[0039] Fine-tuning stage: According to the output format of the key field of the target identification document type, for example, for a driver's license identification document, define the output key field to include the fields of name, ID number, birth date, and address, and identify each field with a clear label; input the identification document image and the labeled structured key field label text into the multi-modal large model, fine-tune the multi-modal large model, and train the multi-modal large model to learn to output the correct field content in the specified format. The fine-tuning stage can use a supervised learning loss to update part of the model parameters, so that the multi-modal large model has basic field recognition and format compliance ability.

[0040] Reinforcement learning training: the reinforcement learning phase trains the main loop based on the group relative strategy optimization mechanism GRPO. At each iteration, the fine-tuned multi-modal large model is input with the sample certificate image from the training data set. The fine-tuned multi-modal large model outputs a key field that follows a predetermined format but may not be fully correct in content. In order to obtain rich exploration, the multi-modal large model is run multiple times for the same input certificate image to obtain several different candidate outputs, which can be achieved by introducing random noise or using temperature sampling, etc. to ensure that the candidate outputs are correct in format while having diversity in field content.

[0041] In the embodiment of the application, the reinforcement learning training process of the multi-modal large model is as follows:

[0042] (21) input the certificate image in the sample into the fine-tuned multi-modal large model, and the multi-modal large model outputs multiple candidate outputs, each of which contains a formatted key field;

[0043] (22) extract the key field in each candidate output and calculate the reward score of each candidate output;

[0044] (23) calculate the average value of the reward scores of all candidate outputs, and the difference between the reward score of each candidate output and the average value of the reward scores is used as the advantage value, which is used as part of the loss value of the multi-modal large model. The multi-modal large model is trained.

[0045] Each candidate output is parsed into a list of field-value pairs for easy comparison. For example, the candidate output text "Name: Zhang San; ID number: 110199001010000; Date of birth: 1990-01-01" is parsed into {"Name": "Zhang San", "ID number": "110199001010000", "Date of birth": "1990-01-01"}, and the reference answer (if labeled) is also represented in the same structure.

[0046] In the embodiment of the application, the extracted key fields are stored in a field name-field value pair data structure, wherein the key fields include numerical fields and text fields, and therefore the reward score of each candidate output is composed of the reward score of the text field and the reward score of the numerical field.

[0047] In the embodiment of the application, the reward score of the text field is obtained as follows:

[0048] The text field is selected from the candidate output, and a standard text field with the same field name is extracted from the sample label. An edit distance between the selected text field of the candidate output and the corresponding standard text field is calculated, and a reward score of the corresponding text field is determined based on the edit distance. The smaller the edit distance, the higher the reward score of the corresponding text field. When there are multiple text fields in the key field, a corresponding weight value is given based on the importance of the text field, and the product of the reward scores of all the text fields and the weight values is summed to form the reward score of the text field.

[0049] In the embodiment of the present application, the reward score of the numerical field is obtained as follows: the numerical field in the candidate output is compared with the corresponding numerical field in the sample label. If they are completely identical, a positive reward score is given. If they are not completely identical, a negative reward score is given.

[0050] After completing the structured analysis of the candidate output, for the text field, the similarity between each text field in the candidate output and the corresponding standard text field in the label is calculated one by one. For example, the Levenshtein edit distance is used to quantify the difference. The smaller the Levenshtein edit distance, the closer the text field of the candidate output to the standard text field in the sample label, and thus a high reward score is given. For numerical or date fields, the numerical field of the candidate output is required to be completely matched with the corresponding standard data field in the label to give a positive reward score to the corresponding numerical field. If they are not completely matched, a negative reward score is given to the corresponding numerical field. For each numerical field, for example, the matching result of the name field is multiplied by a weight of 2.0, and the ordinary field is multiplied by a weight of 1.0 to reflect the importance of the name field. In addition, the reward function also detects the correctness of the output format, such as whether the field is missing or the symbol is complete. If the format completely conforms to the specification, an additional fixed reward can be given. If the format is incorrect, a certain score will be deducted. Finally, the weighted scores of the fields and the format scores are added to obtain the total reward score R of the candidate output.

[0051] After the reward score calculation of all group candidate outputs is completed, a group relative strategy optimization (GRPO) stage is entered. For multiple candidate outputs generated for the same input, the average value bar{R} of the reward scores R of all candidate outputs is calculated; then the advantage value A is calculated for each output, for example, the advantage value A is defined as A = R - bar{R}. The advantage value A of the output with a reward higher than the average value is positive, and the advantage value A of the output with a reward lower than the average value is negative, and these advantage values are used as substitute reward signals and are transmitted to the strategy gradient update module of the PPO algorithm. When the strategy is updated, the strategy probability corresponding to the output with the advantage value A > 0 is amplified, and the strategy probability corresponding to the output with the advantage value A < 0 is reduced, so as to guide the model to adjust the parameters in the direction of generating an output with a higher relative reward. Compared with directly using the absolute reward R, using the advantage value A can effectively normalize the reward scale of different samples and highlight the relative performance difference of the model on the current input, so that the optimization process is more stable and efficient.

[0052] In a specific implementation, a clip loss function of proximal policy optimization (PPO) can be used to limit the amplitude of each parameter update, so as to ensure that the strategy will not change dramatically due to one round of feedback. After each round of update, the above process is repeated for the next training sample. As the training proceeds, the model is continuously strengthened to follow the correct format and accurately identify the content of each field. In the verification process, the method of the present application has achieved a large increase in accuracy on multiple certificate OCR benchmark data sets, and the robustness of the model to field missing and layout changes in complex scenarios has also been significantly enhanced.

[0053] It should be noted that the method of the present application is not limited to driving license OCR, and is also applicable to document OCR tasks with structured fields such as vehicle registration certificate, household booklet, marriage certificate, online car certificate, online driver certificate, etc. The fields and weights in the reward function can be customized according to different certificate types, and the long text comparison method can also be flexibly used. The word-level accuracy and other indicators can be used to construct. The number of candidate outputs in the group comparison can be adjusted as needed, and more candidates are helpful to provide more accurate relative feedback signals under the condition of allowed computing power. The fine-tuning stage can also use existing small sample learning methods (such as LoRA adaptation, Prompt Tuning, etc.) to efficiently adjust the large model, so that it can be more quickly integrated into a specific task.

[0054] In the embodiment of the present application, after step (2), the following steps are further included:

[0055] (3) inputting the certificate image to be recognized into the trained multi-modal large model, and the multi-modal large model outputs the key fields in the structured certificate image.

[0056] The application realizes effective learning in a sparse environment by decomposing a long text task into field-level feedback and optimization, and generating relative advantages by combining intra-group comparison, thereby greatly improving the recognition accuracy of the model on complex certificate content while ensuring the output format specification, and exhibiting good generalization ability and reasoning level, and being able to train a multi-modal large model with excellent performance for a certificate OCR task while reducing the workload of manual labeling, and having broad application prospects.

[0057] The application is described exemplarily, and it is obvious that the specific implementation of the application is not limited by the above manner, as long as various non-essential improvements are made by adopting the method concept and technical scheme of the application, or the concept and technical scheme of the application is directly applied to other occasions without improvement, which are all within the protection scope of the application.

Claims

1. A multimodal large model training method for document OCR tasks, characterized by: The method is specifically as follows: (1) Fine-tune a large multimodal model using a small number of labeled samples; (2) Reinforcement learning training is performed on the multimodal large model using the remaining labeled samples until the recognition accuracy of the multimodal large model on the validation set reaches the set standard, completing the training of the multimodal large model; Among them, the sample annotation labels are the structured outputs of key fields in the document image.

2. The multimodal large model training method for the document OCR task as claimed in claim 1, characterized in that: The reinforcement learning training process of the multimodal large model is as follows: (21) Inputting the ID image in the sample into the fine-tuned multimodal large model, the multimodal large model outputs multiple candidate outputs, each candidate output contains formatted key fields; (22) Perform structural analysis on each candidate output and calculate the reward score for each candidate output; (23) Calculate the average reward score of all candidate outputs, take the difference between the reward score of each candidate output and the average reward score as the advantage value, and take the advantage value as part of the loss value of the multimodal large model.

3. The multimodal large model training method for the document OCR task as claimed in claim 1, characterized in that: The extracted key fields are stored in a data structure of field name-field value pairs.

4. The multimodal large model training method for the document OCR task as claimed in claim 3, characterized in that: Key fields include numeric fields and text fields.

5. The multimodal large model training method for the document OCR task as claimed in claim 4, characterized in that: The reward score of each candidate output consists of the reward score of the text field and the reward score of the numeric field.

6. The multimodal large model training method for the document OCR task as claimed in claim 5, characterized in that: The method for obtaining the reward score of the text field is as follows: Select a text field from the candidate output, and extract the standard text field with the same field name from the sample annotation; calculate the edit distance between the text field selected from the candidate output and the corresponding standard text field, and determine the reward score of the corresponding text field based on the edit distance.

7. The multimodal large model training method for the document OCR task as claimed in claim 6, characterized in that: When there are multiple text fields in the key field, the corresponding weights are assigned based on the importance of the text fields, and the product of the reward scores of all text fields and the weights is summed to form the reward score of the text field.

8. The multimodal large model training method for the document OCR task as claimed in claim 5, characterized in that: The specific method for obtaining the reward score of the numerical field is as follows: Compare the numerical fields in the candidate output with the corresponding numerical fields in the sample annotation. If the two are completely consistent, a positive reward score is given. If the two are not completely consistent, a negative reward score is given.

9. The multimodal large model training method for the document OCR task as claimed in claim 1, characterized in that: After step (2), the method further includes: (3) The document image to be identified is input into the trained multimodal large model, and the multimodal large model structuredly outputs the key fields in the document image.

Images