A visual language model training method and system based on evidence anchoring reward

Abstract

This invention discloses a visual language model training method and system based on evidence-anchored rewards, relating to the fields of artificial intelligence and computer vision. The method includes: constructing a multimodal training sample set, collecting samples carrying question text, images to be analyzed, standard answers, and structured data sources, and extracting truth values ​​of related answers from these samples; inputting the question and image into a visual language model to obtain inference text and predicted answers; extracting evidence fragments referenced by the model from the inference text through numerical, quoted text, and semantic matching; calculating evidence recall and truth value coverage, and combining this with answer correctness to derive evidence-anchored rewards and answer correctness rewards, respectively; calculating a combined reward; and updating model parameters using a policy optimization algorithm to achieve evidence-anchored model training. This invention introduces an evidence-anchored reward mechanism to quantify the verifiability of inference evidence, suppress inference illusions, and improve the model's factual fidelity and logical credibility.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and computer vision technology, and in particular to a method and system for training visual language models based on evidence-anchored rewards. Background Technology

[0002] Visual-Language Models (VLMs), as multimodal artificial intelligence models that integrate visual perception and language understanding, have seen rapid development and large-scale deployment in downstream tasks such as image understanding and visual question answering in recent years. Chart QA and Document QA are two highly valuable sub-tasks of visual question answering. Chart QA requires the model to understand the structure and numerical relationships of statistical charts such as bar charts, line charts, and pie charts and answer questions. Document QA requires the model to parse scanned documents, standardized forms, and other document images, accurately extracting key semantics and effective information from the text and images. Both types of tasks place stringent demands on the VLM's cross-modal alignment and logical reasoning capabilities.

[0003] Currently, the mainstream training paradigms for Virtual Model Learning (VLM) mainly include two technical routes: supervised fine-tuning and reinforcement learning. While supervised fine-tuning can quickly improve the model's fitting accuracy on the target task, it heavily relies on large-scale, high-quality manually labeled data, resulting in high labeling costs and lengthy cycles. Furthermore, the generalization performance of the model across tasks and scenarios after training is limited. In recent years, reinforcement learning algorithms such as Group Relative Policy Optimization (GRPO) have been gradually introduced into the VLM training process. GRPO generates multiple candidate responses for the same input and completes policy iteration updates based on the relative reward signals within the group, demonstrating significant advantages in tapping the model's deep reasoning potential and improving the logic of responses.

[0004] Existing reinforcement learning-based VLM training schemes still suffer from several technical shortcomings. First, most existing reward function designs only consider the correctness of the final predicted answer, without constraining or verifying the visual evidence cited in the model's inference chain. This can easily lead to the model outputting the correct answer based on flawed logic or fictitious reasoning paths, causing serious inference chain illusion problems. Second, the lack of automated evidence matching and verification mechanisms makes it impossible to perform hierarchical comparisons between the evidence fragments claimed in the model's reasoning text and the structured data sources accompanying the images, making it difficult to quantitatively assess the factual fidelity and credibility of the reasoning process. Finally, existing training architectures are mostly customized and optimized for single Chart QA or Document QA tasks, lacking a unified training paradigm compatible with both types of task scenarios, resulting in weak model versatility and scenario transfer capabilities.

[0005] There is an urgent need to build a VLM reinforcement learning training scheme that integrates evidence anchoring constraints, is adaptable to both Chart QA and Document QA scenarios, and balances answer accuracy and reasoning fidelity. Summary of the Invention

[0006] The main objective of this invention is to provide a method for training visual language models based on evidence-anchored rewards.

[0007] Another objective of this invention is to propose a visual language model training system based on evidence-anchored rewards.

[0008] The third objective of this invention is to provide an electronic device.

[0009] The fourth objective of this invention is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above objectives, a first aspect of the present invention proposes a method for training a visual language model based on evidence-anchored rewards, comprising:

[0011] Construct a training sample dataset by collecting training samples containing question text, images to be parsed, standard reference answers, and structured data sources, and extract a set of truth values ​​of evidence related to the reference answers from the structured data sources; The question text and the image to be parsed are input into the visual language model to obtain the model response result containing the reasoning text and the predicted answer; Based on the inference text, a set of evidence fragments claimed by the model is extracted through numerical pattern extraction, quotation mark text extraction, and semantic pattern matching extraction. The evidence fragment set is matched with the evidence truth set at multiple levels. The evidence recall rate and truth coverage rate are calculated based on the matching results. The correctness reward is determined based on the predicted answer and the reference answer. The evidence anchoring reward is determined based on the evidence recall rate and the truth coverage rate. A combined reward is calculated based on the answer correctness reward and the evidence anchoring reward. The parameters of the visual language model are updated using a policy optimization algorithm to achieve the training of the visual language model based on evidence anchoring.

[0012] Optionally, a training sample dataset is constructed, which collects training samples containing question text, images to be parsed, standard reference answers, and structured data sources, and extracts a set of truth values ​​of evidence related to the reference answers from the structured data sources, including: Collect the basic training data required for the visual question answering task, and collect each complete single sample unit containing the question text, the image to be parsed, the standard reference answer, and the supporting structured data source. Organize and summarize them by category to form a large-scale training sample dataset. The collected structured data sources are classified and their attributes are identified to accurately distinguish the underlying data table resources corresponding to chart materials and the optical character recognition text resources corresponding to document materials, so as to realize the classification and archiving management of data sources; Differentiated truth extraction logic is used for different types of training samples to locate the key content of the standard reference answer from the corresponding structured data source, and filter and integrate it to form a set of evidence truths that match the correlation.

[0013] Optionally, the question text and the image to be parsed are input into a visual language model to obtain a model response result containing inference text and predicted answer, including: The training sample dataset is disassembled and separated into independent question text information and image information to be parsed, preserving the semantic integrity of the text and the visual feature integrity of the image, forming dual-modal input materials for the model; According to the preset input interface specifications of the visual language model, the structured question text and the original image to be parsed are modally fused and synchronously input into the visual language model to be trained. The system utilizes the built-in multimodal feature encoding, cross-modal semantic alignment, and logical reasoning modules of the visual language model to jointly analyze, correlate, and logically organize textual semantic information and image visual information. The model autonomously completes the logical reasoning, key information analysis, and answer generation output, ultimately obtaining a standardized model response result that simultaneously carries the logical deduction and reasoning text as well as the predicted answer to the corresponding question.

[0014] Optionally, based on the inference text, a set of evidence fragments claimed by the model is extracted using numerical pattern extraction, quotation mark text extraction, and semantic pattern matching extraction methods, including: The inference text output by the model is preprocessed by full-text sentence segmentation, word segmentation and semantic block segmentation to remove invalid and redundant words and retain the core inference statements that have factual support. Numerical pattern extraction rules are used to perform full-domain traversal matching on the preprocessed inference text, identify numerical expressions in various standardized formats in the text, and accurately extract the corresponding numerical text fragments. The quotation mark text extraction rules are used to locate independent statements in the inference text that are enclosed by target punctuation marks, and the complete semantic text inside the quotation marks is extracted separately as an independent piece of evidence. Configure a trigger word lexicon containing the target semantic connotations of observation, reading, display, and numerical identification. Use semantic pattern matching to traverse reasoning text statements, identify target statements containing preset semantic trigger words, and extract complete semantic fragments associated with the trigger words. The evidence fragments obtained through the three extraction methods are summarized, redundancy is removed, and they are sorted and integrated to form a complete and non-repeating set of model claim evidence fragments.

[0015] Optionally, the set of evidence fragments is matched with the set of truth values ​​at multiple levels. Evidence recall and truth value coverage are calculated based on the matching results. A correctness reward is determined based on the predicted answer and the reference answer, including: The model evidence fragment set and the evidence truth value set are mapped one-to-one to establish a corresponding relationship between the evidence elements to be matched and the truth value elements. Following a hierarchical order from rigorous to broad and from precise to fuzzy, the system progressively conducts four levels of verification: precise matching, numerical tolerance matching, inclusion relationship matching, and fuzzy similarity matching, and then filters and determines the validity of the matching at each level. The number of valid evidence that passed matching at each level, the number of unmatched evidence, and the number of truth values ​​covered were statistically analyzed. Based on the statistical quantitative results, the two core evaluation indicators, evidence recall rate and truth value coverage rate, were calculated in a standardized manner. We differentiate between two types of visual question answering samples: chart-based and document-based. We then adapt different answer comparison and judgment logics to each type, comparing the predicted answer with the standard reference answer from the dimensions of content error and text similarity. Based on the comparison and judgment results under different sample scenarios, the levels are divided and numerical values ​​are assigned according to the preset reward quantification rules, and the corresponding sample's answer correctness reward is quantified.

[0016] Optionally, the evidence anchoring reward is determined based on the evidence recall rate and the truth value coverage rate; a combined reward is calculated based on the answer correctness reward and the evidence anchoring reward; and the parameters of the visual language model are updated using a policy optimization algorithm to achieve evidence anchoring-based visual language model training, including: The set of evidence fragments output by the model is subjected to non-empty validity judgment, and different reward calculation logics are adopted according to whether there is valid evidence content in the set. When the evidence fragment set is in a valid non-empty state, the evidence recall rate, truth coverage rate and preset weight coefficient are combined for weighted fusion calculation to quantify and solve for the corresponding evidence anchoring reward. When the evidence fragment set is in an empty invalid state, a preset negative penalty value is directly assigned as the evidence anchoring reward. The quantitatively determined answer correctness reward and evidence anchoring reward are combined and weighted, and the overall combined reward corresponding to a single training sample is calculated in accordance with the preset combination rules. Using combined rewards as feedback constraints for model optimization, an appropriate policy optimization algorithm is introduced to construct a gradient propagation path in reverse along the direction of reward maximization. The network weights and internal hyperparameters of the visual language model are iteratively adjusted layer by layer based on the gradient backpropagation mechanism. Multiple rounds of iterative updates and convergence optimizations are continuously completed, and finally, a visual language model with strong evidence anchoring reasoning ability is obtained through training convergence.

[0017] To achieve the above objectives, a second aspect of the present invention proposes a visual language model training system based on evidence-anchored rewards, comprising: The sample construction module is used to construct a training sample dataset, collect training samples containing question text, images to be parsed, standard reference answers, and structured data sources, and extract a set of truth values ​​of evidence related to the reference answers from the structured data sources; The model inference module is used to input the question text and the image to be parsed into the visual language model to obtain the model response result containing the inference text and the predicted answer; The evidence extraction module is used to extract a set of evidence fragments claimed by the model based on the reasoning text through numerical pattern extraction, quotation mark text extraction and semantic pattern matching extraction methods. The matching and calculation module is used to perform multi-level matching between the evidence fragment set and the evidence truth value set, calculate the evidence recall rate and truth value coverage rate according to the matching results, and determine the answer correctness reward according to the predicted answer and the reference answer. The optimized training module is used to determine the evidence anchoring reward based on the evidence recall rate and the truth coverage rate, calculate the combined reward based on the answer correctness reward and the evidence anchoring reward, and update the parameters of the visual language model using a policy optimization algorithm to achieve evidence anchoring-based visual language model training.

[0018] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0019] To achieve the above objectives, a third aspect of this application provides an electronic device, including a processor and a memory; wherein the processor runs a program corresponding to the executable program code by reading executable program code stored in the memory, for implementing a visual language model training method based on evidence-anchored rewards as described in the first aspect embodiment.

[0020] To achieve the above objectives, a fourth aspect of this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a visual language model training method based on evidence-anchored rewards as described in the first aspect embodiment.

[0021] The embodiments of the present invention have the following beneficial effects: (1) This scheme introduces an evidence anchoring reward mechanism. In the VLM reinforcement learning training paradigm, it breaks through the limitation of traditional rewards that only consider the correctness of the final answer. Simultaneously, it quantifies and incentivizes the verifiability of the evidence cited in the model reasoning chain. It constrains the model reasoning logic generation process from the training source, effectively suppresses the illusion defect of the reasoning chain, and significantly improves the factual fidelity and logical credibility of the model reasoning process.

[0022] (2) Construct a multi-level progressive evidence matching and verification system, and set up four verification dimensions in order of logical level: precise matching, numerical tolerance matching, inclusion relationship matching and fuzzy similarity matching. It can be compatible with diverse evidence expression forms such as text, numerical values ​​and semantic fragments, realize full coverage retrieval and effective verification of heterogeneous evidence, and greatly improve the evidence matching recall rate and evidence verification robustness.

[0023] (3) A general unified training framework adapted to multi-task scenarios is constructed, which can simultaneously support two typical tasks: Chart QA (chart visual question answering) and Document QA (document visual question answering), and has good task generalization and scenario adaptation capabilities. Experimental tests show that on the standard chart visual question answering dataset, the relaxation accuracy of the model after training is significantly improved from the baseline level; on the document visual question answering dataset, the average normalized Levenstein similarity of the model is significantly increased compared with the baseline, and the overall reasoning and question answering performance of the model under both tasks has achieved a substantial leap.

[0024] (4) Create a pseudo-label closed-loop training mechanism driven by evidence audit. By conducting cross-audit and validity verification of the model's inference output, high-credibility samples are selected and standardized evidence pseudo-labels are generated. The pseudo-labels are then injected into the subsequent training process as new true value samples to form a self-supervised closed-loop optimization link. This continuously strengthens the model's evidence anchoring ability and cross-modal reasoning representation ability, further consolidating and improving the overall training efficiency of the model. Attached Figure Description

[0025] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a visual language model training method based on evidence-anchored rewards, provided for an embodiment of the present invention; Figure 2 This is a flowchart illustrating the overall architecture of the evidence extraction module provided in an embodiment of the present invention. Figure 3 A flowchart of a multi-level evidence matching and verification process provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of multi-component combined reward weighted calculation provided in an embodiment of the present invention; Figure 5A flowchart of the closed-loop iterative training process for verifying pseudo-labels provided in an embodiment of the present invention; Figure 6 This is a structural diagram of a visual language model training system based on evidence-anchored rewards, provided in an embodiment of the present invention. Detailed Implementation

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] The following describes, with reference to the accompanying drawings, a visual language model training method and system based on evidence-anchored rewards according to an embodiment of the present invention.

[0029] Example 1 This invention provides a method for training a visual language model based on evidence-anchored rewards. Figure 1 This is a flowchart illustrating a visual language model training method based on evidence-anchored rewards, provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: Step S1: Construct a training sample dataset by collecting training samples containing question text, image to be parsed, standard reference answer, and structured data source, and extract a set of truth values ​​of evidence related to the reference answer from the structured data source.

[0030] The first step in constructing the training sample dataset is to collect basic training data. In this embodiment, the collected basic data must fully cover the four core elements required for visual question answering tasks: question text, image to be parsed, standard reference answer, and structured data source. These four elements must correspond one-to-one and be closely related; none can be missing. Specifically, staff will collect individual training sample units one by one. Each sample unit must fully contain the above four elements to avoid missing elements or disordered relationships. Then, all sample units are organized and summarized according to the type of visual question answering task (such as chart-based or document-based tasks), ultimately forming a large-scale training sample dataset, providing a sufficient data foundation for batch training of the model.

[0031] After completing the basic data collection and aggregation, this embodiment of the application will further classify and identify the attributes of the aggregated structured data sources. Since the structured data sources corresponding to different types of visual question answering tasks (charts, documents) are fundamentally different, failure to classify and manage them will lead to deviations in subsequent evidence truth extraction, affecting the model training effect. Therefore, this embodiment of the application will accurately distinguish between the underlying data table resources (such as CSV format data tables) corresponding to chart-type materials and the optical character recognition (OCR) text resources corresponding to document-type materials, and will archive and manage the two types of data sources separately, clarifying the storage path, data format, and attribute characteristics of each type of data source, laying the foundation for subsequent differentiated evidence truth extraction.

[0032] For different types of training samples, this application employs differentiated evidence truth value extraction logic to ensure that the extracted evidence truth values ​​can accurately correlate with the standard reference answer, truly playing the role of evidence anchoring. Specifically, for chart-based visual question-answering samples, the structured data source is the underlying data table corresponding to the chart. The method for extracting the evidence truth value set is as follows: First, accurately search for target cells matching the standard reference answer in the underlying data table, determine the specific location of the cell and its corresponding value or text content. Then, extract the header information corresponding to the target cell (used to clarify the meaning and category of the data), as well as the content of related cells in the same row (used to supplement the complete evidence context). These are then integrated to form the evidence truth value set corresponding to the chart-based sample, ensuring that the evidence truth values ​​can fully support the rationality of the reference answer.

[0033] For document-based visual question-answering samples, the structured data source is the OCR text obtained after optical character recognition processing of the document image. The method for extracting the evidence truth set is as follows: First, locate the core word that matches the standard reference answer in the word sequence of the OCR text. This core word is the key support point of the reference answer. Then, with the core word as the center, extract the adjacent words within a preset window range before and after it. In this embodiment, the preset window range is set to 3 adjacent words before and after. By extracting the core word and its contextual adjacent words, a complete semantic fragment is formed as the evidence truth set corresponding to the document-based sample, ensuring that the evidence truth set can accurately reflect the textual context corresponding to the reference answer and provide a reliable comparison basis for subsequent evidence matching verification.

[0034] Throughout the implementation of step S1, this embodiment of the application will strictly control the integrity, relevance, and standardization of the data. From data collection and classification to evidence truth extraction, each step will be strictly verified to avoid affecting the accuracy and effect of subsequent model training due to data errors, classification confusion, or truth extraction deviations. This provides a solid data guarantee for the entire visual language model training process based on evidence anchoring rewards.

[0035] Step S2: Input the question text and the image to be parsed into the visual language model to obtain the model response result containing the reasoning text and the predicted answer.

[0036] Next, step S2, the reasoning generation step, is executed. This step follows the data preparation step mentioned above. Based on the completed training sample dataset, the matching question text and the image to be parsed are fed into the visual language model for multimodal inference. Finally, the complete model response result, which includes both logical reasoning text and the predicted answer to the question, is output.

[0037] In the actual inference generation process, the independent question text information and image information to be parsed are first extracted from the training sample dataset. During the extraction process, the original semantic logic of the text statements is strictly protected from being destroyed, while the visual contours, numerical labels, text content and local feature details of the image itself are fully preserved without cropping, compression or feature loss processing. This forms a bimodal input material that is adapted to the model input format, ensuring the basic effectiveness of subsequent cross-modal parsing.

[0038] After preparing the bimodal input materials, the input interface specifications of the visual language model are strictly followed. The question text is grammatically standardized and structured, while the original pixels and format of the image to be parsed are preserved. The standardized text information and image information are fused and encapsulated across modalities and then sent into the visual language model to be trained for processing according to the model receiving standards.

[0039] In this embodiment, Qwen2.5-VL-7B-Instruct is selected as the basic visual language model for the entire training process. The model's generation logic is constrained and guided by pre-set customized system prompts. The model is required to strictly follow the fixed generation paradigm of first observing the image and associating evidence information, then conducting multi-layer logical reasoning analysis, and finally summarizing and outputting the corresponding question answer. This standardizes the model's reasoning behavior from the perspective of generation logic and strengthens the reasoning habit of evidence-dependent reasoning.

[0040] In the preferred embodiment of this application, the model response adopts a standardized structured output format. The overall response content is divided into two major modules: the reasoning part and the answer part. Independent reasoning tag pairs and answer tag pairs are configured to define the content. The tag pairs are used to achieve accurate segmentation of the reasoning text and the predicted answer, which facilitates the automatic splitting and extraction of reasoning content and answer content in subsequent stages, and reduces the processing difficulty of subsequent evidence extraction and reward calculation.

[0041] After receiving the fused multimodal input data, the model sequentially calls its built-in multimodal feature encoding module, cross-modal semantic alignment module, and logical reasoning module. First, it independently encodes the semantic information of the question text and the visual features of the image to be analyzed. Then, it completes the cross-modal association alignment between the text semantics and image features, and further conducts multi-layered association inference and logical relationship analysis. Relying on the model's pre-trained knowledge and multimodal understanding capabilities, it autonomously completes key information screening, evidence content analysis, reasoning link analysis, and final answer generation. The final output is a standardized model response result with a well-formatted and logically complete output, carrying a complete logical deduction and reasoning text and the corresponding predicted answer to the question.

[0042] To enrich the candidate space of training samples and improve the training effect of strategy optimization, in each training iteration, for the combined input of the same question text and the image to be parsed, the model will generate multiple sets of candidate response contents with different expression logics in batches. In this embodiment, the sampling temperature parameter is fixed at 0.7. While ensuring the diversity of reasoning content, it avoids the problem of logic divergence and reasoning content deviating from the real information of the image due to excessive sampling temperature, thus taking into account both generation diversity and reasoning rigor.

[0043] Step S3: Based on the reasoning text, extract the set of evidence fragments claimed by the model through numerical pattern extraction, quotation mark text extraction, and semantic pattern matching extraction.

[0044] Before formally extracting evidence fragments, the original reasoning text output by the model needs to undergo preprocessing. In this embodiment, the entire reasoning text is segmented into sentences, precisely segmented into words, and semantically segmented. Redundant modifiers and invalid function words that have no substantial semantic meaning and only serve to connect the sentences are removed. Core reasoning sentences that have factual basis and can support the logic of answer deduction are selected and retained. By preprocessing, the text volume is reduced, invalid interference content is removed, the computational burden of subsequent multi-modal evidence extraction is reduced, and the accuracy of evidence fragment capture is improved.

[0045] The overall logical framework for evidence extraction in this stage can be found in [reference needed]. Figure 2 The functional units in the diagram cooperate and work in parallel to complete the entire extraction process from the original reasoning text to a standardized set of evidence fragments. In the specific implementation process, the preprocessed reasoning text 201 serves as the unified input source for the entire module and is fed in parallel into three independently working extraction units: the numerical pattern extraction unit 202, the quotation mark text extraction unit 203, and the semantic pattern extraction unit 204. The three types of units do not interfere with each other and execute their respective evidence extraction tasks synchronously.

[0046] For the numerical pattern extraction unit 202, its operating logic is based on preset numerical pattern extraction rules. It performs sentence-by-sentence full-domain traversal and rule matching on the preprocessed reasoning text, intelligently identifying numerical expressions in various standard formats within the text. In the actual matching process, regular expressions are used for rule matching, which can accurately locate and extract key content such as integers, decimals, percentages, large numbers separated by commas with thousands separators, and monetary figures with various currency symbols from the reasoning text. All successfully matched numerical text fragments are extracted separately as an important type of factual evidence.

[0047] For the quotation mark text extraction unit 203, it mainly relies on preset quotation mark text extraction rules to accurately locate independent statements and proper nouns completely enclosed by target punctuation marks such as single or double quotation marks in the reasoning text. The text content with complete and independent semantics within the quotation mark boundaries is extracted separately as dedicated evidence fragments. This type of quotation-marked text is mostly directly quoted from image annotations, chart labels, and key original text in documents, possessing extremely high factual credibility and forming an indispensable component of the evidence fragments.

[0048] The semantic pattern extraction unit 204 is pre-configured with a dedicated semantic trigger word lexicon, which includes keywords with target semantic connotations such as observation, reading, display, and numerical identification. This unit uses semantic pattern matching as its core to traverse all sentences in the reasoning text, screening and identifying target sentences containing preset semantic trigger words. Once a corresponding trigger word is matched, the complete semantic description fragment following the trigger word is immediately extracted as evidence. The preset semantic trigger words cover commonly used expressions such as observation, reading, display as, and value, accurately capturing the subjective reasoning basis obtained by the model based on image observations and enriching the semantic dimension of the evidence fragment.

[0049] After the three extraction units independently capture various evidence fragments, all results are uniformly summarized and sent to the merging and deduplication unit 205. This unit first centrally collects and integrates evidence fragments from multiple sources, and then performs processing such as removing redundant content, merging duplicate fragments, merging semantically similar content, and regularizing and sorting the overall content. This completely eliminates duplicate and redundant invalid evidence items, sorts out the content arrangement logic, and finally integrates and generates a complete set of model claim evidence fragments with clear items and no duplicates or redundancies. This unit finally outputs a standardized set of evidence fragments 206, completing the entire evidence extraction process, which can be directly used by the next stage of the evidence matching and verification process.

[0050] Step S4: Perform multi-level matching between the evidence fragment set and the evidence truth set, calculate the evidence recall rate and truth coverage rate according to the matching results, and determine the answer correctness reward based on the predicted answer and the reference answer.

[0051] Before formally conducting matching verification, a line-by-line, element-by-element mapping relationship is established between the set of evidence fragments output by the model and the preset set of evidence truth values. This achieves a one-to-one correspondence between evidence fragments and truth value entries, laying the data foundation for subsequent multi-level progressive matching. The entire matching verification implementation process can be referenced. Figure 3 The flowchart shown above illustrates a multi-level evidence matching process. The process uses two initial input data sources: evidence fragment set input 301 and evidence truth value set input 302. Following a logical order from precise to lenient and from strict to fuzzy, the process flows through each matching level to complete a progressive verification and judgment.

[0052] First, the system enters the precise matching layer 303 to perform the first-level precise matching. During this process, the evidence fragment text and the evidence truth text are standardized in a unified manner to eliminate irrelevant interference factors such as extra spaces, differences in capitalization, and format symbols. After standardization, the system directly performs a full text comparison. If the two texts are completely identical, the system directly determines that the match is successful and there is no need to proceed to the subsequent lenient level verification.

[0053] After completing the precise matching and filtering, the second layer of numerical tolerance matching is then performed. For evidence content with numerical attributes, irrelevant identifiers such as currency symbols, percentage signs, and thousandths commas in the evidence fragments and truth texts are first uniformly removed. Then, the cleaned character content is uniformly converted into standard floating-point numerical format for comparison. In this embodiment, a preset matching threshold of five percent is set, and the degree of numerical similarity is quantified by a relative error formula. The relative error calculation formula is as follows:

[0054] in, This represents the relative error of the numerical value. The floating-point value represents the converted evidence fragment. The floating-point value corresponding to the true value of the evidence is used to determine that the two numerical pieces of evidence match, provided that the relative error of the calculation is less than 5%.

[0055] Following numerical tolerance matching, the third layer of inclusion relationship matching is carried out. This layer is for non-pure numerical text evidence. It determines the subordinate relationship between the normalized evidence fragment and the evidence truth value. As long as the evidence fragment is completely contained within a certain evidence truth value text, or the content of the evidence truth value covers the core semantics of the evidence fragment, it can be determined that the inclusion matching condition is met. This further relaxes the matching constraints and adapts to the flexible differences in natural language expression.

[0056] Finally, the fourth layer of fuzzy similarity matching is performed in the fuzzy similarity matching layer 304. For text entries that failed to match in the first three layers, the character-level edit distance ratio is used for fuzzy semantic similarity quantification. In this embodiment, the preset fuzzy threshold is set to 80, and the edit distance ratio is calculated using the following formula:

[0057] In the formula, For the standardized evidence fragment text, For the normalized truth text of evidence, The length of the text is calculated, and the edit distance represents the difference in the number of characters added, deleted, or modified between the two text segments. When the calculated edit distance ratio is greater than or equal to 80, the two segments are considered to be semantically similar, and the fuzzy match is considered successful.

[0058] After completing the four-level progressive matching verification, two key quantitative indicators were calculated based on the overall matching statistics: evidence recall (305) and truth value coverage (306). Evidence recall represents the proportion of valid truth values ​​among the evidence claimed by the model, and its calculation formula is as follows: In the formula, r is the evidence recall rate. The number of successfully matched evidence fragments, This represents the total number of evidence fragments extracted by the model. The truth value coverage rate characterizes the degree to which the model evidence effectively covers the truth values ​​of the standard evidence; the calculation formula is... In the formula, c is the truth coverage. The number of truth values ​​of evidence that were successfully matched. The total number of entries in the preset set of truth values ​​for evidence.

[0059] After obtaining two quantitative indicators at the evidence level, we further distinguish between two major application scenarios: chart-based visual question answering samples and document-based visual question answering samples. We adopt differentiated answer comparison rules and compare the model's predicted answer with the standard reference answer from the dimensions of numerical error and text semantic similarity. Based on the comparison results, we complete the level division and numerical assignment according to the preset reward rules, and finally quantify the answer correctness reward corresponding to a single sample.

[0060] For visual question-answering scenarios involving charts, a relaxed precision judgment method is used to assign reward values. If the relative error between the predicted answer and the reference answer is less than 5%, the answer is directly judged as correct and a reward value of 1 is assigned. If the numerical error exceeds the threshold, the strict precision text matching mode is switched. If the text is completely consistent, the reward value is 1, and if the content is inconsistent, the reward value is set to 0.

[0061] For document-based visual question answering scenarios, instead of using a discrete reward format of either 0 or 1, the average normalized Levenstein similarity is introduced as a continuous reward value, calculated using the following formula:

[0062] in, The predicted answer text output by the model. Let K be the text of the k-th standard reference answer, and K be the total number of reference answers corresponding to a single sample. This formula can finely characterize the semantic similarity between the predicted answer and the reference answer, generating a continuous and smooth reward value, which is more in line with the text matching characteristics of document-based question answering.

[0063] Step S5: Determine the evidence anchoring reward based on the evidence recall rate and the truth coverage rate, calculate the combined reward based on the answer correctness reward and the evidence anchoring reward, and update the parameters of the visual language model using a policy optimization algorithm to achieve evidence anchoring-based visual language model training.

[0064] Before the formal reward calculation begins, the evidence fragment set output from the evidence extraction stage is first assessed for its validity (not empty). Based on whether the set contains valid evidence, two different reward calculation logics are executed. If the evidence fragment set is valid and not empty, a weighted fusion calculation is performed using evidence recall, truth coverage, and preset weight coefficients to quantitatively calculate the corresponding evidence anchoring reward. If the evidence fragment set is empty and contains no valid evidence, a preset negative penalty value is directly assigned as the evidence anchoring reward. This penalty mechanism constrains the model to develop the habit of actively citing evidence for reasoning.

[0065] In this embodiment of the application, evidence anchoring reward A piecewise mathematical model is used for the solution, and the standard expression is as follows:

[0066] in, For evidence recall rate, For true value coverage, As the coverage weighting coefficient, in this embodiment, The value is set to 0.5; when the model does not output any evidence fragments, A fixed negative penalty value of -0.2 is used to force the model to actively discover and output relevant evidence fragments during the reasoning process.

[0067] The overall logic for generating combined rewards in this step can be found in [reference]. Figure 4 The diagram shown illustrates the combined reward calculation. The entire calculation architecture uses three independent inputs: answer correctness reward 401, evidence anchoring reward 402, and optional format compliance reward 403. All reward components are uniformly fed into the weighted summation unit 404 to complete the fusion calculation and output the final usable combined reward value.

[0068] In a simplified implementation that does not introduce additional format constraints, the combined reward is solely based on the correctness of the answer. Evidence-anchored rewards The weighted fusion method yields the result; in the preferred implementation method that also considers output format standardization, a format compliance reward is incorporated. The formula for calculating the combined reward standard is as follows:

[0069] In the formula, Rewards are given for correct answers. Representing evidence-based rewards, Rewards for compliance with formatting requirements; These are the weighting coefficients for the corresponding reward components, which are uniformly set in this embodiment. .

[0070] Among them, format compliance rewards The model assigns scores based on the completeness of the labels in its response. The specific rules are as follows: 1.0 points are assigned when both the reasoning label pair and the answer label pair are complete and compliant; 0.3 points are assigned when only the reasoning label pair is included; 0.5 points are assigned when only the answer label pair is included; and 0 points are assigned when both types of label pairs are missing. The scores are used to quantify and constrain the model to generate output content in a standardized structured format.

[0071] After obtaining the combined reward, this reward value is used as a feedback constraint signal for model policy optimization. Based on a preset policy optimization algorithm, a gradient propagation and parameter update path is constructed backward along the optimization direction that maximizes the reward. This embodiment uses the GRPO algorithm to complete the iterative policy update of the visual language model. This algorithm generates multiple candidate responses for the same set of image and text inputs, ranks them within the group according to the combined reward size, and uses high-reward responses as positive samples and low-reward responses as negative samples to participate in policy gradient updates. Simultaneously, a KL divergence penalty term is introduced for constraint control, with the penalty coefficient set... This effectively avoids excessive deviation from the original benchmark model during model strategy iteration, ensuring training stability and knowledge retention.

[0072] To reduce the computational cost of full parameter fine-tuning and improve training efficiency, this embodiment employs LoRA low-rank adaptation for efficient parameter fine-tuning. The LoRA rank is set to 64, and the scaling factor is... The target module was set to 128, and fine-tuning was performed to cover the query, key, value, and output projection layers within the attention mechanism, as well as the gated projection, up projection, and down projection layers of the feedforward network. The AdamW optimizer was used for parameter updates during training, with the initial learning rate set to [value missing]. The gradient accumulation step count was configured to 8, and the overall training cycle was set to 2 iterations. Relying on the gradient backpropagation mechanism, the network weight parameters and internal hyperparameters of the visual language model were adjusted iteratively layer by layer. After multiple rounds of iterative updates and convergence optimization, a high-performance visual language model with strong evidence anchoring reasoning ability, high answer accuracy, and standardized output format was finally trained.

[0073] In addition, this application embodiment also adds a verification pseudo-label closed-loop training process, the architecture of which can be referred to Figure 5 The flowchart shown is a closed-loop training process for verifying pseudo-labels. When the process runs, it first obtains the inference output of the trained model, which corresponds to the inference output 501 of the trained model in the flowchart. Then it is sent to the evidence audit and verification unit 502, which cross-verifies the evidence fragments extracted from the inference text with the original structured data source to determine the authenticity and validity of the evidence content.

[0074] After the evidence audit is completed, the sample screening is carried out in the strict correctness screening unit 503. The screening criteria are set separately for two types of question-and-answer scenarios. For chart-based visual question-and-answer samples, the model's predicted answer is required to achieve an exact character match with the standard reference answer. For document-based visual question-and-answer samples, the average normalized Levinstein similarity index (ANLS) is required to reach 1.0. Only samples that meet both the evidence verification and strict correctness criteria can proceed to the next stage.

[0075] For qualified samples that pass the double screening, a dedicated pseudo-label file is generated by the pseudo-label generation unit 504, which solidifies and records the evidence fragments that have been confirmed as valid through cross-validation as the pseudo-evidence truth value label for that sample. In the subsequent round of model training, the pseudo-label matching and injection unit 505, equipped with a multi-level key matching mechanism, completes the accurate alignment and association between pseudo-labels and training samples. The multi-level matching keys used include dataset identifiers, data split identifiers, question text, image identifiers, and answer text. Through multi-dimensional key-value binding, it is ensured that each pseudo-label record uniquely matches only one training sample, eliminating the problems of duplicate pseudo-label injection and disordered sample association.

[0076] After the pseudo-labels are accurately injected, the updated and expanded training sample set is sent to the next training unit 506 to continue a new round of GRPO strategy optimization training. Through the pseudo-label closed-loop self-iterative mechanism, high-quality evidence ground truth samples are continuously expanded, and the evidence extraction capability and multimodal reasoning accuracy of the visual language model are continuously improved through iteration, so as to achieve long-term iterative optimization of model performance.

[0077] In the application of one embodiment of the present invention, the implementation process is as follows: In practical application scenarios, the feasibility, rationality and practical application effect of the technical solution of this application are verified from multiple dimensions through chart visual question answering, document visual question answering, evidence weight ablation experiments and pseudo-label closed-loop training, combined with multiple sets of experimental embodiments. The aforementioned model parameters, reward calculation formula and matching judgment rules are used uniformly throughout the process, and all quantitative indicators and experimental parameters are clearly defined.

[0078] In this application, Qwen2.5-VL-7B-Instruct was used as the basic visual language model for all experiments. The GRPO strategy optimization algorithm was adopted during the training process, and the KL divergence penalty coefficient was introduced. The LoRA low-rank adaptation fine-tuning method is adopted, with the rank set to 64 and the scaling factor... The optimizer used is AdamW, and the learning rate is set to [value missing]. The gradient accumulation step count is 8, and the training iteration cycle is set to 2 rounds; the combined reward is uniformly calculated using the following standard formula:

[0079] in, Rewards for correct answers, Anchoring rewards to evidence For format compliance rewards, a fixed weighting coefficient is set. By adjusting the weight of evidence-anchored rewards Conduct controlled experiments; use a segmented calculation method for evidence-anchored rewards:

[0080] Among them, the coverage weighting coefficient The value is fixed at 0.5. For evidence recall rate, This represents the truth value coverage. For document-based visual question answering, the Average Normalized Levenstein Similarity (ANLS) is uniformly used as the evaluation metric, calculated using the following formula:

[0081] First, we conducted experiments to validate the visual question-answering scenario using charts. The experiments employed the ChartQA standard dataset, which was divided into three sets: a training set containing 1000 manually labeled hard samples, a validation set containing 1920 samples, and a test set containing 2500 samples. Each sample was accompanied by a chart image, a question / answer, a standard reference answer, and the underlying CSV data table corresponding to the chart. This fully complies with the truth value extraction rules for chart-type samples in step S1 of this application, allowing for matching target cells from the underlying data table and constructing a truth value set by associating the table header and corresponding cell content. The original Qwen2.5-VL-7B-Instruct model, which was not trained using the reinforcement learning process of this application, was used as the baseline model. The baseline model achieved a relaxation accuracy of 17.88% and a precise accuracy of 7.84% on the ChartQA test set. This indicates that the basic model can only answer questions based on pre-trained knowledge and lacks the ability to extract effective evidence from charts and underlying data tables to support reasoning. By disabling evidence-anchored rewards and training the model solely based on correct answer rewards, the model's relaxation accuracy improved to 60.72% and precise accuracy to 35.68% on the test set after training, resulting in a significant improvement in answer prediction accuracy. Based on this, optimal weights were then set. By introducing evidence-anchored rewards into the calculation of combined rewards, the model significantly improves the evidence fidelity index corresponding to the reasoning chain while maintaining the original answer accuracy rate. This proves that the evidence-anchored rewards added in this application can constrain the model to no longer simply fit the answer results, but actively rely on charts, images and underlying data tables to extract real and valid evidence, construct a traceable reasoning logic chain, and effectively suppress the defect of the model making unfounded guesses.

[0082] Simultaneously, document-based visual question-answering scenarios were tested and validated using the DocVQA dataset. The training set contained 1000 samples, and the validation set contained 5349 samples. Each sample included a document image, a question-and-answer section, and a list of multiple reference answers. The dataset was adapted to the document-type sample processing logic of this application. Reference answer keywords were located and matched based on the document OCR text word sequence, and the evidence truth set was constructed by extracting three adjacent words before and after each keyword. The original Qwen2.5-VL-7B-Instruct model was used as the baseline model, with the ANLS score as the core evaluation metric. The baseline model achieved an ANLS score of only 0.2476 on the validation set, indicating weak document image-text understanding and answer matching capabilities. The initial setup was similarly configured... Training using only correct answer rewards resulted in an ANLS score of 0.7491 on the validation set after training, significantly enhancing the model's text matching and cross-modal understanding capabilities. Further adjustments were made... With the addition of evidence anchoring rewards, the ANLS metric further improved to 0.7598, while the effective accuracy rate of ANLS greater than or equal to 0.5 reached 79.47%, which fully demonstrates that evidence anchoring rewards can also further optimize the model's inference quality and improve the semantic matching degree between the answer and the reference text in document visual question answering scenarios.

[0083] To verify the rationality of the combined reward weighting configuration, this application anchors the reward weighting coefficients to the evidence. Set up multiple groups of ablation control experiments, and configure them sequentially. Training was conducted using four sets of parameters respectively. Experimental results show that when The model focuses only on answer fitting, ignoring evidence extraction and the normativity of the reasoning process; [Settings] At that time, the weight of evidence anchoring reward was relatively low. Although it could slightly improve the standardization of reasoning evidence, its effect on constraining and guiding the model's evidence extraction behavior was limited, and the improvement in evidence fidelity was not significant. When the value is 0.3, the model achieves the optimal balance between answer prediction accuracy and the fidelity of inference evidence, resulting in the best overall performance. Setting the weight too high... At times, the model may overemphasize the extraction and matching of evidence fragments, slightly crowding out the training weights for answer optimization, resulting in a slight decrease in answer accuracy. This experimental conclusion represents a non-obvious technical effect, fully validating the scientific validity and adaptability of the proposed combined reward multi-weight collaborative design scheme, and also defining the limitations of this application. The optimal parameter configuration provides experimental support.

[0084] Building upon this foundation, this application further conducted a closed-loop training experiment to verify pseudo-labels. Using the ChartQA visual question-answering validation set, a comprehensive evidence audit was performed on the model inference outputs of 1920 validation set samples. Samples were filtered according to strict correctness selection rules, with chart-type samples requiring an exact match between the predicted and reference answers. Ultimately, 688 qualified pseudo-label candidate samples were selected from all samples. After evidence cross-validation and label generation, 554 samples successfully generated pseudo-labels carrying valid verification evidence, achieving an overall pseudo-label verification coverage rate of 80.52%, and an evidence verification coverage rate of 63.81% for non-answer dimensions.

[0085] In subsequent training, the multi-level key matching mechanism of this application is adopted. Based on dataset identifiers, data splitting identifiers, question texts, image identifiers, and answer texts, unique bindings are performed, and the generated pseudo-labels are accurately injected into the training sample set, eliminating duplicate matching and erroneous associations. After pseudo-label injection, the proportion of samples in the training set with high-quality human-annotated and automatically verified evidence annotations significantly increases. Through the next round of GRPO closed-loop iterative training, the high-quality evidence ground truth sample library can be continuously expanded, constantly strengthening the model's evidence extraction, multi-level matching, and inference link construction capabilities, achieving self-iterative optimization and long-term improvement of model performance.

[0086] Example 2 This invention provides a visual language model training system based on evidence-anchored rewards. Figure 6 This is a schematic diagram illustrating the structure of a visual language model training system based on evidence-anchored rewards, as provided in an embodiment of the present invention. Figure 6 As shown, the system includes: The sample construction module 100 is used to construct a training sample dataset, collect training samples containing question text, image to be parsed, standard reference answer and structured data source, and extract a set of truth values ​​of evidence related to the reference answer from the structured data source; The model inference module 200 is used to input the question text and the image to be parsed into the visual language model to obtain the model response result containing the inference text and the predicted answer; The evidence extraction module 300 is used to extract a set of evidence fragments claimed by the model based on the reasoning text through numerical pattern extraction, quotation mark text extraction and semantic pattern matching extraction methods. The matching and calculation module is used to perform multi-level matching between the evidence fragment set and the evidence truth value set, calculate the evidence recall rate and truth value coverage rate according to the matching results, and determine the answer correctness reward according to the predicted answer and the reference answer. The optimized training module 400 is used to determine the evidence anchoring reward based on the evidence recall rate and the truth coverage rate, calculate the combined reward based on the answer correctness reward and the evidence anchoring reward, and update the parameters of the visual language model using a policy optimization algorithm to achieve evidence anchoring-based visual language model training.

[0087] Regarding the system in the above embodiments, the specific ways in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0088] Example 3 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0089] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0090] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0091] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0092] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for training a visual language model based on evidence-anchored rewards, characterized in that, include: Construct a training sample dataset by collecting training samples containing question text, images to be parsed, standard reference answers, and structured data sources, and extract a set of truth values ​​of evidence related to the reference answers from the structured data sources; The question text and the image to be parsed are input into the visual language model to obtain the model response result containing the reasoning text and the predicted answer; Based on the reasoning text, a set of evidence fragments claimed by the model is extracted using numerical pattern extraction, quotation mark text extraction, and semantic pattern matching extraction methods. The evidence fragment set is matched with the evidence truth set at multiple levels. The evidence recall rate and truth coverage rate are calculated based on the matching results. The correctness reward is determined based on the predicted answer and the reference answer. The evidence anchoring reward is determined based on the evidence recall rate and the truth coverage rate. A combined reward is calculated based on the answer correctness reward and the evidence anchoring reward. The parameters of the visual language model are updated using a policy optimization algorithm to achieve the training of the visual language model based on evidence anchoring.

2. The method according to claim 1, characterized in that, Construct a training sample dataset by collecting training samples containing question text, images to be parsed, standard reference answers, and structured data sources. Extract a set of truth values ​​related to the reference answers from the structured data sources, including: Collect the basic training data required for the visual question answering task, and collect each complete single sample unit containing the question text, the image to be parsed, the standard reference answer, and the supporting structured data source. Organize and summarize them by category to form a large-scale training sample dataset. The collected structured data sources are classified and their attributes are identified to accurately distinguish the underlying data table resources corresponding to chart materials and the optical character recognition text resources corresponding to document materials, so as to realize the classification and archiving management of data sources; Differentiated truth extraction logic is used for different types of training samples to locate the key content of the standard reference answer from the corresponding structured data source, and filter and integrate it to form a set of evidence truths that match the correlation.

3. The method according to claim 2, characterized in that, The question text and the image to be parsed are input into a visual language model to obtain a model response result containing inference text and predicted answer, including: The training sample dataset is disassembled and separated into independent question text information and image information to be parsed, preserving the semantic integrity of the text and the visual feature integrity of the image, forming dual-modal input materials for the model; According to the preset input interface specifications of the visual language model, the structured question text and the original image to be parsed are modally fused and synchronously input into the visual language model to be trained. The system utilizes the built-in multimodal feature encoding, cross-modal semantic alignment, and logical reasoning modules of the visual language model to jointly analyze, correlate, and logically organize textual semantic information and image visual information. The model autonomously completes the logical reasoning, key information analysis, and answer generation output, ultimately obtaining a standardized model response result that simultaneously carries the logical deduction and reasoning text as well as the predicted answer to the corresponding question.

4. The method according to claim 3, characterized in that, Based on the inference text, a set of evidence fragments claimed by the model is extracted using numerical pattern extraction, quotation mark text extraction, and semantic pattern matching extraction methods, including: The inference text output by the model is preprocessed by full-text sentence segmentation, word segmentation and semantic block segmentation to remove invalid and redundant words and retain the core inference statements that have factual support. Numerical pattern extraction rules are used to perform full-domain traversal matching on the preprocessed inference text, identify numerical expressions in various standardized formats in the text, and accurately extract the corresponding numerical text fragments. The quotation mark text extraction rules are used to locate independent statements in the inference text that are enclosed by target punctuation marks, and the complete semantic text inside the quotation marks is extracted separately as an independent piece of evidence. Configure a trigger word lexicon containing the target semantic connotations of observation, reading, display, and numerical identification. Use semantic pattern matching to traverse reasoning text statements, identify target statements containing preset semantic trigger words, and extract complete semantic fragments associated with the trigger words. The evidence fragments obtained through the three extraction methods are summarized, redundancy is removed, and they are sorted and integrated to form a complete and non-repeating set of model claim evidence fragments.

5. The method according to claim 4, characterized in that, The set of evidence fragments is matched with the set of truth values ​​at multiple levels. Evidence recall and truth value coverage are calculated based on the matching results. A correctness reward is determined based on the predicted answer and the reference answer, including: The model evidence fragment set and the evidence truth value set are mapped one-to-one to establish a corresponding relationship between the evidence elements to be matched and the truth value elements. Following a hierarchical order from rigorous to broad and from precise to fuzzy, the system progressively conducts four levels of verification: precise matching, numerical tolerance matching, inclusion relationship matching, and fuzzy similarity matching, and then filters and determines the validity of the matching at each level. The number of valid evidence that passed matching at each level, the number of unmatched evidence, and the number of truth values ​​covered were statistically analyzed. Based on the statistical quantitative results, the two core evaluation indicators, evidence recall rate and truth value coverage rate, were calculated in a standardized manner. We differentiate between two types of visual question answering samples: chart-based and document-based. We then adapt different answer comparison and judgment logics to each type, comparing the predicted answer with the standard reference answer from the dimensions of content error and text similarity. Based on the comparison and judgment results under different sample scenarios, the levels are divided and numerical values ​​are assigned according to the preset reward quantification rules, and the corresponding sample's answer correctness reward is quantified.

6. The method according to claim 5, characterized in that, The evidence anchoring reward is determined based on the evidence recall rate and the truth coverage rate. A combined reward is calculated based on the answer correctness reward and the evidence anchoring reward. The parameters of the visual language model are updated using a policy optimization algorithm to achieve evidence anchoring-based visual language model training, including: The set of evidence fragments output by the model is subjected to non-empty validity judgment, and different reward calculation logics are adopted according to whether there is valid evidence content in the set. When the evidence fragment set is in a valid non-empty state, the evidence recall rate, truth coverage rate and preset weight coefficient are combined for weighted fusion calculation to quantify and solve for the corresponding evidence anchoring reward. When the evidence fragment set is in an empty invalid state, a preset negative penalty value is directly assigned as the evidence anchoring reward. The quantitatively determined answer correctness reward and evidence anchoring reward are combined and weighted, and the overall combined reward corresponding to a single training sample is calculated in accordance with the preset combination rules. Using combined rewards as feedback constraints for model optimization, an appropriate policy optimization algorithm is introduced to construct a gradient propagation path in reverse along the direction of reward maximization. The network weights and internal hyperparameters of the visual language model are iteratively adjusted layer by layer based on the gradient backpropagation mechanism. Multiple rounds of iterative updates and convergence optimizations are continuously completed, and finally, a visual language model with strong evidence anchoring reasoning ability is obtained through training convergence.

7. A visual language model training system based on evidence-anchored rewards, characterized in that, include: The sample construction module is used to construct a training sample dataset, collect training samples containing question text, images to be parsed, standard reference answers, and structured data sources, and extract a set of truth values ​​of evidence related to the reference answers from the structured data sources; The model inference module is used to input the question text and the image to be parsed into the visual language model to obtain the model response result containing the inference text and the predicted answer; The evidence extraction module is used to extract a set of evidence fragments claimed by the model based on the reasoning text through numerical pattern extraction, quotation mark text extraction and semantic pattern matching extraction methods. The matching and calculation module is used to perform multi-level matching between the evidence fragment set and the evidence truth value set, calculate the evidence recall rate and truth value coverage rate according to the matching results, and determine the answer correctness reward according to the predicted answer and the reference answer. The optimized training module is used to determine the evidence anchoring reward based on the evidence recall rate and the truth coverage rate, calculate the combined reward based on the answer correctness reward and the evidence anchoring reward, and update the parameters of the visual language model using a policy optimization algorithm to achieve evidence anchoring-based visual language model training.

8. The system according to claim 7, characterized in that, The sample building module is also used for: Collect the basic training data required for the visual question answering task, and collect each complete single sample unit containing the question text, the image to be parsed, the standard reference answer, and the supporting structured data source. Organize and summarize them by category to form a large-scale training sample dataset. The collected structured data sources are classified and their attributes are identified to accurately distinguish the underlying data table resources corresponding to chart materials and the optical character recognition text resources corresponding to document materials, so as to realize the classification and archiving management of data sources; Differentiated truth extraction logic is used for different types of training samples to locate the key content of the standard reference answer from the corresponding structured data source, and filter and integrate it to form a set of evidence truths that match the correlation.

9. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

Images