Model training method and device, visual inference and device

Abstract

This specification provides a model training method and apparatus, and a visual reasoning apparatus. The model training method includes: acquiring a sample dataset, wherein each sample data item in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain; using a visual reasoning model to be trained, performing reasoning based on the sample image and sample question to obtain a predicted answer and a corresponding predicted thought chain; using a pre-trained scoring model, based on the sample thought chain, evaluating whether the conclusion of each reasoning step in the predicted thought chain is derived from the preceding reasoning steps and visual evidence in the sample image, to obtain a logical validity score; and training the visual reasoning model to be trained based on the logical validity score to obtain a trained visual reasoning model. This improves the training effect of the visual reasoning model, thereby enhancing the reliability and accuracy of the visual reasoning model in complex visual scenes.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to a model training method, a visual reasoning method, a model training device, and a visual reasoning device. Background Technology

[0002] With the rapid development of artificial intelligence technology, multimodal large language models are increasingly widely used in the field of visual reasoning. By integrating image understanding and text reasoning, they can complete complex tasks such as image-text question answering and mathematical reasoning. Visual reasoning ability has become an important indicator for measuring the comprehensive performance of multimodal models.

[0003] Currently, multimodal large language models are typically trained on question-and-answer data that includes images and questions, enabling the model to generate a reasoning process in the form of thought chains while outputting the answer. To improve reasoning ability, verifiable reward reinforcement learning is usually adopted, using whether the model's output answer matches the standard answer as a reward signal to update the model parameters.

[0004] However, in practical applications, because the reasoning process of the model's generated thought chain is difficult to quantify and evaluate, existing training methods typically only use the correctness of the final answer as a reward signal. This fails to effectively constrain whether the conclusions of intermediate reasoning steps are derived from preceding reasoning steps and visual evidence in the image. Consequently, in complex visual scenes, models are prone to problems such as lack of visual evidence for reasoning steps, incoherent reasoning chains, and inconsistencies between reasoning conclusions and image content. Therefore, there is an urgent need for a model training method that yields better training results and improves the reasoning performance of the trained model. Summary of the Invention

[0005] In view of the above, embodiments of this specification provide a model training method. One or more embodiments of this specification also relate to a visual reasoning method, a model training apparatus, a visual reasoning apparatus, a computing device, a computer-readable storage medium, and a computer program product, to solve the aforementioned technical problems.

[0006] According to a first aspect of the embodiments of this specification, a model training method is provided, comprising: Obtain the sample dataset, in which each sample data includes a sample image, a sample question, a sample answer, and a sample thought process chain; Using a visual reasoning model to be trained, reasoning is performed based on sample images and sample questions to obtain predicted answers and corresponding predictive thought chains; Using a pre-trained scoring model, based on the sample thought chain, the conclusions of each reasoning step in the predicted thought chain are derived from the preceding reasoning steps and visual evidence in the sample image, and a logical validity score is obtained. Based on the logical validity score, the visual reasoning model to be trained is trained to obtain the trained visual reasoning model.

[0007] According to a second aspect of the embodiments of this specification, a visual reasoning method is provided, comprising: Obtain the target image and target question to be reasoned; Using a visual reasoning model, reasoning is performed based on the target image and the target question to obtain the target answer and the corresponding target thought chain. The visual reasoning model is trained according to the model training method described above.

[0008] According to a third aspect of the embodiments of this specification, a model training apparatus is provided, comprising: The first acquisition module is configured to acquire a sample dataset, wherein each sample data in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain; The first reasoning module is configured to use the visual reasoning model to be trained to reason based on sample images and sample questions to obtain the predicted answer and the corresponding predicted thought chain. The evaluation module is configured to use a pre-trained scoring model to evaluate whether the conclusions of each reasoning step in the predicted thinking chain are derived from the preceding reasoning steps and visual evidence in the sample images, based on the sample thinking chain, and obtain a logical validity score. The training module is configured to train the visual reasoning model to be trained based on logical validity scoring, and obtain the trained visual reasoning model.

[0009] According to a fourth aspect of the embodiments of this specification, a visual reasoning device is provided, comprising: The second acquisition module is configured to acquire the target image and target question to be reasoned. The second reasoning module is configured to use a visual reasoning model to reason based on the target image and the target question to obtain the target answer and the corresponding target thought chain. The visual reasoning model is trained according to the model training method described above.

[0010] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above method.

[0011] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0012] According to a seventh aspect of the embodiments of this specification, a computer program product is provided, including computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0013] One embodiment of this specification implements a model training method, comprising: acquiring a sample dataset, wherein each sample data in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain; using a visual reasoning model to be trained, performing reasoning based on the sample image and the sample question to obtain a predicted answer and a corresponding predicted thought chain; using a pre-trained scoring model, taking the sample thought chain as a benchmark, evaluating whether the conclusion of each reasoning step in the predicted thought chain is derived from the preceding reasoning steps and visual evidence in the sample image, and obtaining a logical validity score; and training the visual reasoning model to be trained based on the logical validity score to obtain a trained visual reasoning model.

[0014] By acquiring a sample dataset containing sample images, sample questions, sample answers, and sample thought chains, we can provide comprehensive training samples for model training. The sample thought chains provide a benchmark for evaluating the reasoning process. By using the visual reasoning model to be trained to reason based on sample images and sample questions and generate predicted answers and corresponding predicted thought chains, the model's reasoning process is explicitly output, observable, and evaluable. By using a pre-trained scoring model, based on the sample thought chains, we can evaluate whether the conclusions of each reasoning step in the predicted thought chain are derived from the preceding reasoning steps and visual evidence in the sample images, obtaining a logical validity score. This allows for fine-grained evaluation of the derivation validity of reasoning steps, prompting each reasoning step to anchor itself to visual evidence in the image and avoiding the model relying on linguistic prior guesses for answers. By training the visual reasoning model to be trained based on the logical validity score, the training reward directly affects the quality of the reasoning process, improving the reliability and accuracy of the model's reasoning in complex visual scenes, thereby improving the model's training effect and making the reasoning of the trained visual reasoning model more accurate. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating a model training method provided in one embodiment of this specification; Figure 2 This is a schematic diagram of the execution framework of a model training method provided in one embodiment of this specification; Figure 3a This is a flowchart of the first stage of a training data construction method provided in one embodiment of this specification; Figure 3b This is a flowchart of the second-stage processing of a training data construction method provided in one embodiment of this specification; Figure 4 This is a flowchart illustrating a visual reasoning method provided in one embodiment of this specification; Figure 5 This is a schematic diagram of the structure of a model training device provided in one embodiment of this specification; Figure 6 This is a schematic diagram of the structure of a visual reasoning device provided in one embodiment of this specification; Figure 7 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0016] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0017] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0018] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0019] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are obtained through open-source datasets or public datasets that comply with their license agreements, or are obtained with full authorization from the relevant parties. Moreover, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0020] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability. Examples include Large Language Models (LLMs) and multi-modal pre-training models.

[0021] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as NLP tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0022] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0023] A Multimodal Large Language Model (MLLM) is a deep learning model capable of simultaneously processing multiple modalities of information, such as images and text, and generating text output. MLLMs are typically trained on large-scale image and text data and possess cross-modal capabilities such as image understanding, visual question answering, and image-text reasoning. They are currently the primary implementation vehicle for visual understanding tasks in the field of artificial intelligence.

[0024] Large Language Model (LLM) is a large-scale neural network model pre-trained on massive amounts of text data. It is usually built with a self-attention architecture and has the ability to understand, generate and reason about natural language. It can perform a variety of natural language processing tasks such as text classification, question answering, summarizing and dialogue.

[0025] Chain-of-Thought (CoT) refers to a reasoning method in which a model generates a series of intermediate reasoning steps before giving the final answer. CoT makes the model's reasoning process explicit, improving its performance on complex tasks such as mathematics and logic through step-by-step derivation. Each step in a CoT typically corresponds to a semantically complete reasoning action.

[0026] Reinforcement learning (RL) is a machine learning paradigm that teaches optimal policies by having an agent interact with its environment and learn based on reward signals. In reinforcement learning, the agent performs actions and receives rewards from the environment, gradually optimizing its policy by maximizing cumulative rewards. It is widely used in scenarios such as game theory, robot control, and training large language models.

[0027] A Visual-Language Model (VLM) is a deep learning model capable of processing both image and text modal information simultaneously. A VLM typically includes two branches: image encoding and text encoding. It is trained on large-scale image-text pairs and possesses cross-modal understanding capabilities such as image description, visual question answering, and image-text retrieval.

[0028] An instruction-guided image editing model is a generative model that can edit input images based on natural language editing instructions. It receives the original image and text instructions describing the editing intent, and outputs an edited image that meets the requirements of the instructions, capable of modifying specified regions or attributes while maintaining the overall structure of the image.

[0029] Autoregressive is a sequence generation method that predicts unit by unit. In each generation step, the autoregressive model predicts the next unit based on the already generated content and appends the prediction to the end of the sequence as input for the next step. This process is repeated until a complete sequence is generated. It is a commonly used generation paradigm in text generation and image generation tasks.

[0030] A token is the smallest semantic unit in text processing. It refers to a discrete unit obtained after segmenting the original text, which can correspond to a word, a subword, or a character. Before inputting text into a model, it is usually segmented into a token sequence. Each token is mapped to an index or vector representation in the vocabulary for subsequent processing by the model.

[0031] Semantic similarity is a quantitative metric that measures how close two text segments are semantically. Semantic similarity can be calculated based on distance in vector space, for example, by mapping text to vectors and then calculating cosine similarity. The higher the similarity value, the closer the two text segments are semantically.

[0032] Object detection is a fundamental task in computer vision, used to locate the position of objects in an image and identify their category. Object detection models output bounding boxes and category labels for each object in an image, serving as a prerequisite for visual reasoning tasks such as counting and spatial relationship analysis.

[0033] Decoding temperature is a control parameter in the sampling process of a text generation model, used to adjust the randomness of the generated results. A higher decoding temperature tends to generate more diverse outputs, while a lower value tends to generate more deterministic and stable outputs.

[0034] A visual encoder is a neural network component used to extract visual features from images. It transforms an input image into a feature representation with semantic information, typically consisting of multiple convolutional or self-attention structures. Shallow features preserve details such as texture and edges, while deeper features represent higher-level information such as object categories and scene semantics.

[0035] An image patch is a fixed-size local region obtained by dividing an image according to its spatial location. When an image is input into a visual model, it can first be divided into multiple image patches, and each image patch can be mapped to a feature vector to form a feature sequence of the image for the model to process.

[0036] STEM (Science, Technology, Engineering, and Mathematics) is a collective term for four disciplines: science, technology, engineering, and mathematics. Problems in the STEM field typically require the integrated application of knowledge from multiple disciplines for reasoning, making them an important source of tasks for evaluating model reasoning abilities.

[0037] With the rapid development of artificial intelligence technology, multimodal large language models are increasingly widely used in the field of visual reasoning. By integrating image understanding and text reasoning, they can complete complex tasks such as image-text question answering and mathematical reasoning. Visual reasoning ability has become an important indicator for measuring the comprehensive performance of multimodal models.

[0038] Currently, multimodal large language models are typically trained on question-and-answer data that includes images and questions, enabling the model to generate a reasoning process in the form of thought chains while outputting the answer. To improve reasoning ability, verifiable reward reinforcement learning is usually adopted, using whether the model's output answer matches the standard answer as a reward signal to update the model parameters.

[0039] However, in practical applications, because the reasoning process of the thought chain generated by the model is difficult to quantify and evaluate, existing training methods usually only use the correctness of the final answer as a reward signal. They cannot effectively constrain whether the conclusions of intermediate reasoning steps are derived from the preceding reasoning steps and visual evidence in the image. This leads to problems such as the lack of visual evidence for reasoning steps, incoherent reasoning chains, and inconsistencies between reasoning conclusions and image content in complex visual scenes.

[0040] In view of this, this specification provides a model training method, and also relates to a visual reasoning method, a model training device, a visual reasoning device, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0041] Specifically, see Figure 1 , Figure 1 A flowchart of a model training method according to an embodiment of this specification is shown, such as... Figure 1 As shown, the method specifically includes the following steps.

[0042] Step 102: Obtain the sample dataset, where each sample data item in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain.

[0043] The model training method provided in one or more embodiments of this specification can be applied to the field of multimodal visual reasoning, especially in scenarios requiring detailed observation and step-by-step reasoning of image content, such as visual understanding and mathematical reasoning of text and image content on community platforms and educational problem-solving. Specifically, this model training method can be implemented on various computing platforms, including but not limited to cloud server clusters, high-performance local workstations, and mobile terminal devices.

[0044] In practical applications, this model training method can be deployed in a multimodal model training platform. This platform can receive training requests from the front end, execute training processes such as acquiring sample datasets, model inference and scoring, and updating model parameters. It can also schedule computing resources in the cloud server cluster on demand to provide computational support for image processing, model inference, and parameter updates during the training process. The trained model can serve businesses such as visual understanding of text and image content, mathematical reasoning, and educational problem solving on community platforms. It can also be deployed on the inference server to provide real-time visual inference capabilities for related applications. The data construction pipeline involved in this method can also serve as a general multimodal training data production tool, continuously supplying training data for various models that require visual inference capabilities, thereby meeting the needs of different business scenarios for visual inference capabilities.

[0045] A sample dataset is a collection of sample data used to train a multimodal large language model. It contains multiple sample data points, each corresponding to a single multimodal visual reasoning training instance. Specifically, the sample dataset can consist of multimodal thought chain training samples. Each sample data point can be manually collected or generated by a synthetic pipeline. The synthetic pipeline can be constructed in two stages: first, the seed data is rewritten and edited to modify the answers; then, the answers are kept unchanged while diversifying the attributes of the free images. Furthermore, the size of the sample dataset can be flexibly set according to training needs, for example, the number of samples can be determined based on available computing resources and the difficulty of the target task. The composition of the sample dataset can be flexibly organized, for example, it can cover sample data of various question types, various visual scenes, or different difficulty levels. During training, the sample dataset can also be dynamically expanded and updated, for example, newly generated sample data can be continuously added to the sample dataset to improve the coverage and diversity of the training samples.

[0046] Sample images are the image portions of the sample data corresponding to the sample questions, used to present the scene content and visual evidence required for visual reasoning to the model. Specifically, sample images can include real scene photographs, synthetic variant images, etc. For example, they can include real photographs of flocks of birds, variant images generated after editing real photographs, and scene images synthesized by image generation models. Furthermore, the source of sample images can be flexibly selected, such as from public datasets, manual collection, or data construction pipeline generation; the storage format of sample images can also be diverse, such as using common image file formats, and the image resolution, size, and color mode can be set according to training needs; the visual content in the sample images needs to correspond to the visual elements examined by the sample questions, such as the number, position, and attributes of objects in the image should be consistent with the question content; during training, sample images can be decoded and preprocessed in batches, converted into tensor forms required for model input.

[0047] Sample questions are the question text corresponding to sample images, used to describe the visual reasoning task that the model needs to answer based on the content of the sample images. Specifically, sample questions can be visual reasoning questions, such as counting, spatial relationship determination, attribute recognition, and conditional selection. Sample questions can be obtained through template instantiation; that is, a question template containing slots can be pre-designed. After determining the sample image, elements such as the subject, quantity, and position are filled into the template by assigning values ​​to the slots, generating question text that matches the image content. Furthermore, the sentence structure of sample questions can be flexibly designed. For example, for a sample image containing both real birds and cartoon birds, the sample question can be set as "The picture includes real birds and cartoon birds. How many birds are there in total?" Other question structures can be set around the spatial relationship, color distribution, or quantity changes of objects in the image to cover diverse visual reasoning examination angles. Sample questions can also carry additional information such as test point annotations and difficulty annotations to support the orderly organization of sample data during training.

[0048] Sample answers are the standard answers corresponding to sample questions, indicating the correct responses the model should provide. They serve as the basis for measuring the correctness of the model's reasoning results during training. Specifically, sample answers can be mechanically derived through editing operations. For example, when performing editing operations such as removal, addition, or attribute rewriting on sample images, the new answer can be directly calculated based on the impact of these operations on the answer, without manual annotation. Sample answers can also be obtained through manual or automatic annotation. Each sample answer corresponds one-to-one with a sample question and maintains consistency with the visual content in the sample image; for example, the sample answer updates accordingly when the number of objects in the image changes. Furthermore, the form of sample answers can be flexible and diverse, including numerical values, category labels, Boolean judgments, phrases, or short sentences, depending on the type and content of the sample question. During training, sample answers can be compared with the model's output answers to determine the correctness of the model's responses, meeting the training needs of different visual reasoning tasks.

[0049] A sample thought chain is the text describing the reasoning process corresponding to a sample image, sample question, and sample answer. It depicts the complete reasoning trajectory of the model from observing the sample image to arriving at the sample answer. Specifically, the sample thought chain can include a four-step structure: overall observation, key element identification, task-oriented reasoning, and conclusion. Overall observation describes the overall structure and object scale of the image; key element identification lists the objects and their attributes related to the question; task-oriented reasoning performs operations such as counting, spatial relationship determination, and condition filtering, applying these to the identified visual elements; and the conclusion provides the final answer. The purpose of the sample thought chain is to serve as a reference reasoning chain for the scoring model in subsequent training phases, scoring the model's output reasoning process, rather than directly constraining the model's output as a supervisory signal. Furthermore, the writing format of the sample thought chain can be flexibly set. For example, it can use 3 to 6 sentences of natural language and end with a fixed marker, such as "Final Answer:" to indicate the final answer, or it can use... <answer>Provide the final answer in the format of "answer< / answer".

[0050] In practical applications, acquiring the sample dataset can be achieved through a variety of methods.

[0051] An optional method is to read sample data from existing datasets or storage systems. Specifically, sample data can be pre-organized into a dataset form and stored in a local storage medium, a distributed file system or a cloud database. Each piece of sample data can be stored in the form of a structured record, and each record completely includes complete fields such as a sample image, a sample question, a sample answer and a sample thought chain. When acquiring the sample dataset, all sample data can be read from the above storage locations, or part of the sample data can be filtered according to preset conditions. For example, filtering can be performed according to question type, difficulty level or data source, and filtering can also be performed according to the quality label or timestamp of the sample data. Format conversion and integrity check can also be performed on the sample data during the reading process. For example, it can be checked whether each piece of sample data simultaneously includes the four parts of the sample image, the sample question, the sample answer and the sample thought chain, and the correspondence between each field can be verified. In addition, a version management mechanism can be used to record the update history and version information of the sample dataset, so as to facilitate the reproduction and tracing of training results, thereby obtaining a sample dataset that meets the training requirements.

[0052] Another optional method is to use a data construction pipeline to synthesize and generate the sample dataset. Specifically, a data construction pipeline can be constructed in which functional modules such as an analyzer, a director, a generator, a planner and a verifier work collaboratively to generate sample data in batches in a two-stage manner. For example, in the first stage, answer rewriting editing can be performed on seed images, and new answers can be mechanically deduced according to the influence law of editing operations on the answers. In the second stage, the answer can be kept unchanged, and diversified expansion can be performed on free image attributes irrelevant to the answer to generate multiple sets of variant images with different appearances but consistent answers, and the sample questions and sample thought chains corresponding to each variant image can be generated synchronously, and finally sample data including sample image, sample question, sample answer and sample thought chain is output. The generated sample data can be aggregated into a sample dataset after quality check. For example, unqualified samples can be discarded according to the judgment result of the verifier, and deduplication and distribution statistics can be performed on sample data generated in different batches to ensure the diversity of the sample dataset. The specific implementation of this data construction pipeline can refer to the description in the subsequent examples of this specification.

[0053] Another option is to obtain sample datasets through a combination of manual compilation and automatic annotation. Specifically, images and question materials from real-world scenarios can be collected manually, such as text and image content from community platforms or practice questions from educational scenarios, and corresponding answers can be determined for the questions. At the same time, automatic annotation tools or models with visual understanding capabilities can be used to perform content analysis on the sample images to assist in generating sample questions, sample answers, and sample thought processes. For example, counting questions and their answers can be automatically generated based on the number, position, and attributes of objects in the image, along with corresponding reasoning text. The automatically generated sample data can be manually reviewed and corrected to form complete sample data, and multiple compiled sample data can be aggregated into a sample dataset. Manually compiled sample data and automatically generated sample data can be used together or organized separately according to their sources to meet different training needs. For sample data with low confidence during the automatic generation process, manual review can be prioritized to control sample quality, thereby improving the efficiency of sample dataset acquisition while ensuring sample data quality.

[0054] In addition, sample datasets can be obtained by combining multiple sources, such as mixing existing data, synthetic data and manually labeled data in a preset ratio, in order to balance data diversity and labeling quality.

[0055] In this step, by acquiring a sample dataset, the multimodal large language model obtains training samples with complete information. Each sample dataset contains corresponding sample images, sample questions, sample answers, and sample thought chains, providing the model with comprehensive training materials covering visual input, reasoning tasks, standard answers, and reasoning processes. This allows the model to simultaneously learn multiple capabilities during training, including image understanding, question parsing, and reasoning expression. The clear correspondence between sample questions and answers also provides the model with a clear learning objective. The sample thought chains can serve as a benchmark for evaluating the reasoning process in subsequent training stages, allowing the intermediate reasoning steps output by the model to be compared and scored, providing a basis for evaluating the model's reasoning quality. The diversity of visual content and question types in the sample dataset also helps the model enhance its generalization ability while maintaining reasoning stability. Acquiring the sample dataset provides a data foundation for subsequent model reasoning and scoring training, which is beneficial for improving the accuracy and reliability of the model's visual reasoning in real-world scenarios, thus creating conditions for improving the reasoning performance of the multimodal large language model in complex image-text scenarios.

[0056] Step 104: Using the visual reasoning model to be trained, reason based on the sample images and sample questions to obtain the predicted answer and the corresponding predicted thought chain.

[0057] The visual reasoning model to be trained is a neural network model capable of receiving image and text inputs and outputting text reasoning results. Specifically, the visual reasoning model to be trained can be a multimodal large language model, which may include components such as a visual encoder and a language decoder. The visual encoder is used to convert sample images into visual features, and the language decoder is used to perform autoregressive reasoning based on visual features and the text content of the sample question to generate output text. For example, the visual encoder can adopt a visual transformer structure to segment the image into multiple image patches and encode them into visual feature sequences, and the language decoder can generate reasoning content word by word based on the visual feature sequences and the word sequence of the question text. Furthermore, the visual reasoning model to be trained can be trained using a group relative policy optimization method for reinforcement learning to improve visual reasoning ability. The specific training implementation method will be described in subsequent steps.

[0058] The predicted answer is the response output by the visual reasoning model after reasoning. Specifically, the predicted answer is a prediction generated by the visual reasoning model based on sample images and sample questions, corresponding to the sample answers. The sample images and sample questions are taken from a sample dataset. The predicted answer can be presented in various forms, such as numerical counting results, categorical judgment results, or textual descriptions. The predicted answer can be compared with the sample answers to measure the accuracy of the model's response. It can also serve as an important basis for judging whether the model has given correct results during training, providing clear feedback for model training and evaluation. Furthermore, the predicted answer is located at the end of the predictive thought chain. The visual reasoning model outputs the predicted answer after generating the reasoning process text. The predictive thought chain and the predicted answer together constitute the model's complete reasoning response to the sample images and sample questions. This facilitates the subsequent extraction of the reasoning process and the final answer from the complete response, which can then be used for process evaluation, answer verification, and model performance analysis, respectively.

[0059] The predictive thought chain is the reasoning process text generated by the visual reasoning model before outputting the predicted answer. Specifically, the predictive thought chain can include multiple reasoning steps, each described in a single sentence. For example, reasoning steps might include observing the overall image, identifying relevant objects, or counting and reasoning about the number of objects. Each reasoning step is written in natural language, without using a numbered list format, progressing step-by-step until the final conclusion is reached. It fully records the model's process of deriving the answer from visual information, and its content organization corresponds to the sample thought chains in the sample dataset. The number of reasoning steps can be flexibly determined based on the complexity of the sample questions. Furthermore, the predictive thought chain can use a fixed ending format, for example, outputting "Final Answer: 4" or "..." at the end of the reasoning text. <answer> 4< / answer> The "" format makes it clear that the predicted answer is distinct from the reasoning process, which facilitates the accurate extraction of the predicted answer from the complete response and also makes it easier to evaluate the organization and closing format of the model's reasoning process, thus providing convenience for the standardization of the model output.

[0060] In practical applications, the visual reasoning model to be trained can be used to reason based on sample images and sample questions to obtain predicted answers and corresponding predictive thought chains, which can be achieved through various methods.

[0061] One alternative approach is to directly input sample images and sample questions into the visual reasoning model to be trained. The model then generates a predicted thought chain and a predicted answer in an end-to-end autoregressive manner. Specifically, after obtaining the sample images, they can be segmented into multiple image blocks. For example, the sample images can be uniformly segmented into 16, 64, or other number of image blocks according to a preset size, and the visual encoder encodes each image block into a visual feature sequence. Simultaneously, the sample questions can be lexicalized to obtain a word sequence of the question text. The visual feature sequence and the word sequence of the question text are concatenated and input into a language decoder. At each step, the language decoder predicts the next word based on the generated content, sequentially generating a predicted thought chain containing multiple reasoning steps and a predicted answer at the end. Thus, the predicted thought chain and predicted answer are obtained simultaneously in a single forward reasoning process. In this approach, the entire reasoning process is automatically completed by the model without the need for additional intermediate processing steps. The implementation is simple and efficient, and the predicted thought chain and predicted answer are continuously connected in the text, directly serving as the model's complete response for subsequent processing.

[0062] Another alternative approach is to first extract visual features from the sample image using a visual encoder, and then perform multi-step reasoning based on these extracted visual features and the sample questions. Specifically, the sample image can be input into the visual encoder, which extracts visual features from the image. For example, it can extract regional features of different areas of the image, patch features corresponding to different image blocks, local features of key objects related to the sample questions, or global features describing the overall layout of the image. Subsequently, the extracted visual features and the text representation of the sample questions are input into a language decoder. The language decoder performs step-by-step reasoning based on the visual features and the question content, generating a predictive thought chain, and outputting the predicted answer at the end of the predictive thought chain. In this approach, visual feature extraction and reasoning text generation are relatively separated. Visual features can be filtered or processed before reasoning to highlight visual information relevant to the sample questions and reduce interference from irrelevant visual content. Simultaneously, the same set of visual features can be used to process multiple sample questions for the same sample image, which helps improve the utilization efficiency of computing resources and the overall processing speed.

[0063] Another alternative approach is to explicitly generate step-by-step reasoning text during the reasoning process, using the output of the previous step as the input for the next, guiding the reasoning forward step by step. Specifically, the visual reasoning model can first generate reasoning text describing the overall observation results, and then, based on this reasoning text and combined with visual elements in the sample image, sequentially generate reasoning text for subsequent steps such as object recognition, conditional selection, and counting reasoning. This ensures that each step of reasoning is progressively grounded in the identified visual elements. When generating each step of reasoning text, the visual reasoning model can verify the reasoning conclusion of the previous step by combining visual features in the sample image. Only after completing all reasoning steps can the predicted answer be output. For example, for a sample question testing the quantity of objects, the model can first identify the target object in the image that meets the conditions, and then perform counting based on the identified object to obtain the predicted answer. Each step of reasoning is built upon the previous reasoning results and specific visual elements. In this approach, the dependencies between the steps of the reasoning process are clear, and the reasoning path is well-defined, which helps improve the interpretability and stability of the reasoning process and facilitates the subsequent location and evaluation of the correctness of each step in the reasoning process.

[0064] In addition, reasoning strategies can be selected based on the specific task type. For example, for counting problems, target detection and counting reasoning can be performed first, and for spatial relationship problems, positional relationship analysis can be performed first, and then predictive answers and predictive thought chains can be generated in a unified manner.

[0065] In this step, the visual reasoning model to be trained is used to reason based on sample images and sample questions to obtain predicted answers and corresponding predicted thought chains. The model's reasoning process is output in explicit text form. During the answering process, the model not only provides the final result but also fully presents the reasoning basis for arriving at the result. The correspondence between reasoning steps and visual evidence in sample images is established. The visual information and reasoning steps on which the reasoning is based can be fully observed. The model's reliance on the actual content of the image to organize the reasoning path can be directly grasped, thereby enabling the verification and evaluation of the correctness of the reasoning process. At the same time, the predicted answer and the predicted thought chain together constitute the model's complete response, corresponding to the sample answers and sample thought chains in the sample dataset. This provides a clear object for subsequent step-by-step evaluation of each reasoning step in the predicted thought chain using a scoring model, making it possible to obtain process-level reward signals. The feedback obtained by the model during training is no longer limited to the correctness of the final result. Process-level supervision information and answer-level supervision information can cooperate to guide the model to optimize its reasoning behavior and reasoning path selection, thus providing a solid foundation for improving the reasoning quality, reliability, and interpretability of the visual reasoning model.

[0066] Step 106: Using the pre-trained scoring model, based on the sample thought chain, evaluate whether the conclusions of each reasoning step in the predicted thought chain are derived from the preceding reasoning steps and the visual evidence in the sample image, and obtain a logical validity score.

[0067] A pre-trained scoring model is used to score the reasoning process. Specifically, the pre-trained scoring model can be a pre-trained visual language model or a large language model. This model has the ability to score the predicted reasoning chain from multiple dimensions based on the reference reasoning chain. For open-source visual language models or large language models, this scoring capability is inherent and does not require additional specialized training for the scoring task. The scoring model can receive sample reasoning chains and predicted reasoning chains as input and score the reasoning quality in the predicted reasoning chain. Furthermore, the logical validity score can be directly output by the pre-trained scoring model. The scoring model does not need to set additional calculation formulas or scoring rules. The model directly gives a score based on its own understanding of the reasoning content. The scoring method is simple and direct and easy to transfer between different models and different tasks.

[0068] Reasoning steps are intermediate reasoning units in the predictive thought chain. Specifically, the predictive thought chain can be divided into multiple reasoning steps, each containing a semantically complete reasoning action. For example, a reasoning step can be expressed as "The image contains 3 real birds and 4 cartoon birds," or as "Add the number of objects in the left area to the number of objects in the right area to get the total." There are various ways to divide reasoning steps. For example, they can be divided by sentence boundaries, with one natural sentence corresponding to one reasoning step; they can be divided by semantic completeness, merging several sentences expressing the same reasoning intention into one reasoning step; or they can be divided by the type of reasoning action, treating different types of reasoning actions such as observation, counting, comparison, and inference as independent reasoning steps. Furthermore, reasoning steps constitute the objects of subsequent step-by-step evaluation. The scoring model evaluates the predictive thought chain step by step, with each reasoning step evaluated sequentially according to its order in the predictive thought chain, thereby achieving a fine-grained quality characterization of the reasoning process.

[0069] The conclusion is the judgment result stated in the reasoning step. Specifically, the conclusion of each reasoning step is a factual statement or inference result obtained based on the input information. For example, in counting reasoning, the conclusion of a reasoning step could be "There are 7 objects in the scene," and in comparison reasoning, the conclusion of a reasoning step could be "The number of objects on the left is greater than the number of objects on the right." The conclusion can be distinguished from other content in the reasoning step. For example, the conclusion sentence can be placed at the end of the reasoning step, with the first half of the step describing the observation or calculation process on which it is based, and the last sentence stating the judgment derived from that process. Alternatively, it can be placed at the beginning of the reasoning step, stating the judgment first and then supplementing the basis. Fixed sentence patterns can also be used to identify the conclusion. Furthermore, the judgment of the conclusion of each reasoning step is the direct object of logical validity assessment. The scoring model needs to determine whether the conclusion is indeed derived from the visual evidence in the preceding reasoning steps and the sample images, rather than from guesswork or linguistic a priori reasoning. The judgment result directly determines the logical validity score of that step.

[0070] Preceding reasoning steps are the reasoning steps in the predictive thought chain that precede the current reasoning step. Specifically, preceding reasoning steps constitute the premises for the current reasoning step. These can include direct preceding steps adjacent to the current step, such as the immediately preceding reasoning step, or earlier reasoning steps, such as multiple consecutive steps preceding the current step and the intermediate conclusions derived from each step. There can be one or more preceding reasoning steps. For example, in a three-step reasoning process of "counting the number of real birds, counting the number of cartoon birds, and calculating the total number," the preceding reasoning steps for calculating the total number are the steps for counting the number of real birds and counting the number of cartoon birds. Furthermore, the conclusions of preceding reasoning steps can serve as the basis for the current reasoning step. The conclusion of the current reasoning step should be obtainable from the conclusions of preceding reasoning steps through reasonable reasoning operations. If there is no deductive relationship between the current conclusion and the preceding conclusion, or if the premises upon which the deduction is based do not appear in the preceding steps, then the logical validity of the current reasoning step is low.

[0071] Visual evidence refers to observable visual information in a sample image that is relevant to reasoning. Specifically, visual evidence includes directly observable content such as the category, color, shape, size, quantity, spatial relationship, texture, and material of objects in the sample image. For example, information such as the number of objects in the image, their relative positions, occlusion relationships between objects, and the distribution of objects on a surface. Reasoning steps should be anchored to this objective visual information, rather than making inferences detached from the image content. Visual evidence is distinct from linguistic a priori knowledge. Linguistic a priori knowledge relies on common sense, experience, or existing knowledge stored in a knowledge base to make guesses about the problem, while visual evidence needs to originate from actual observation of the sample image. The two are fundamentally different in their sources.

[0072] Furthermore, in the logical validity assessment, visual evidence is used to determine whether the conclusion of the reasoning step is objectively supported by the image content. The derivation of the step is considered valid only when the conclusion of the step can correspond to the specific visual information in the sample image. The clearer the correspondence, the more fully the visual evidence supports the conclusion.

[0073] Logical validity score is a rating that characterizes the validity of a reasoning step. Specifically, a higher logical validity score indicates that the conclusion of the reasoning step is more genuinely derived from the preceding reasoning steps and visual evidence in the sample image, rather than based on guesswork or linguistic priors. The logical validity score can take values ​​within a preset range, such as between 0 and 1. A value of 1 indicates that the conclusion of the step is entirely derived from the preceding reasoning steps and visual evidence, while a value of 0 indicates that there is no deductive relationship between the conclusion of the step and its premises.

[0074] Furthermore, the scores of multiple reasoning steps can be aggregated into a logical validity score for the entire predictive thinking chain. There are various ways to aggregate the scores. For example, the average score of each step can be taken as the overall score, or the scores of each step can be weighted according to their importance in the reasoning chain. For example, steps that carry key reasoning actions can be given greater weight, thereby obtaining an aggregated score that reflects the overall logical validity of the entire predictive thinking chain, which is used to measure the reasoning quality of the entire predictive thinking chain.

[0075] In practical applications, a pre-trained scoring model can be used to evaluate whether the conclusions of each reasoning step in the predicted thinking chain are derived from the preceding reasoning steps and the visual evidence in the sample image, thereby obtaining a logical validity score. This can be achieved through various methods.

[0076] One alternative approach is for the scoring model to perform a premise-conclusion comparison on each reasoning step in the predictive thought chain, comparing the conclusion of each reasoning step with the conclusion of the preceding reasoning step and the visual evidence in the sample image to determine whether the conclusion is derived from the premise. Specifically, for reasoning steps in the form of "there are 3 real birds and 4 cartoon birds in the image, therefore there are 7 objects in the scene," the scoring model can check whether the conclusion "there are 7 objects in the scene" is consistent with the number of observable objects in the sample image, and whether it is obtained by summing the number information of "3 real birds" and "4 cartoon birds" stated in the preceding steps. Based on the sample thought chain, the scoring model can refer to the derivation method of the corresponding reasoning step in the sample thought chain to determine whether the derivation path of this step in the prediction thought chain is consistent with the reference path. The scoring model can also refer to the image attributes that are kept consistent by freezing predicates (i.e., the judgment criterion for verifying whether the image attributes are consistent before and after editing, the specific definition of which is described in the following embodiment) during the training data construction stage, as a reference for judging whether the visual evidence is sufficient. Steps whose conclusions cannot be supported by the preceding reasoning steps and visual evidence can be assigned a lower logical validity score.

[0077] Another alternative approach is to organize the sample thought chain, the prediction thought chain, and the scoring instructions into a scoring prompt, input it into a pre-trained scoring model, and then have the scoring model output the logical validity score for each reasoning step. Specifically, the scoring instructions can specify the requirements of the scoring task, such as "Please evaluate whether the conclusion of each reasoning step is derived from its preceding reasoning steps and the content of the sample image, and give the logical validity score for that step." The scoring prompt can sequentially include the sample image, the sample thought chain, the prediction thought chain, and the scoring instructions. After reading the scoring prompt, the scoring model directly outputs the logical validity score for each reasoning step, or it can output the overall logical validity score for the entire prediction thought chain. Since the pre-trained scoring model inherently has the ability to perform multi-dimensional scoring of the prediction thought chain based on the sample thought chain, the logical validity score can be directly output by the scoring model without the need for additional calculation formulas within the scoring model. This method is simple to organize, allows for the batch construction of unified scoring prompts for multiple prediction thought chains and parallel scoring, improving evaluation efficiency, and facilitates batch execution during training.

[0078] Another alternative approach is for the scoring model to first generate a derivation description for each reasoning step in the predictive thought chain, and then determine the logical validity score for each step based on this description. Specifically, the scoring model can generate a descriptive text for each reasoning step, describing which conclusions from preceding reasoning steps and which visual evidence in the sample image led to the conclusion of that step, and whether the derivation process is coherent and reasonable. For example, the derivation description could state, "The conclusion of this step is obtained by adding the number of real birds obtained in the previous step to the number of cartoon birds; the numbers correspond one-to-one with the objects in the image." Subsequently, the scoring model determines the logical validity score for that step based on the completeness of the derivation chain reflected in the derivation description and the consistency between the premises and the conclusion. The more complete the derivation description and the more sufficient the premise support, the higher the score can be assigned. This method retains interpretable scoring criteria while providing a logical validity score, facilitating subsequent manual review of the scoring quality and helping to pinpoint specific weaknesses in the predictive thought chain. The derivation description can be output and archived along with the score.

[0079] In addition, the results of multiple evaluation methods can be combined to determine the logical validity score. For example, the premise-conclusion comparison results and the score prompt output results can be weighted and integrated to improve the stability of the score.

[0080] In this step, a pre-trained scoring model is used as a benchmark to evaluate whether the conclusions of each reasoning step in the predictive thinking chain are derived from the preceding reasoning steps and visual evidence in the sample images. This yields a logical validity score, allowing for fine-grained independent evaluation of the derivation validity of each reasoning step in the predictive thinking chain. The scoring signal covers every step of the reasoning process. Comparison with the sample thinking chain provides a clear reference object for the evaluation, ensuring the objectivity and consistency of the evaluation results. The evaluation process encourages the reasoning steps to anchor their derivation to visual evidence in the sample images, constraining the model to complete reasoning based on image content rather than linguistic priors. This provides a reliable process-level scoring foundation for subsequent training of the visual reasoning model based on the logical validity score, thereby improving the reliability of the visual reasoning model's reasoning process.

[0081] Step 108: Based on the logical validity score, train the visual reasoning model to be trained to obtain the trained visual reasoning model.

[0082] The trained visual reasoning model is obtained by training the untrained visual reasoning model based on a logical validity score. Specifically, the trained visual reasoning model has the ability to perform step-by-step reasoning based on image evidence. During the reasoning process, the model can progressively use specific content in the image as the basis for deduction, anchoring each reasoning step to visual evidence. The reasoning chain maintains a correspondence with the actual content of the image, and each conclusion of the reasoning process can be corroborated by the image. The entire reasoning chain is traceable. Furthermore, compared with the untrained visual reasoning model, the parameters of the trained visual reasoning model have been updated through a reinforcement learning process, resulting in corresponding changes in the model's reasoning behavior and improved reasoning quality. After training, it can be directly used to perform visual reasoning tasks, such as image-text question answering and mathematical reasoning. For the input image and question, it outputs a reasoning result containing reasoning steps and the final answer, which can be directly used by downstream applications.

[0083] In practical applications, the visual reasoning model to be trained is trained based on logical validity scoring, and the trained visual reasoning model can be obtained through various methods.

[0084] One alternative approach is to directly use the logical validity score as a reward signal during reinforcement learning training on the visual reasoning model. Specifically, during training, sample images and sample questions are input into the visual reasoning model to generate a response containing reasoning steps and the final answer. The scoring model then evaluates the logical validity of the generated response, checking whether the conclusion of each reasoning step is derived from previous reasoning results and image evidence, thus obtaining a logical validity score. Using the logical validity score as a reward signal, the training loss and gradient are calculated based on the score, and the model parameters are updated, making the model more inclined to produce reasoning processes with higher logical validity in subsequent inferences. During parameter updates, regularization constraints can be introduced to control the magnitude of parameter updates to maintain training stability. For example, reinforcement learning algorithms such as group relative policy optimization can be used to implement the update process, and the algorithm details can be configured according to actual needs; for example, the learning rate can be set to 1×10⁻⁶. -6 The batch size is 32. Multiple responses (e.g., 8) are generated for each sample, and scores are calculated and parameters are updated uniformly. The specific values ​​can be adjusted according to the model size and computing resources.

[0085] Another alternative approach is to combine the logical validity score with other reward components to determine a comprehensive training signal, and then train the visual reasoning model based on this comprehensive training signal. Specifically, while obtaining the logical validity score, other reward components can also be acquired, such as answer matching rewards (determined by matching the final answer with the sample answer), format rewards (determined by whether the response has both a thought process and an answer structure), and other evaluation results reflecting reasoning quality. The logical validity score and other reward components are then weighted and combined to obtain the comprehensive training signal. Gradients are then calculated based on this comprehensive training signal to update the model parameters, achieving a balance between multiple optimization objectives. For example, the logical validity score and answer matching reward can be weighted and added together to obtain the comprehensive training signal; alternatively, the format reward can be combined, setting the weights of the answer reward, reasoning process reward, and format reward to 0.5, 0.3, and 0.2 respectively, so that answer correctness, reasoning process quality, and response format all receive optimization signals. The weight values ​​and reward aggregation methods can be implemented conventionally according to the training objectives.

[0086] Another option is to use multi-round iterative training to gradually improve the reasoning quality of the visual reasoning model under training until a preset convergence condition is met, resulting in a fully trained visual reasoning model. Specifically, in each iteration, the model in this round performs reasoning on sample images and sample questions, generating a reasoning response. A scoring model evaluates the logical validity of the reasoning response, obtaining a logical validity score. Then, based on the logical validity score, the parameters of the model in this round are updated, resulting in an updated model, and the next iteration begins. This process of response generation, logical validity scoring, and parameter updating is repeated, allowing the model's reasoning quality to gradually improve with each iteration. When the preset convergence condition is met, iterative training stops, and the model in this round is considered the fully trained visual reasoning model. For example, the convergence condition can be set to stop training when the improvement in the logical validity score for multiple consecutive rounds is less than a preset threshold (e.g., the improvement for three consecutive rounds is less than 0.01). Alternatively, the convergence condition can be set to reach a preset maximum number of iterations, or when the model's reasoning performance on validation samples no longer improves. The specific condition can be set according to actual training needs.

[0087] In addition, training methods can be dynamically adjusted according to the progress of the training phase. For example, in the early stage of training, logical validity scores can be used as the main training signal, and in the middle and later stages of training, other signals can be combined for joint training.

[0088] In this step, the visual reasoning model to be trained is trained based on logical validity scoring. This allows the optimization signals of the training process to directly affect the quality of the reasoning process itself, guiding the model to learn to anchor each step of the deduction to image evidence and reason based on the actual content of the image, rather than relying solely on linguistic priors. This enables the model to gradually master the ability to start from image evidence and make evidence-based deductions step by step in complex visual scenes. The trained visual reasoning model can then produce well-founded, logically coherent reasoning processes and correct results in visual reasoning tasks such as image-text question answering and mathematical reasoning. Since each step in the reasoning process corresponds to image evidence, the model's output reasoning results are traceable and interpretable, and the model's generalization ability in diverse visual scenes is enhanced, providing support for the model's application in real-world complex scenarios.

[0089] In the embodiments of this specification, by acquiring a sample dataset containing sample images, sample questions, sample answers, and sample thought chains, a complete training sample can be provided for model training. The sample thought chains provide a reference benchmark for evaluating the reasoning process. By using the visual reasoning model to be trained to reason based on the sample images and sample questions and generate predicted answers and corresponding predicted thought chains, the model's reasoning process is explicitly output, observable, and evaluable. By using a pre-trained scoring model, based on the sample thought chains, the conclusions of each reasoning step in the predicted thought chain are evaluated to determine whether they are derived from the preceding reasoning steps and visual evidence in the sample images, thus obtaining a logical validity score. This allows for fine-grained evaluation of the derivation validity of the reasoning steps, prompting each reasoning step to anchor itself to visual evidence in the images and avoiding the model relying on prior language guesses for answers. By training the visual reasoning model to be trained based on the logical validity score, the training reward directly affects the quality of the reasoning process, improving the reliability and accuracy of the model's reasoning in complex visual scenes, thereby improving the model's training effect and making the reasoning of the trained visual reasoning model more accurate.

[0090] In an optional embodiment of this specification, before training the visual reasoning model to be trained based on logical validity scoring to obtain the trained visual reasoning model, the method further includes: Using a scoring model, based on the semantic matching results between the reasoning steps in the sample thinking chain and the predicted thinking chain, the coverage of the predicted thinking chain to each reference reasoning step in the sample thinking chain is determined, and a reasoning integrity score is obtained. Using a scoring model, a process consistency score is obtained by scoring based on whether there is a logical contradiction between adjacent reasoning steps in the predictive thought chain. Based on logical validity scoring, the visual reasoning model to be trained is trained to obtain a fully trained visual reasoning model, including: The logical validity score, reasoning integrity score, and process consistency score are weighted and aggregated to obtain a reasoning process reward. Based on the reward for the reasoning process, the visual reasoning model to be trained is trained to obtain a fully trained visual reasoning model.

[0091] Semantic matching results represent the semantic correspondence between reasoning steps in the sample thought chain and reasoning steps in the prediction thought chain. Specifically, semantic matching results can include a measure of the semantic similarity between two reasoning steps, such as a semantic similarity score, or a matching result indicating whether the two reasoning steps express the same reasoning content. Superficial differences in expression can be ignored during semantic matching; the same reasoning content expressed in different terms or sentence structures can still be considered a semantic match. Furthermore, semantic matching results are used to determine the extent to which the prediction thought chain covers the sample thought chain. The reference reasoning steps in the sample thought chain are semantically compared one by one with the reasoning steps in the prediction thought chain, providing a basis for calculating the reasoning completeness score.

[0092] Reference reasoning steps are the reasoning steps in the sample thinking chain that serve as a benchmark for comparison. Specifically, the sample thinking chain can be divided into multiple reference reasoning steps, each corresponding to a semantically complete reasoning stage, such as an overall observation step, a key element identification step, or a task-oriented reasoning step. The reference reasoning steps are arranged in order within the sample thinking chain, forming a reference system for evaluating the completeness of the predictive thinking chain. Furthermore, the number of reference reasoning steps can be flexibly determined based on the content of the sample thinking chain, and different sample thinking chains may correspond to different numbers of reference reasoning steps. During comparative evaluation, the sequential order of the reference reasoning steps is maintained, and the reasoning steps in the predictive thinking chain are matched with the reference reasoning steps in their original order to ensure consistency in the reasoning order.

[0093] Coverage is a quantitative measure of how well the predictive thought chain covers the sample thought chain. Specifically, coverage reflects how well the reasoning steps in the predictive thought chain cover each reference reasoning step in the sample thought chain. Higher coverage indicates that the predictive thought chain covers more reference reasoning steps and the reasoning process is more complete. Coverage can be calculated based on semantic matching results. For example, the number of reasoning steps that semantically match the reference reasoning steps can be counted, and the ratio of the number of matches to the total number of reference reasoning steps can be calculated. Alternatively, the semantic similarity of the matching step pairs can be averaged to obtain the average coverage. Furthermore, unmatched reference reasoning steps can be scored as zero in the coverage calculation. Coverage can be normalized to a preset value range, such as between 0 and 1, to facilitate aggregation and comparison with scores from other dimensions.

[0094] The reasoning completeness score is a measure of how well the predicted thought chain covers the sample thought chain. Specifically, the reasoning completeness score is determined based on coverage; higher coverage results in a higher score. The score can be within a preset range, such as 0 to 1. A value of 1 indicates that the predicted thought chain fully covers all reference reasoning steps in the sample thought chain, while a value of 0 indicates that no reference reasoning steps are covered. The scoring model can determine coverage based on semantic matching results to obtain the reasoning completeness score. Furthermore, the reasoning completeness score measures the completeness of the predicted thought chain in the reasoning process. When key reasoning steps are omitted, the score decreases accordingly, encouraging the model to cover all stages of observation, identification, reasoning, and conclusion during reasoning.

[0095] The reasoning completeness score, also known as the completeness score, is used to measure the extent to which the predictive thinking chain covers each reference reasoning step in the sample thinking chain. It is a scoring dimension in the three-dimensional scoring system of the reasoning process, alongside the logical validity score and the process consistency score. The three dimensions evaluate the reasoning process from multiple perspectives, namely, the derivation validity of the reasoning steps, the complete coverage of the reasoning process, and the connection between the preceding and following parts of the reasoning chain.

[0096] Logical contradictions occur when reasoning steps conflict in content or deductive relationships. Specifically, logical contradictions can exist between adjacent reasoning steps. For example, one reasoning step states that the quantity of a certain object is 3, while the next step states that the quantity of the same object is 4, or the conclusion of the previous step is directly contradicted by the conclusion of the next step. Circular reasoning is also a form of logical contradiction; for example, two reasoning steps mutually support each other, forming a loop that fails to provide effective support for the conclusion. Furthermore, logical contradictions can also manifest as conflicts between reasoning steps and visual evidence. When the visual content stated in a reasoning step is inconsistent with the actual content in the sample image, a contradiction can also be identified. The existence of logical contradictions between adjacent reasoning steps can serve as a criterion for assessing process consistency.

[0097] The process consistency score is a rating that characterizes the degree of coherence between adjacent reasoning steps in a predictive thought process. Specifically, the process consistency score is determined based on the results of checking for logical contradictions between adjacent reasoning steps. A higher score indicates no logical contradictions and a natural progression between adjacent reasoning steps; a lower score indicates logical contradictions or circular reasoning. The scoring model examines the logical relationships between each pair of adjacent reasoning steps and summarizes the results into a process consistency score. Furthermore, the process consistency score can take values ​​within a preset range, such as between 0 and 1. A higher score indicates smooth transitions and a coherent reasoning chain, while a lower score indicates contradictions or circular reasoning within the reasoning process.

[0098] The reasoning process reward is a comprehensive reward value determined based on logical validity, reasoning completeness, and process consistency scores, used to characterize the overall quality of the model's reasoning process. Specifically, the reasoning process reward is obtained by weighted aggregation of the logical validity, reasoning completeness, and process consistency scores. These three dimensions measure the derivation validity of the reasoning steps, the completeness of the reasoning process, and the consistency of its connections, respectively. The weights for each dimension during weighted aggregation can be set according to the training objective; for example, logical validity weight 0.4, reasoning completeness weight 0.3, and process consistency weight 0.3, with each weight being non-negative and their sum being a fixed value. The reasoning process reward can take values ​​within a preset range, such as between 0 and 1. Furthermore, as a process-level reward signal in reinforcement learning training, the reasoning process reward guides the model to improve the logical quality of its reasoning process. A higher reasoning process reward value indicates that the model's generated reasoning process is superior in terms of logical derivation, complete coverage, and consistency.

[0099] In practical applications, a scoring model can be used to determine the coverage of the predictive thinking chain to each reference reasoning step in the sample thinking chain based on the semantic matching results between the reasoning steps in the sample thinking chain and the predictive thinking chain, thereby obtaining a reasoning integrity score. This can be achieved through various methods.

[0100] One alternative approach is to determine the matching relationship between reference inference steps and inference steps in the predictive thought chain through optimal alignment, and then calculate the coverage based on the matching relationship. Specifically, a mapping is constructed from each reference inference step in the sample thought chain to each inference step in the predictive thought chain. The mapping needs to satisfy the constraints of monotonicity, partiality, and injectivity, that is, the corresponding order cannot be reversed, one predictive inference step can only correspond to one reference inference step, the same step cannot repeatedly claim multiple reference inference steps, and reference inference steps can also not participate in the mapping. Among the mappings that satisfy the constraints, the mapping that maximizes the sum of semantic similarities between the mapped step pairs is selected as the optimal alignment result. For example, the sum of semantic similarities of all mapped step pairs under each candidate mapping can be calculated, and the mapping with the largest sum of similarities is determined as the optimal alignment. Then, the average of the matching scores of each reference inference step under the optimal alignment is calculated as the average coverage to obtain the inference completeness score.

[0101] Another alternative approach is to semantically match each reference reasoning step in the sample thought chain with the reasoning steps in the prediction thought chain, and then calculate the coverage based on the matching results. Specifically, a reference reasoning step is selected sequentially, and semantically matching reasoning steps are searched in the prediction thought chain. For example, the semantic similarity between the reference reasoning step and each reasoning step in the prediction thought chain can be calculated. Reasoning steps with semantic similarity meeting a preset threshold are identified as matching steps, or the reasoning step with the highest semantic similarity is identified as the matching step. The order of the reference reasoning steps is maintained during matching, and the order is not reversed. For successfully matched reference reasoning steps, the semantic similarity of the corresponding reasoning step is included in the coverage. Reference reasoning steps that do not match are scored as zero. Finally, the scores of all reference reasoning steps are summed and averaged to obtain the reasoning completeness score.

[0102] Another alternative approach is to organize the sample thought chain, the predicted thought chain, and the scoring instruction into a scoring prompt, input it into a pre-trained scoring model, and have the model directly output the reasoning completeness score. Specifically, the scoring instruction can specify the requirements of the evaluation task, such as "Please evaluate the coverage of the predicted thought chain to each reference reasoning step in the sample thought chain and provide a reasoning completeness score." The scoring prompt can sequentially contain the sample thought chain, the predicted thought chain, and the scoring instruction. After reading the scoring prompt, the scoring model, based on its understanding of the semantic correspondence between the reasoning steps in the two thought chains, directly outputs the reasoning completeness score without requiring additional matching and statistical processes outside the scoring model. Alternatively, the scoring model can first provide the matching status of each reference reasoning step and then summarize and output the reasoning completeness score. This method facilitates the batch construction of unified scoring prompts and parallel scoring, making it suitable for scenarios involving large-scale scoring during training.

[0103] In addition, the reasoning integrity score output by the scoring model can be calibrated by combining the results of manual spot checks, so as to improve the consistency between the score and human judgment.

[0104] In practical applications, a scoring model can be used to score the process consistency based on whether there are logical contradictions between adjacent reasoning steps in the predictive thought chain. This can also be achieved through various methods.

[0105] One possible approach is to use a scoring model to check the logical relationships between adjacent reasoning steps in the predictive thought chain, one by one, and summarize the results into a process consistency score. Specifically, two adjacent reasoning steps are selected sequentially according to the reasoning order. The model checks for conflicts between the conclusion of the preceding step and the premises and conclusion of the following step. For example, it checks whether the quantity, location, attributes, etc., stated in the following step are consistent with the conclusion of the preceding step. It also checks whether there is a circular argument between the two steps, i.e., the conclusion of the following step is used as a premise for the preceding step. Simultaneously, it checks whether there is a natural progression between adjacent steps, such as whether the following step further advances the reasoning based on the conclusion of the preceding step. Higher scores are assigned to adjacent step pairs that do not have logical contradictions and are naturally progressive, while lower scores are assigned to adjacent step pairs that have logical contradictions or circular arguments. The scores of all adjacent step pairs are then summarized, for example, by taking the average, to obtain the overall process consistency score of the predictive thought chain.

[0106] Another alternative approach is to organize the predictive thought chain and scoring instructions into a scoring prompt, input it into a pre-trained scoring model, and have the model directly output a process consistency score. Specifically, the scoring instructions can specify the requirements for process consistency evaluation, such as "Please check for logical contradictions or circular arguments between adjacent reasoning steps in the predictive thought chain, and evaluate whether each step progresses naturally, providing a process consistency score." After reading the scoring prompt, the scoring model checks the logical relationships between adjacent reasoning steps one by one, summarizes the results into a process consistency score, and outputs it directly. No additional calculation formulas are needed within the scoring model. Alternatively, sample thought chains can be included in the scoring prompt, allowing the scoring model to refer to the progression of sample thought chains to determine whether the connections between steps in the predictive thought chain are natural. This method is simple to implement, facilitates batch scoring, and is suitable for large-scale evaluation scenarios during training.

[0107] In addition, the types of logical contradictions can be further distinguished, such as different types of contradictions such as conclusion conflict, omission of conditions and circular reasoning, and then statistically summarized into a process consistency score.

[0108] Once the reasoning integrity score and process consistency score are obtained, the logical validity score, reasoning integrity score, and process consistency score can be weighted and aggregated to obtain a reasoning process reward.

[0109] One alternative approach is to assign weights to the logical validity score, reasoning completeness score, and process consistency score, and then sum the products of each score and its corresponding weight to obtain the reasoning process reward. Specifically, logical validity weight, reasoning completeness weight, and process consistency weight can be set, with each weight being no less than 0, and the sum of the three weights being a fixed value, such as 1. The values ​​of each weight can be determined based on the training objective; for example, a logical validity weight of 0.4, a reasoning completeness weight of 0.3, and a process consistency weight of 0.3 can be set to maintain a balanced contribution of the three dimensions to the reasoning process reward. The weight values ​​can also be adjusted according to the degree of emphasis. Then, the three scores are multiplied by their corresponding weights, and the three products are summed to obtain the reasoning process reward, comprehensively reflecting the overall quality of the predictive thought chain across the three dimensions of logical validity, completeness, and consistency.

[0110] Another alternative approach is to first perform interval verification and normalization on the scores for each dimension, and then perform weighted aggregation to obtain the inference process reward. Specifically, check whether the three scores are in a consistent value range. If the value ranges are inconsistent, first normalize each score to a unified range, and then sum them according to their weights. The weights can be fixed in advance by the training configuration, or they can be dynamically adjusted as the training progresses. For example, in the early stages of training, the weight of reasoning completeness can be relatively high to guide the model to cover complete reasoning steps first. In the later stages of training, the weights of logical validity or process consistency can be appropriately increased to strengthen the rigor of reasoning. During the aggregation process, maintain the constraint that each weight is not less than 0 and the sum of the weights is a fixed value to ensure the stability of the inference process reward. The weighted sum result is determined as the inference process reward and used in subsequent training steps.

[0111] In addition, the scale calibration of the reasoning process reward can be performed based on the weighted aggregation, for example, by standardizing the reasoning process reward according to the historical score distribution to keep the reward value stable.

[0112] If a reward is obtained for the reasoning process, the visual reasoning model to be trained can be trained based on the reasoning process reward to obtain a trained visual reasoning model.

[0113] One alternative approach is to use the reasoning process reward as a reward signal during reinforcement learning training to train the visual reasoning model. Specifically, during training, sample images and sample questions are input into the visual reasoning model to generate responses containing predicted thought chains and predicted answers. A scoring model is used to evaluate the predicted thought chains, obtaining logical validity scores, reasoning completeness scores, and process consistency scores. These three scores are then weighted and aggregated to obtain the reasoning process reward. Using this reward as a reward signal, the training loss and gradient are calculated to update the parameters of the visual reasoning model, making the model more inclined to generate higher-quality reasoning processes in subsequent inferences. Parameter updates can be implemented using reinforcement learning algorithms such as group relative policy optimization. For example, multiple responses are generated for the same batch of samples, and the normalized dominance value within the group is calculated based on the reasoning process reward of each response before updating the parameters. Regular constraints are introduced to control the magnitude of parameter updates, and the specific parameters can be configured according to the model size and training requirements.

[0114] Another alternative approach is to employ a multi-round iterative method, training the visual reasoning model based on inference process rewards until a preset convergence condition is met, resulting in a fully trained visual reasoning model. Specifically, in each iteration, a predictive thought chain is generated based on the current model. A scoring model is used to evaluate the logic validity score, reasoning completeness score, and process consistency score. These scores are then weighted and aggregated to obtain the inference process reward, and the model's parameters are updated based on this reward. After the update, the next iteration begins, repeating the response generation, scoring evaluation, reward aggregation, and parameter updates, gradually improving the model's inference quality with each iteration. Training stops when a preset convergence condition is met. For example, the convergence condition could be set to ensure that the increase in inference process rewards for multiple consecutive rounds is less than a preset threshold, or it could be set to reach a preset maximum number of iterations. Once the convergence condition is met, the current model is considered the fully trained visual reasoning model.

[0115] In addition, the reward for the reasoning process can be compared with the reference reward output by the benchmark model and used for training to suppress the effect of reward scale drift.

[0116] In this embodiment, based on the logical validity score, a scoring model is further used to determine the reasoning completeness score based on the semantic matching results between the reasoning steps in the sample thinking chain and the predicted thinking chain. A process consistency score is determined based on whether there are logical contradictions between adjacent reasoning steps in the predicted thinking chain. This expands the evaluation of the reasoning process from a single dimension to a multi-dimensional evaluation covering logical validity, reasoning completeness, and process consistency. Clear quantitative signals are obtained regarding the completeness of reasoning steps and the coherence of reasoning links. The three scores are weighted and aggregated to obtain a reasoning process reward, integrating the overall quality of the reasoning process into a unified training signal. This constrains the model's reasoning behavior from multiple perspectives—logic, completeness, and consistency—guiding the model to ensure that each derivation is based on evidence and that the reasoning links are fully covered and naturally connected. The overall quality of the reasoning chain is comprehensively improved, providing stronger training support for enhancing the reasoning ability of the visual reasoning model.

[0117] In one optional embodiment of this specification, before training the visual reasoning model to be trained based on the reasoning process reward to obtain the trained visual reasoning model, the method further includes: The answer reward is determined based on the predicted answer and the sample answer, and the format reward is determined based on the predicted answer and the predicted thought chain. Based on the reward for the reasoning process, the visual reasoning model to be trained is trained to obtain a trained visual reasoning model, including: The total reward is determined based on the answer reward, reasoning process reward, and format reward. The visual reasoning model to be trained is then trained based on the total reward to obtain the trained visual reasoning model.

[0118] The answer reward is a value determined based on the matching between the predicted answer and the sample answer. Specifically, the answer reward represents the degree of consistency between the predicted answer and the sample answer. A higher reward value is assigned when the predicted answer and sample answer are semantically equivalent, and a lower reward value is assigned when they do not match. The answer reward can be in a binary form, for example, setting the reward to 1 when the predicted answer matches the sample answer and 0 when they do not match, or it can be in a continuous value form, giving a score within a range based on the degree of matching. When matching the predicted answer and the sample answer, superficial differences such as numerical format and capitalization can be ignored, and a semantic equivalence judgment method can be used to determine whether the two match. Furthermore, as a reward component for measuring the correctness of the model's answers during training, the answer reward provides the model with explicit feedback on correctness. During training, the model tends to generate predicted answers that match the sample answers, thereby improving the accuracy of the model's answers.

[0119] The format reward is a reward value determined based on the response structure of the predicted answer and the predicted thought chain. Specifically, the format reward characterizes the compliance of the model response in terms of organizational form. When the model response includes both the thought chain and the answer, the format reward can take a higher value; when the response lacks either the thought chain or the answer, the format reward can take a lower value. The format reward can be in a binary form; for example, it can be set to 1 if the response includes both the thought chain and the answer, and 0 otherwise. Determining whether a response contains two parts can be based on the presence of a marked answer closing format in the response, or on the parsing results of the response content. Furthermore, the format reward guides the model to organize its output according to the structure of thought chain plus answer, clearly separating the reasoning process from the final answer in the response, thus facilitating the extraction of the predicted thought chain and the predicted answer from the response.

[0120] The total reward is a comprehensive reward value determined based on the answer reward, reasoning process reward, and format reward. Specifically, the total reward characterizes the overall performance of the model response in three aspects: answer correctness, reasoning process quality, and response format. A higher total reward indicates better overall performance of the model response in these three aspects. The total reward can be determined by weighting the answer reward, reasoning process reward, and format reward according to preset weights. The weights corresponding to each reward component can be set according to the training objective to differentiate the contributions of different reward components to the total reward. The weight settings can satisfy preset magnitude relationships; for example, the weight corresponding to the answer reward can be set to be the largest to highlight the dominant role of answer correctness in training. Furthermore, the total reward can serve as a reward signal in the reinforcement learning training process. Based on the total reward, the training loss and gradient are calculated, and the model parameters are updated, enabling the model to simultaneously optimize multiple objectives such as answer correctness, reasoning process quality, and response format during training.

[0121] In practical applications, determining the answer reward based on the predicted answer and the sample answer can be achieved through various methods.

[0122] In an optional method, semantic equivalence matching can be performed between a predicted answer and a sample answer, and a binarized answer reward is determined based on the matching result. Specifically, it is compared whether the predicted answer and the sample answer are semantically equivalent. For example, for numerical answers, the numerical values are compared for consistency; for categorical answers, the categorical judgments are compared for consistency; for text answers, it is compared whether the semantic expressions point to the same result. When performing semantic equivalence judgment, surface differences such as numerical format and case can be ignored. For example, the predicted answer "4" and the sample answer "four" can be judged as semantically equivalent, and the predicted answer "red" and the sample answer "red" (Chinese character "红") can be judged as semantically equivalent. When the predicted answer and the sample answer are semantically equivalent, the answer reward is set to 1, and when they are not semantically equivalent, the answer reward is set to 0, thereby obtaining a binarized answer reward. This method is simple and clear in implementation, the answer reward directly reflects the correctness of the model's answer, and provides a clear correctness feedback signal for training.

[0123] In another optional method, a scoring model can be used to perform matching judgment on the predicted answer and the sample answer, and the answer reward is determined based on the judgment result. Specifically, the predicted answer and the sample answer are input into the scoring model, and the scoring model judges whether the two are semantically equivalent. The scoring model can output a judgment result of equivalence or mismatch, and can also output a score of matching degree. The answer reward is determined based on the judgment result. For example, when it is judged that the predicted answer and the sample answer are semantically equivalent, the answer reward is 1, and when it is judged that they do not match, the answer reward is 0. The matching degree score can also be mapped to a continuous reward value within an interval, so that partially correct predicted answers can obtain progressive reward feedback. Matching judgment takes semantic equivalence as the standard, and surface differences such as numerical format and case can be ignored. This method leverages the language understanding ability of the scoring model to process answers with various expression forms, and is suitable for training scenarios where answer expression methods are flexible.

[0124] In addition, a continuous value method can also be used to determine the answer reward. For example, the value is taken within a preset range according to the similarity between the predicted answer and the sample answer, and the higher the similarity, the larger the reward value.

[0125] In practical applications, determining the format reward based on the predicted answer and the predicted thought chain can also be implemented through multiple methods.

[0126] One alternative approach is to perform a structure check on the model response to determine whether it contains both a predicted thought process and a predicted answer. Based on this check, a binarized format reward is determined. Specifically, the model response, consisting of the predicted thought process and the predicted answer, is used as the inspection object. The check examines whether the response contains a thought process and an answer component. For example, it checks whether the response includes text describing the reasoning process and whether it concludes with an answer. If the response contains both a thought process and an answer, the format reward is set to 1; if either component is missing, the format reward is set to 0. Alternatively, a format reward value within a range can be given based on the completeness of the response structure. This method uses the structure requirement of a thought process plus an answer as the criterion to guide the model to output a standardized response containing a complete reasoning process and a clear answer.

[0127] Another alternative approach is to utilize fixed-marker parsing of the model response within the predictive thought chain, determining the formatted reward based on the parsing results. Specifically, the predictive thought chain can employ a fixed-marker ending format, such as outputting a "Final Answer:" marker at the end of the inference text, or using a different format. <answer> Answer< / answer> The formatted marking clearly distinguishes the predicted answer from the reasoning process. When parsing the model's response, it checks whether the aforementioned fixed marking exists in the response. If a fixed marking exists, the content corresponding to the marking is determined as the predicted answer, and the content before the marking is determined as the predicted thought chain. In this case, it can be determined that the response contains both the thought chain and the answer, and the format reward is set to 1. If the response does not contain a fixed marking, or if the thought chain and the answer cannot be parsed from the response, the format reward is set to 0. This method has clear parsing rules, stable and reliable determination results for the format reward, and facilitates the extraction of predicted answers for answer reward calculation during training.

[0128] In addition, the value of the format reward can be refined based on the completeness of the thought chain and answer in the response. For example, a higher value can be taken when both parts are complete, and a lower value can be taken when only part is complete.

[0129] Once the answer reward and format reward are determined, the total reward can be determined based on the answer reward, reasoning process reward, and format reward. The visual reasoning model to be trained can then be trained based on the total reward to obtain the trained visual reasoning model.

[0130] One alternative approach is to assign weights to the answer reward, reasoning process reward, and format reward, and then sum the products of each reward component and its corresponding weight to obtain the total reward. The visual reasoning model to be trained is then trained based on this total reward. Specifically, weights can be set for the answer reward, the reasoning process reward, and the format reward. The values ​​of each weight can be determined according to the training objective. For example, the weights can be set to satisfy a preset size relationship to differentiate the contributions of answer correctness, reasoning process quality, and response format to the total reward, while also satisfying normalization constraints to keep the total reward value stable. The three reward components are multiplied by their corresponding weights, and the three products are summed to obtain the total reward. Reinforcement learning training is then performed based on the total reward, using it as a reward signal to calculate the training loss and gradient, updating the parameters of the visual reasoning model to be trained. This allows the model to comprehensively optimize answer correctness, reasoning process quality, and response format during training, resulting in a trained visual reasoning model.

[0131] Another alternative approach is to first perform interval verification and normalization on each reward component, then perform weighted summation to determine the total reward, and train the model based on the total reward using a multi-round iterative method. Specifically, check the value ranges of the answer reward, reasoning process reward, and format reward. If the value ranges are inconsistent, first normalize each reward component to a unified range, for example, normalize them all to between 0 and 1, and then perform weighted summation according to preset weights to obtain the total reward. During training, update the parameters of the visual reasoning model to be trained round by round based on the total reward. For example, reinforcement learning algorithms such as group relative policy optimization can be used to generate multiple responses for the same set of samples, calculate the total reward for each response, determine the update direction based on the reward normalized within the group, and update the model parameters. Repeat the response generation, reward calculation, and parameter update process until the preset convergence condition is met, and determine the converged model as the completed visual reasoning model.

[0132] Another option is to combine answer rewards and format rewards sequentially to determine the total reward based on the reasoning process reward, and then train the model based on the total reward. Specifically, one can first obtain the reasoning process reward, and then combine the answer reward, reasoning process reward, and format reward according to their weights to obtain the total reward. For example, one can first add the answer reward and reasoning process reward according to their corresponding weights to obtain the intermediate reward, and then add the intermediate reward and format reward according to their corresponding weights to obtain the total reward. Alternatively, different reward components can be used according to the training stage. For example, in the early stage of training, the total reward can be mainly determined based on the answer reward and format reward, so that the model first masters the standard answering method. In the middle and later stages of training, the reasoning process reward can be included in the total reward to strengthen the guidance of the reasoning process. Then, the visual reasoning model to be trained can be trained based on the total reward, the model parameters can be updated, and the model can gradually converge to obtain the trained visual reasoning model.

[0133] In addition, the reward combination can be dynamically adjusted in conjunction with the training phase. For example, in the early stage of training, the focus can be on answer rewards and format rewards, while in the middle and later stages of training, the proportion of rewards for the reasoning process can be gradually increased.

[0134] In the embodiments of this specification, answer rewards are determined based on predicted answers and sample answers, and format rewards are determined based on predicted answers and predicted thought chains. This allows the correctness of the model's response and the standardization of its structure to receive quantitative feedback signals. Answer rewards directly reflect the correctness of the model's answer, while format rewards constrain the model to organize its output according to the structure of thought chain plus answer. By determining the total reward based on answer rewards, reasoning process rewards, and format rewards, the rewards of the three dimensions are integrated into a unified training signal. The visual reasoning model to be trained is trained based on the total reward, so that the correctness of the answer, the quality of the reasoning process, and the format of the response are synergistically optimized during training. The model pursues correct answers while also considering the quality of the reasoning process and the standardization of the output. The training signal is more comprehensive than a single reward, which helps the trained visual reasoning model to simultaneously possess the ability to answer accurately, reason with evidence, and output in a standard way in visual reasoning tasks, thereby improving the overall reasoning performance.

[0135] In one optional embodiment of this specification, the total reward is determined based on the answer reward, the reasoning process reward, and the format reward, including: Determine the answer weight corresponding to the answer reward, the reasoning process weight corresponding to the reasoning process reward, and the format weight corresponding to the format reward, wherein the answer weight is greater than the reasoning process weight, and the reasoning process weight is greater than the format weight; Based on the answer weight, reasoning process weight, and format weight, the answer reward, reasoning process reward, and format reward are weighted and summed to obtain the total reward.

[0136] Answer weight is the weighting coefficient corresponding to the answer reward in the total reward composition, used to characterize the relative importance of the final answer's correctness in the training objective. Specifically, the answer weight can be set to the maximum value among the three weights, so that the answer reward occupies the largest share in the weighted summation. For example, the answer weight can be set to a value such as 0.5 or 0.6. Furthermore, the value of the answer weight can be flexibly determined under the constraint that it is greater than the weight of the inference process and that the sum of the three weights is a preset fixed value (e.g., the sum is 1). Different configurations can be adopted for different training stages or different model sizes. For example, the answer weight can be appropriately increased in the early stage of training to strengthen the model's emphasis on answer correctness.

[0137] The inference process weight is a weighted coefficient corresponding to the inference process reward in the total reward composition, used to characterize the relative importance of the inference process quality in the training objective. Specifically, the inference process weight can be set to a value greater than the format weight and less than the answer weight, so that the inference process reward occupies a moderate share in the weighted summation, and the inference quality obtains a meaningful gradient signal. For example, the inference process weight can be set to a value of 0.3. Furthermore, the value of the inference process weight can be adjusted under the premise of satisfying the relationship with the answer weight, format weight, and summation constraints. For example, when the model's correct answer rate is high, the inference process weight can be appropriately increased, so that the optimization focus is shifted towards the inference process quality.

[0138] Format weight is the weighting coefficient corresponding to the format reward in the total reward composition, used to characterize the relative importance of response format compliance in the training objective. Specifically, the format weight can be set to the minimum value among the three types of weights, so that the impact of determining whether the response has both a thought chain and an answer structure on the total reward is relatively limited. For example, the format weight can be set to a value of 0.2 or 0.1. Furthermore, the value of the format weight can be flexibly adjusted under the constraint that it is less than the inference process weight and the sum of the three types of weights is a preset fixed value, or it can be dynamically set according to the training stage. For example, the format weight can be appropriately increased in the early stage of training and decreased after the output structure stabilizes.

[0139] In practical applications, determining the answer weight corresponding to the answer reward, the reasoning process weight corresponding to the reasoning process reward, and the format weight corresponding to the format reward can be achieved through various methods.

[0140] One alternative method is to directly set the values ​​of the three types of weights under preset constraints. Specifically, the relative magnitudes of the three types of weights can be determined first, i.e., the answer weight is the largest, the reasoning process weight is in the middle, and the format weight is the smallest. Then, under the premise of satisfying this magnitude relationship and the sum of the three types of weights being a preset fixed value (e.g., a sum of 1), values ​​are assigned to the three types of weights respectively. For example, the answer weight, reasoning process weight, and format weight can be set to 0.5, 0.3, and 0.2 respectively, or they can be set to 0.6, 0.3, and 0.1 respectively, or other combinations of values ​​that satisfy the constraints can be used. Once the weight values ​​are determined, they can remain unchanged throughout the training process. This method is simple to implement, the meaning of the weights is clear, and it is easy to interpret and reproduce the training process.

[0141] Another alternative approach is to dynamically determine the three weights based on the training phase and model performance. Specifically, an initial set of weights satisfying the size relationship and summation constraints can be set first. During training, the weight values ​​are adjusted based on the model's performance on validation samples. For example, when the model's correct answer rate is low, a larger answer weight is maintained, focusing the optimization on answer correctness. When the model's correct answer rate is high but the quality of the inference process improves slowly, the inference process weight can be appropriately increased while the format weight is correspondingly decreased, allowing for a more sufficient gradient in inference quality. Each adjustment maintains the size relationship and summation of the three weights. This method balances the optimization emphasis on answer correctness and inference quality.

[0142] In addition, the three weights can be adjusted based on the model's performance feedback on the validation set. For example, the weight of the inference process can be appropriately increased when the quality of the model's inference process improves slowly.

[0143] Once the answer weight, reasoning process weight, and format weight are determined, the answer reward, reasoning process reward, and format reward can be weighted and summed based on these weights to obtain the total reward.

[0144] One alternative method is to multiply the three types of rewards by their respective weights and then sum them to obtain the total reward. Specifically, the answer reward can be multiplied by its weight, the reasoning process reward by its weight, and the format reward by its weight. These three products are then summed to obtain the total reward. For example, with answer weight, reasoning process weight, and format weight of 0.5, 0.3, and 0.2 respectively, the total reward equals 0.5 times the answer reward plus 0.3 times the reasoning process reward plus 0.2 times the format reward. Since the answer weight is the largest, the correctness of the answer consistently dominates the contribution to the total reward. This method is simple to calculate and facilitates batch weighted summation of candidate responses during training.

[0145] Another alternative approach is to add a constraint term to the weighted summation to obtain the total reward. Specifically, a weighted summation of the three types of rewards can be performed first to obtain a basic aggregation result. Then, an adjustment term can be added. For example, if the predicted answer does not match the sample answer but the inference process reward is high, a penalty can be imposed on the basic aggregation result, preventing the model from obtaining a high total reward by relying on a formally complete but flawed inference process. Alternatively, the weighted summation result can be limited to a preset range to prevent the total reward of individual candidate responses from deviating excessively from other responses within the group. This approach constrains the behavior of relying on inference process rewards to gain an undue advantage, making the total reward more accurately reflect the true quality of the candidate responses.

[0146] In addition, a nonlinear transformation can be performed on the total reward after weighted summation, such as applying truncation or mapping, to keep the value of the total reward within a preset range.

[0147] In the embodiments of this specification, answer weights, reasoning process weights, and format weights that satisfy preset size relationships are determined for answer rewards, reasoning process rewards, and format rewards, respectively. The total reward is obtained by weighted summation of the three types of rewards. This ensures that the correctness of the final answer always dominates the composition of the total reward, the quality of the reasoning process obtains a continuous and effective gradient signal, and the standardization of the response format is stably guided. The model is unlikely to rely on a reasoning process that is formally complete but has an incorrect conclusion to obtain a high total reward. The total reward can more realistically reflect the overall quality of the candidate response. The weight configuration is flexible and can adapt to the optimization needs of different model sizes and training stages.

[0148] In one optional embodiment of this specification, the visual reasoning model to be trained is trained based on the total reward to obtain a trained visual reasoning model, including: Using a group relative strategy optimization algorithm, a set of candidate responses is generated based on sample images and sample questions, using the visual reasoning model to be trained; Based on the total reward, determine the relative strength of each candidate response within a set of candidate responses; Based on relative advantages, the parameters of the visual reasoning model to be trained are updated to obtain the trained visual reasoning model.

[0149] The group relative policy optimization algorithm is a policy optimization algorithm used for reinforcement learning training. It estimates the relative merits of each candidate response based on a set of candidate responses and updates the model parameters accordingly. Specifically, this algorithm generates multiple candidate responses for each sample image and sample question, calculates the relative advantage of each candidate response using the within-group reward distribution, and does not require a value network or external baseline. For example, the relative advantage can be obtained by normalizing the mean and standard deviation of the total within-group reward. Furthermore, this algorithm can introduce amplitude control and difference constraints in the parameter update stage. For example, a pruned surrogate objective function can be used to limit the amplitude of a single update, or a difference constraint relative to the model before the update can be imposed on the update. This algorithm independently completes the reinforcement learning training process without the need for a separate supervised fine-tuning stage.

[0150] Candidate responses are optional outputs generated by the visual reasoning model to be trained for a given sample image and sample question. Specifically, a candidate response can consist of two parts: a predicted thought chain and a predicted answer. The predicted thought chain describes the model's reasoning process from observing the sample image to arriving at the predicted answer, while the predicted answer is located at the end of the response. Multiple candidate responses can be generated for the same sample image and sample question; for example, eight candidate responses can be generated as a group. Furthermore, a group of candidate responses constitutes the basic unit for calculating relative advantage in the group relative policy optimization algorithm, and the total reward of each candidate response needs to be compared with other responses within the group. The number of candidate responses can be flexibly set according to the model size and computational resources.

[0151] Relative advantage is a quantitative indicator that characterizes the superiority or inferiority of a candidate response relative to other candidate responses within the same group. Specifically, relative advantage can be calculated based on the distribution of total rewards within the group. For example, the total reward of each candidate response can be subtracted from the mean of the total rewards within the group, and then divided by the standard deviation of the total rewards within the group to obtain a normalized relative advantage. Candidate responses with total rewards higher than the group mean have a positive relative advantage, while those with total rewards lower than the group mean have a negative relative advantage. Furthermore, a numerical stability term can be introduced during the calculation to avoid numerical anomalies when the standard deviation is close to 0. Relative advantage is used to weight and update model parameters; the output probability corresponding to a candidate response with a positive relative advantage is increased, while the output probability corresponding to a candidate response with a negative relative advantage is suppressed.

[0152] In practical applications, the group relative strategy optimization algorithm can be used to generate a set of candidate responses based on sample images and sample questions, using the visual reasoning model to be trained. This can be achieved through various methods.

[0153] One alternative approach is to input sample images and sample questions into the visual reasoning model to be trained, and repeatedly generate candidate responses through sampling until a predetermined number of candidate responses are obtained. Specifically, in each sampling process, the model autoregressively generates a complete response based on the input content of the sample image and sample question. The response includes a predicted thought chain and a predicted answer. Repeatedly performing the sampling operation can generate multiple candidate responses for the same sample image and sample question; for example, eight candidate responses can be generated as a group. The diversity between candidate responses can be controlled by adjusting generation parameters such as temperature during sampling. This method directly reuses the model's existing generation capabilities, achieving simplicity.

[0154] Another alternative approach is to use batch processing to generate candidate responses in parallel for multiple sample instances. Specifically, a training batch containing multiple sample instances can be constructed, each instance including a sample image and a corresponding sample question. The sample images and sample questions within the batch are organized into a unified tensor input. The visual reasoning model to be trained generates responses for multiple sample instances at once using batch processing, generating multiple candidate responses for each sample instance. The multiple candidate responses corresponding to the same sample instance are then grouped together, for example, eight candidate responses can be generated for each sample instance. During the generation process, each candidate response is sampled based on the same set of current model parameters, ensuring consistency in comparisons within the group.

[0155] In addition, a curriculum-based sampling strategy can be used to generate candidate responses. For example, a lower decoding temperature can be used to generate stable responses in the early stages of training, and the decoding temperature can be increased in the later stages of training to increase the diversity of candidate responses.

[0156] Given a set of candidate responses, the interface further determines the relative strength of each candidate response within that set.

[0157] One alternative approach is to normalize the total reward of each candidate response using the mean and standard deviation of the total reward within the group to obtain the relative advantage. Specifically, first calculate the mean of the total reward of all candidate responses in a group, then calculate the standard deviation of the total reward for that group. Next, subtract the group mean from the total reward of each candidate response, and then divide by the group standard deviation. The result is the relative advantage of that candidate response. A small positive stabilizing term can be added to the group standard deviation during the calculation to prevent numerical anomalies. Candidate responses with a total reward higher than the group mean have a positive relative advantage, while those with a total reward lower than the group mean have a negative relative advantage. This method ensures that the advantage estimation is not affected by the absolute scale of the reward, maintaining comparability between batches.

[0158] Another alternative approach is to determine the relative advantage of each candidate response based on its ranking within the group. Specifically, a group of candidate responses can be sorted from highest to lowest total reward, and each response can be assigned a corresponding relative advantage based on its ranking. For example, the candidate response with the highest total reward can be given the largest positive relative advantage, the candidate response with the lowest total reward can be given a negative relative advantage, and candidate responses in the middle ranking can be assigned values ​​within a preset range based on their relative position. Alternatively, the initial advantage after normalization can be pruned, compressing values ​​exceeding the preset range to boundary values. This method is insensitive to the shape of the reward distribution within the group and can still stably distinguish the relative merits of candidate responses even when the differences within the group are small.

[0159] In addition, the relative advantage can be weighted and adjusted based on the confidence level of the reward, for example, by appropriately reducing the weight of the advantage estimate for candidate responses with low confidence.

[0160] Once the relative advantage is determined, the parameters of the visual reasoning model to be trained can be updated based on the relative advantage to obtain the trained visual reasoning model.

[0161] One alternative approach is to construct a surrogate objective function based on relative advantage to update the model parameters. Specifically, the ratio of the generation probability of each word in each candidate response to the current model parameters and the model parameters before the update can be calculated as the importance ratio. This importance ratio is then multiplied by the relative advantage of the corresponding candidate response to obtain the optimized signal for that word. The importance ratio is then pruned, for example, by limiting it to a preset pruning range, such as between 0.8 and 1.2. Subsequently, the optimized signals of each word are aggregated to perform gradient updates, increasing the generation probability of outputs related to positively dominant candidate responses and decreasing the generation probability of outputs related to negatively dominant candidate responses. This approach balances optimization efficiency and training stability.

[0162] Another alternative approach is to impose a difference constraint relative to the model before the update during the parameter update process. Specifically, a constraint term can be added to the optimization objective, such as calculating the difference between the output distribution of the model before and after the update under the same input, multiplying this difference by a preset coefficient, and then adding it to the optimization objective to control the update magnitude. Simultaneously, training hyperparameters can be set, for example, a learning rate of 1×102. -6 The batch size is set to 32 or 96 according to the model size. After generating eight candidate responses for each sample instance, the parameters are updated uniformly. The above process is repeated until the preset convergence condition is met, such as the total reward increase is less than the preset threshold for multiple consecutive rounds, or the preset maximum number of training rounds is reached. The converged model is then used as the visual reasoning model that has been trained.

[0163] In addition, learning rate scheduling strategies can be incorporated during parameter updates, such as decaying the learning rate with each training round, to improve the convergence stability of training.

[0164] In the embodiments of this specification, a group of candidate responses is generated based on sample images and sample questions using a group relative policy optimization algorithm. The relative advantage of each candidate response is determined according to the distribution of the total reward within the group, and the parameters of the visual reasoning model to be trained are updated based on the relative advantage. The parameter update signal comes from the relative comparison between candidate responses within the group. The advantage estimation method with intra-group normalization can suppress the influence of reward scale differences. The amplitude control and difference constraint in the parameter update process can prevent the policy update from being too fast. The model gradually increases the probability of generating candidate responses with high total reward, and the correctness of the answer and the quality of the reasoning process are improved simultaneously, thereby obtaining a visual reasoning model with better reasoning performance and stability.

[0165] In one optional embodiment of this specification, a schematic diagram illustrating the execution framework of a model training method is also provided. Specifically, see [link to documentation]. Figure 2 , Figure 2 This diagram illustrates an execution framework for a model training method according to an embodiment of this specification. Figure 2 As shown.

[0166] Figure 2 The left side of the diagram illustrates the scoring criterion construction process. Sample images, sample questions, sample answers, and sample thought chains from the sample dataset are input into the visual reasoning model. The visual reasoning model outputs a predicted answer and a predicted thought chain. The scoring model evaluates each step of the reasoning in the predicted thought chain, determining a logical validity score, a reasoning completeness score, and a process consistency score along three dimensions: logical validity, reasoning completeness, and process consistency. The logical validity score measures whether the reasoning steps are based on visual evidence and are logically rigorous, focusing on perceptual and illusory errors. The reasoning completeness score measures whether all necessary intermediate steps are included to reach the final answer, focusing on missing or skipped reasoning. The process consistency score measures whether the reasoning steps are consistent and without contradictions, focusing on accumulated reasoning errors. The scores of these three dimensions together constitute the scoring criteria for the reasoning process reward.

[0167] Figure 2 The right side of the diagram illustrates the weighted aggregation process for rewards. The total reward is a weighted aggregation of the answer reward, reasoning process reward, and format reward. Specifically, the total reward equals the sum of the format reward multiplied by its weight, the reasoning process reward multiplied by its weight, and the answer reward multiplied by its weight. The reasoning process reward is equal to the sum of the logical validity score multiplied by its weight, the reasoning completeness score multiplied by its weight, and the process consistency score multiplied by its weight. The format reward measures whether the response output format is well-structured, while the answer reward measures whether the predicted answer is correct. The sum of all weights is a fixed value, with the answer reward having the largest weight, followed by the reasoning process reward, and then the format reward having the smallest weight, indicating that the correctness of the predicted answer is the most important factor.

[0168] In the embodiments of this specification, by means of Figure 2 The execution framework shown expands the quality assessment of the reasoning process from a single dimension to three dimensions: logical validity, reasoning completeness, and process consistency. It also combines answer rewards and format rewards for weighted aggregation, so that the training signal simultaneously covers reasoning quality, answer correctness, and output standardization. The constraint of having the largest weight for answer rewards ensures that the correctness of the final answer is always the dominant goal of training. This improves the reliability of the reasoning process while preventing the model from cheating the process rewards by outputting formatted but erroneous results.

[0169] In an optional embodiment of this specification, the method further includes: before obtaining the sample dataset: Obtain the initial dataset, which includes the initial image, initial question, and initial answer; Based on the change target of the initial answer, the initial image is edited to obtain the edited image; The answer is retrieved based on the initial question and the edited image; If the read answers meet the change objectives, a sample dataset is determined based on the edited image, the initial question, and the read answers.

[0170] The initial dataset is the original data set used to construct the sample dataset, consisting of a set of triplets comprising an initial image, an initial question, and an initial answer. Specifically, each data point in the initial dataset may include an initial image, an initial question, and the corresponding initial answer; these three elements correspond to each other to form a visual question-and-answer instance. The sample dataset can be obtained by editing, reading, and validating the initial data in the initial dataset. Furthermore, the source of the initial dataset can be flexibly chosen; for example, it can come from manually collected real photos and their corresponding question-and-answer pairs, or from publicly available visual question-and-answer datasets or existing question-and-answer records. The size can be set according to the number of target samples and available computing resources.

[0171] The initial image is the original image in the initial dataset that corresponds to the initial question. It is used to present the scene content required for visual reasoning and is also the object of answer rewriting and editing. Specifically, the initial image can include real scene photographs, image samples from public datasets, etc. The image contains visual elements related to the content tested in the question, such as objects, quantities, locations, and attributes. For example, it can include a scene photograph containing multiple birds. The visual content in the initial image corresponds to the initial question, and the initial answer is determined based on the visual content in the initial image. Furthermore, the clarity and resolution of the initial image can be set according to actual needs. Visual elements need to be clearly distinguishable to facilitate accurate execution of editing operations and answer reading.

[0172] The initial questions are the question texts corresponding to the initial images in the initial dataset, used to describe the visual reasoning task that requires answers based on the content of the initial images. Specifically, the initial questions can include visual reasoning questions of various types, such as selection, numerical calculation, symbolic reasoning, illustration interpretation, chart reading, spatial reasoning, process analysis, and comparison judgment. For example, for a scene image containing real birds and cartoon birds, the initial question could be set as "The image includes real birds and cartoon birds, how many birds are there in total?" The initial questions correspond one-to-one with the initial answers and serve as the input questions in the answer reading stage. Furthermore, the initial questions can be labeled with information such as question type and test points to support the organization and filtering of the initial data, and the sentence structure and difficulty can be flexibly designed according to the needs of the examination.

[0173] The initial answer is the original standard answer corresponding to the initial question, used to indicate the correct response that should be given based on the initial image and the initial question before editing. Specifically, the initial answer can come from manual annotation or from existing annotations in existing question-and-answer data, and its form can be flexible and diverse, such as numerical values, category labels, Boolean judgments, phrases, or short sentences. The initial answer is the basis for determining the change target, which sets the expected direction and result of the change based on the initial answer. Furthermore, the initial answer should be consistent with the visual content in the initial image, and the read answer can be compared with the initial answer to determine whether the answer has actually changed.

[0174] A change objective is a desired outcome set for the initial answer, indicating what changes should be made to the initial answer and the state the edited answer should achieve. Specifically, the change objective can be set for the specific content of the initial answer, such as changing the answer from 7 to 6. The change objective is linked to the editing operations performed on the initial image; the editing operations need to act on the visual elements in the initial image that carry the logic of the answer, causing the change in image content to cause the answer to change in the desired direction. Furthermore, the change objective can describe the type and magnitude of the answer change, such as numerical increases or decreases, literal substitution results, truth value reversals, or category shifts. When determining the change objective, achievable goals can be set in conjunction with the content tested in the initial question.

[0175] An edited image is the image obtained after performing editing operations on the initial image, and it is the object of the answer retrieval stage. Specifically, an edited image can be generated by an instruction-based image editing model that performs operations such as removal, addition, and attribute modification on the initial image according to editing instructions. After editing, the scene content of the image undergoes local changes; for example, after removing a bird, the number of birds in the image decreases accordingly. The edited image needs to maintain a reasonable scene and photorealistic quality; apart from the edited local content, the remaining visual elements should remain consistent with the initial image. Furthermore, the edited image is used as input for answer retrieval along with the initial question, and can also serve as the image basis for subsequently determining the sample dataset.

[0176] Answer retrieval is a process based on editing the image and extracting the answer from the initial question. Specifically, answer retrieval can be performed by a verification model with visual understanding capabilities. After reading the edited image and the initial question, the verification model derives the answer based on the actual visual content observed in the image, rather than inferring from the text in the question or prior knowledge. For example, it can retrieve the number of birds in the edited image to obtain a quantity-related answer. The retrieval result is then compared with the target change to determine whether the editing achieved the intended answer change. Furthermore, to ensure the reliability of the retrieval results, multiple independent verification models can perform answer retrieval separately, and the consistency of each retrieval result can be assessed.

[0177] The answer retrieval stage is the stage where the answer is obtained, representing the content of the answer corresponding to the edited image. Specifically, the answer retrieval is obtained by the validation model based on the edited image and the initial question. The format can include quantity, category label, Boolean judgment, phrase, etc. For example, if the number of birds in the edited image is 6, the answer retrieval could be 6. The answer retrieval is used to compare with the target change. When the answer retrieval matches the target change, it indicates that the editing operation achieved the expected answer change. Furthermore, if the answer retrieval matches the target change, it can serve as the basis for generating sample answers, forming the input for subsequent data construction stages along with the edited image and the initial question.

[0178] In practical applications, obtaining the initial dataset can be achieved through various methods.

[0179] One alternative approach is to obtain the initial dataset from public datasets or existing question-and-answer corpora. Specifically, records containing images, questions, and answers can be read from publicly available visual question-and-answer datasets, or question-and-answer pairs that meet certain criteria can be filtered from existing text-and-image question-and-answer corpora. The image, question, and answer in each record are organized into a triplet to form the initial dataset. During the acquisition process, the data can be filtered and organized. For example, it can be filtered by question type, difficulty, or visual scene to remove records with blurry images, unclear question descriptions, or incorrect answer annotations. The numerical format and expression of the answers can also be standardized to ensure the overall quality and usability of the initial dataset.

[0180] Another alternative approach is to obtain the initial dataset through manual collection and annotation. Specifically, image materials from real-world scenes can be collected manually, such as real photographs containing multiple objects with clear visual structures. For each image, corresponding questions and answers can be created, ensuring the questions and answers strictly correspond to the image content. This process forms the initial dataset. To improve efficiency, models with visual understanding capabilities can be used to assist in generating questions and answers. For example, by analyzing the number, location, and attributes of objects in the images, questions and answers can be automatically generated. These are then manually reviewed and corrected before being incorporated into the initial dataset. This method facilitates control over the quality of the initial data.

[0181] In addition, initial data from multiple sources can be mixed and used together. For example, publicly available data, collected data, and existing question-and-answer corpora can be combined in a preset ratio to expand the size and coverage of the initial dataset.

[0182] Given the initial dataset, the initial image can be edited based on the change target for the initial answer to obtain the edited image.

[0183] One possible approach is to first perform structured analysis on the initial image and initial questions to generate a set of candidate editing operations, and then select and execute the editing operation based on the change target. Specifically, the analyzer can parse the content of the initial image and generate candidate editing operations based on the content examined by the initial questions. The number of candidate editing operations can be set to, for example, 3 to 5. Each candidate editing operation can include an editing operator and an answer effect type. The editing operator can be taken from five types of primitives, such as removal, addition, attribute rewriting, equivalence substitution, and contextual substitution. The answer effect type can be taken from four types, such as numerical increase / decrease, literal substitution, truth value inversion, and category transfer. Then, the appropriate editing operation is selected based on the change target. For example, when the change target is to decrease a number by one, a removal-type editing operation can be selected.

[0184] Another alternative approach is to generate differentiated edit pairs based on the change objective and execute the edits separately, with the edit operations mechanically deriving the changed answer. Specifically, two edit operations can be generated to form a differentiated edit pair. The two edit operations differ in any element of the edit operator or answer effect type, thus constituting differentiation. For example, a removal-type edit and an attribute rewriting-type edit can be generated. Both edit operations perform the complete editing and verification process to increase the sample size. The changed answer can be mechanically derived by the edit operations according to the answer effect type, without manual annotation or model guessing. For example, removing an object that meets the counting condition will necessarily result in a -1 answer, while adding a similar object will necessarily result in a +1 answer. Furthermore, after the edit is executed and verified, a preliminary thought chain outline can be generated. This preliminary thought chain outline can include information such as the question type, key reasoning areas, and applied transformations, which is used to assist in writing the sample thought chain in subsequent stages.

[0185] In addition, manual review can be used to ensure the selection of editing operations. For example, a human can select the operation with the highest degree of matching with the change target from the candidate editing operations.

[0186] Once the edited image is obtained, the answer can be read based on the initial question and the edited image, thus obtaining the answer.

[0187] One alternative approach is to input the initial question and the edited image into a validation model with visual understanding capabilities, allowing the model to directly output the answer. Specifically, the edited image can be used as visual input, and the initial question as the question input. These are organized into a question-and-answer prompt and input into the validation model. The model observes the actual visual content in the edited image, combines it with the requirements of the initial question to derive and output the answer. For example, for an edited image containing multiple birds, the validation model can count the number of birds in the image and use that number as the answer. Along with outputting the answer, the validation model can also output the confidence level of that answer. This method is simple to implement, as the answer is directly obtained by the validation model based on the image content.

[0188] Another alternative approach is to have multiple independent verification models perform answer readings separately, and then determine the consistency of the reading results to identify the correct answer. Specifically, two independent verification models can read the answer based on the edited image and the initial question, respectively, each obtaining its corresponding answer. If the answers output by the two verification models are consistent, the consistent result is determined as the correct answer. Each verification model can also include a confidence level when outputting its answer, and the decision can be made by considering the confidence level. If the correct answer is inconsistent with the target change, the edited image can be discarded, or the editing operation can be re-executed and the image read again.

[0189] In addition, the confidence level results can be used to determine whether to re-execute the editing. For example, if the confidence level of the answer is low, the editing operation can be re-executed before reading.

[0190] Once the answers are obtained, the sample dataset can be determined based on the edited image, the initial question, and the read answers.

[0191] One alternative approach is to directly determine the sample dataset by using the edited image, initial question, and read answer as components of the sample data, provided the read answer meets the change objective. Specifically, the edited image can be used as the sample image, the initial question as the sample question, and the read answer as the sample answer. A sample thought chain corresponding to each sample data point is generated, and the sample image, sample question, sample answer, and sample thought chain are organized into a single sample data point. Multiple sample data points can be aggregated into a single sample dataset; for example, sample data obtained after editing and reading each initial data point can be uniformly merged into the sample dataset. This method is straightforward, and the edited image and initial question can be used in the training phase without additional rewriting.

[0192] Another alternative approach is to determine the sample dataset by further diversifying the images, initial questions, and answers, based on the premise that the read answers meet the change objectives. Specifically, the fixed image attributes that carry the answer logic in the edited images can be kept unchanged, while diversifying the edited free image attributes that are unrelated to the answers. This generates multiple sets of variant images with different appearances but consistent answers. Combined with the slot set determined in the analysis phase, the elements in the question, such as the subject, quantity, and position, are instantiated by slot assignment to generate variant questions and variant answers corresponding to each variant image. Then, the corresponding sample thought chain is generated and aggregated into a sample dataset.

[0193] In addition, sample quality checks can be performed after the sample dataset is determined, such as sampling the sample data according to a preset ratio and verifying the consistency between the images, questions, answers and thought processes.

[0194] In this embodiment, an initial dataset containing an initial image, an initial question, and an initial answer is obtained. The initial image is then edited based on the change target for the initial answer, so that the change in the answer is directly linked to the observable change in the image content. The answer is read based on the initial question and the edited image. If the read answer meets the change target, a sample dataset is determined. The correspondence between the sample image, sample question, and sample answer is verified through the editing and reading process. This provides a sample data foundation with accurate answers and strict correspondence between images and questions for the training of the multimodal large language model, which is beneficial to improving the credibility and construction efficiency of the training data.

[0195] In one optional embodiment of this specification, the initial image is edited based on the change target for the initial answer to obtain an edited image, including: Analyze the initial image to determine its fixed and free image properties; Based on the change target for the initial answer, determine the first editing operation for the fixed image attributes of the initial image; Perform the first editing operation on the initial image to obtain the edited image.

[0196] Image attributes are feature information describing the visual content of an image, used to characterize various aspects of the visual representation of objects in the image. Specifically, image attributes can include object attributes such as object category, quantity, color, texture, shape, size, and material; environmental attributes such as lighting direction, light intensity, color temperature, camera angle, and shooting distance; and scene and layout attributes such as scene type, object position, spatial arrangement, and rendering style. The visual content in an image can be decomposed into multiple image attributes according to a unified attribute system. Furthermore, image attributes can be categorized into different types based on their relationship to the answer; for example, they can be divided into image attributes that carry the logic of the answer and image attributes unrelated to the answer, providing a basis for selection in editing operations.

[0197] Fixed image attributes are image attributes that carry the logic of the answer and are bound to the answer; they are the objects of editing operations. Specifically, when a fixed image attribute changes, the answer read from the image changes accordingly. For example, in counting questions, the number of target objects in the image is a fixed image attribute; if the number changes, the answer changes accordingly. Fixed image attributes typically correspond to the key visual content tested in the initial question, such as the number of objects, spatial relationships, and positions. Furthermore, the determination of fixed image attributes can be based on the content tested in the initial question, identifying the visual elements that affect the answer determination as fixed image attributes, which need to remain unchanged in subsequent diversified expansion stages to ensure the stability of the answer.

[0198] Free image attributes are image attributes unrelated to the answer; changes to them will not alter the answer. Specifically, free image attributes describe visual content in an image that does not affect the determination of the answer, such as background arrangement, lighting atmosphere, and object decoration. In counting problems, the color and style of the background are free image attributes; changing the background will not affect the reading of the quantity. Free image attributes do not need to be strictly maintained during the editing process and can be changed in subsequent diversification stages to introduce visual diversity. Furthermore, free image attributes can include multiple dimensions such as background type, color scheme, perspective, and rendering style. Each free image attribute can change independently or in combination.

[0199] The first editing operation is an operation on a fixed image attribute of the initial image, determined based on the change target for the initial answer, used to change the answer in the desired direction. Specifically, the first editing operation can include editing operators such as removal, addition, attribute rewriting, simulcasting, and contextual substitution. For example, when the change target is to change the answer from 7 to 6, the first editing operation could be to remove an object that meets the counting condition. The first editing operation acts on a fixed image attribute, and the resulting image is the edited image. Furthermore, the first editing operation can correspond to a preset answer effect type, such as numerical increase or decrease, literal substitution, truth value inversion, or category shift. The changed answer can be mechanically derived from the first editing operation according to the answer effect type.

[0200] In practical applications, analyzing the initial image and determining its fixed and free image properties can be achieved through various methods.

[0201] One alternative approach is to decompose the initial image into image attributes based on multiple preset semantic axes, and then determine fixed and free image attributes in conjunction with the content of the initial question. Specifically, the visual content of the initial image can be decomposed into multiple image attributes according to a unified semantic axis system. Details of the semantic axis system can be found in the description of subsequent embodiments. Then, based on the content of the initial question, the relationship between each image attribute and the answer is determined. For example, image attributes that carry the answer logic and are bound to the answer can be defined as fixed image attributes, while image attributes unrelated to the answer can be defined as free image attributes. This method uses a unified attribute system as a framework, and the resulting division covers all aspects of the image, facilitating precise execution of subsequent editing operations.

[0202] Another alternative approach is to have an analysis model with visual understanding capabilities analyze the initial image and directly output fixed and free image attributes. Specifically, the initial image and initial question can be input into the analysis model. The model understands the image content and the focus of the question, determining which visual elements in the image affect the answer and which are irrelevant. For example, it can determine that the number of objects affecting the count is a fixed image attribute, while the background and decorations that do not affect the answer are free image attributes. The analysis model can output structured attribute partitioning results, such as lists of fixed and free image attributes. This method, leveraging the model's analytical capabilities, is suitable for scenarios with complex image content and where manual partitioning is costly.

[0203] In addition, manual annotation can be used to assist in the determination of attribute status. For example, the attribute status label can be determined by manual review of semantic axes with low confidence in the automatic determination results.

[0204] Once the fixed image attributes and free image attributes of the initial image are determined, the first editing operation of the fixed image attributes of the initial image can be determined based on the change target for the initial answer.

[0205] One alternative approach is to first generate multiple candidate editing operations for a fixed image attribute, and then determine the first editing operation from these candidates based on the change target. Specifically, a set of candidate editing operations can be generated, with the number of operations set to, for example, 3 to 5. Each candidate editing operation applies to a fixed image attribute and employs different editing operators or corresponds to different answer effect types. Subsequently, the answer change results corresponding to each candidate editing operation are compared with the change target, and the candidate editing operation whose answer change result matches the change target is selected as the first editing operation. For example, when the change target is a numerical decrement, a removal-type candidate editing operation that decrements the answer by one is selected.

[0206] Another alternative method is to directly determine the first editing operation based on the correspondence between the target change and the fixed image attributes. Specifically, the corresponding editing method can be determined on the fixed image attributes according to the answer change type indicated by the target change. For example, when the target change is truth flipping, the visual content carrying the truth judgment in the fixed image attributes can be modified to change the judgment result from yes to no; when the target change is literal replacement, the visual elements in the fixed image attributes corresponding to the answer content can be replaced; when the target change is category migration, the category of the object in the fixed image attributes can be migrated to another category. After determining the first editing operation, the edited answer can be derived to verify the consistency between the editing operation and the target change.

[0207] In addition, the first editing operation can be selected based on the estimated reasonableness of the scene after editing. For example, if the estimated editing operation has little impact on the scene structure and can accurately change the answer, this operation should be selected first.

[0208] Once the first editing operation is determined, the first editing operation can be performed on the initial image to obtain the edited image.

[0209] One alternative approach is to rewrite the first editing operation as a natural language editing instruction, which is then used by an instruction-based image editing model to edit the initial image. Specifically, the first editing operation can be rewritten as a single, continuous natural language instruction. The instruction length can be set to, for example, 150 to 250 words. The instruction content must meet preset requirements, such as describing the scene and including the key points of the question, accurately describing the unique editing operation and its spatial details, weaving the lock list into the description in clause form, and ending with a declaration of completeness. The instruction format prohibits bullet points and tag-style paragraph headers, and key constraints can be emphasized using reinforcement words. The editing instruction is then input into the instruction-based image editing model, which performs the editing while maintaining photorealistic quality and fixed image attribute constraints, resulting in an edited image.

[0210] Another alternative approach is to mechanically derive the changed answer during the first editing operation and verify the editing result accordingly. Specifically, the changed answer can be mechanically derived from the first editing operation based on the answer effect type. For example, if the first editing operation is to remove a cartoon bird that meets the counting condition, the changed answer changes from 7 to 6, without requiring the model to guess again. After obtaining the edited image through the first editing operation, the derived answer can be used as the expected result and compared with the reading answer obtained from subsequent answer readings to determine whether the editing was accurately achieved. The correspondence between the editing operator and the answer effect type can be pre-established using mapping rules.

[0211] In addition, a quick self-check can be performed on the edited image after the first editing operation is performed, such as checking whether the edited image is obviously damaged or obviously inconsistent with the editing instructions. If it fails, the editing can be re-executed.

[0212] In this embodiment, by analyzing the initial image, the image attributes of the initial image are divided into fixed image attributes and free image attributes, clarifying the visual content to which the answer is attached and the freely changeable visual content; based on the change target for the initial answer, a first editing operation is determined to act on the fixed image attributes, so that the editing operation strictly corresponds to the change target; the first editing operation is performed on the initial image to obtain the edited image, so that the answer changes in the desired direction as the image content changes, and the edited image maintains the reasonableness of the scene and photorealistic feel, providing a data foundation for the determination of the sample dataset where the image and the answer correspond consistently.

[0213] In one optional embodiment of this specification, analyzing the initial image to determine its fixed image properties and free image properties includes: Based on multiple preset semantic axes, the initial image is decomposed into image attributes, and combined with the test points of the initial question, the attribute state labels corresponding to the image attributes on each preset semantic axis are determined. The attribute state labels include frozen state, free state and condition-dependent state. Image attributes in the frozen state and conditionally dependent state are determined as fixed image attributes of the initial image, and image attributes in the free state are determined as free image attributes of the initial image.

[0214] Preset semantic axes are a dimensional system used for the structured decomposition of visual content in images. Specifically, preset semantic axes can be organized by groups, such as lighting, viewpoint, object, background, layout, and style groups. Each semantic axis group can have multiple semantic axes, for example, twenty, or other numbers depending on actual needs. Furthermore, each semantic axis corresponds to a type of observable and editable visual feature of the image. Image content can be decomposed onto each preset semantic axis to obtain corresponding attribute values. The number and composition of preset semantic axes can be flexibly adjusted, providing a unified analytical framework for attribute state determination and subsequent editing.

[0215] Image attribute decomposition is the process of breaking down image content into multiple dimensional attribute values ​​according to preset semantic axes. Specifically, performing image attribute decomposition on an initial image can determine the attribute values ​​on each preset semantic axis. For example, it can determine that the lighting direction is sidelight, the object color is yellow, and the number of objects is 7. The decomposition results can be recorded in the form of an attribute table, such as recording the attribute name, attribute value, and state information corresponding to each semantic axis. Decomposition can be performed by an analyzer with visual understanding capabilities or by humans combining image observation. Furthermore, image attribute decomposition is the basis for attribute state determination. By comparing the decomposed attribute values ​​with the test points in the question, the degree of correlation between each image attribute and the answer can be determined.

[0216] The key points of a question are the crucial content of the image tested in the initial question, representing the visual basis upon which the answer depends. Specifically, key points can include key elements and points in the image, such as the number of objects, the spatial relationships between objects, and the positions of objects. These elements can be atomically edited; for example, when testing quantitative relationships, the number of objects in the image is the key point, and when testing spatial relationships, the relative positions of the objects are the key points. Furthermore, key points can be obtained through semantic analysis of the initial question, or determined by an analyzer combining the initial image and the initial question. Key points directly determine which image attributes are relevant to the answer.

[0217] Attribute status labels are status markers that identify the relationship between image attributes and answers, guiding the handling of various image attributes in subsequent editing and diversification processes. Specifically, attribute status labels can include three types: frozen state, free state, and conditionally dependent state. A frozen state indicates that the image attribute carries the answer logic and needs to be maintained during editing and diversification. A free state indicates that the image attribute is unrelated to the answer and can be radically changed to introduce diversity. A conditionally dependent state indicates that the processing method of the image attribute depends on the specific editing operator. Furthermore, attribute status labels can be obtained by the analyzer in conjunction with the question's test points, axis by axis, or they can be directly output by the visual language model and then corrected through rule validation. The final result is output in the form of a structured attribute table, serving as the direct basis for attribute division.

[0218] A frozen state is a state that identifies image attributes directly related to the answer logic, used to constrain the processing of key visual information during editing and diversification. Specifically, image attributes in a frozen state carry key information about the answer logic. For example, in a question testing the quantity of objects, the quantity attribute is frozen; changing this attribute will change the answer. Image attributes in a frozen state need to be maintained during editing and diversification; for example, the quantity and category of objects should remain consistent before and after editing. Furthermore, the frozen state can be determined by the analyzer in conjunction with the question's test points. Image attributes directly related to the test points and involved in answer derivation can be marked as frozen. Image attributes in a frozen state constitute the main part of fixed image attributes and participate in the construction of frozen predicates.

[0219] A free state is a state that identifies image attributes unrelated to the answer, used to define the range of attributes that can be radically altered to introduce visual diversity. Specifically, image attributes in a free state are not related to the question's focus and do not participate in the derivation of the answer. For example, in a question testing the quantity of objects, attributes such as background color, lighting direction, and rendering style are in a free state; changing these attributes does not affect the answer value. Image attributes in a free state can be significantly altered during editing and diversification to generate image variants with drastically different appearances but consistent answers. Furthermore, a free state can be determined by the analyzer in conjunction with the question's focus; image attributes unrelated to the focus and not involved in the answer logic can be marked as free states; image attributes in a free state constitute a set of free image attributes.

[0220] Conditional dependency states identify the state of image attributes whose processing method depends on the specific editing operator, and are used to handle attribute classifications with unclear relationships. Specifically, the relationship between image attributes in conditional dependency states and the answer is not absolute; whether they affect the answer depends on the editing operator used. For example, the surface attribute needs to be maintained in removal-type editing, but may change with the scene in context-displacement-type editing. For such attributes, a conservative strategy can be adopted to incorporate them into fixed image attributes to avoid editing or diversification processes disrupting the answer logic. Furthermore, the determination of conditional dependency states can be combined with the candidate editing operation set, judging the influence of each candidate editing operator on the attribute one by one; in cases where it is difficult to determine, it can be preferentially treated as a fixed image attribute.

[0221] In practical applications, image attribute decomposition of the initial image based on multiple preset semantic axes, and determination of attribute status labels on each semantic axis in conjunction with the test points of the initial question, can be achieved through various methods.

[0222] One possible approach is to decompose the initial image axis by axis according to a preset semantic axis system, and then determine the state axis by axis in conjunction with the question's test points. Specifically, twenty semantic axes can be set and categorized into six groups: the lighting group includes lighting direction, lighting intensity, and color temperature; the perspective group includes camera angle, shooting distance, and camera orientation; the object group includes color, texture, shape, size, and material; the background group includes scene type, clutter level, and color scheme; the layout group includes object position, spatial arrangement, and scale relationship; and the style group includes rendering style, contrast, and saturation. The analyzer extracts attribute values ​​axis by axis. Furthermore, the attribute values ​​on each semantic axis are compared with the question's test points one by one. Attributes that carry the logic of the test points are marked as frozen, attributes that are unrelated to the test points are marked as free, and attributes that depend on the editing operator are marked as conditionally dependent.

[0223] Another alternative approach is to first extract the key points of the question, and then determine the status labels based on the degree of association between the key points and the attributes of each semantic axis. Specifically, semantic analysis can be performed on the initial question to extract the key elements and points being tested, such as quantity, spatial relationships, and location, forming a description of the key points. Then, the values ​​of the attributes on each semantic axis are compared and correlated with the description of the key points. For example, the attribute values ​​on the quantity axis are directly associated with counting-related key points, while the attributes on the background and style groups are not associated with counting-related key points. Attributes that directly carry the logic of the key points are designated as frozen states, attributes unrelated to the key points are designated as free states, and attributes with uncertain associations are designated as conditionally dependent states. Furthermore, the judgment results can be validated using rules or manually reviewed and corrected to ensure that the status labels align with the tested meaning. Figure 1 To.

[0224] In addition, the default attribute states of each semantic axis can be preset according to the question type, and then modified according to the specific question's test points to reduce the computational overhead of attribute state determination.

[0225] Once the attribute state labels on each semantic axis are determined, the image attributes in the frozen state and conditionally dependent state can be further determined as the fixed image attributes of the initial image, and the image attributes in the free state can be determined as the free image attributes of the initial image.

[0226] One alternative approach is to categorize image attributes based on their state labels along each semantic axis. Specifically, image attributes in a frozen state can be designated as fixed image attributes, and those in a conditionally dependent state can also be included in the fixed image attribute set. For example, in counting problems, the quantity and surface area attributes of objects are both categorized as fixed image attributes. Image attributes in a free state are designated as free image attributes, forming a free image attribute set. This free and fixed image attribute set complement each other, covering all semantic axes. Furthermore, the handling of conditionally dependent states follows a conservative strategy: if it cannot be confirmed that an attribute is irrelevant to the answer, it is preferentially categorized as a fixed image attribute to avoid disrupting the answer's logic.

[0227] Another alternative approach is to construct a frozen predicate as a unified constraint throughout the entire process, building upon the attribute partitioning. Specifically, an extraction operator can be defined for each attribute in the fixed image attribute set. This extraction operator is used to extract the value of that attribute from the image, such as extracting the number, category, and key spatial relationships of objects from the image. The frozen predicate can be described as ensuring that all fixed image attributes have consistent values ​​before and after editing. For example, fixed attributes are extracted from the images before and after editing and compared; if all are consistent, the frozen predicate value is 1. Furthermore, the frozen predicate can simultaneously serve as the acceptance criterion for image generation results, the basis for constructing diverse boundary conditions in the second stage, and the scoring reward for the reinforcement learning stage. The partitioning results can be output in structured forms such as attribute tables, preservation lists, and validation items.

[0228] In addition, the attribute status determination results of each semantic axis can be organized into an attribute status table and saved along with the sample data for direct reference in subsequent stages.

[0229] In this embodiment, the initial image is decomposed based on multiple preset semantic axes, and the attribute state labels on each semantic axis are determined in conjunction with the test points of the initial question. Then, the image attributes in the frozen state and conditionally dependent state are determined as fixed image attributes, and the image attributes in the free state are determined as free image attributes according to the state labels. This organizes the image content into a structured and measurable attribute dimension system, and the scope of preservation and modification in the editing and diversification process is clearly defined. The conditionally dependent attributes are incorporated into the fixed attributes according to a conservative strategy, which avoids the unintentional destruction of the answer logic due to differences in editing operators. This provides precise constraint boundaries for subsequent editing operations and diversification expansion, and provides a foundation for the credibility and visual diversity of the sample data.

[0230] In one optional embodiment of this specification, a sample dataset is determined based on the edited image, the initial question, and the read answer, including: Analyze the edited image to determine its fixed and free image attributes; While keeping the fixed image attributes of the image being edited unchanged, determine the second editing operation for the free image attributes of the image being edited; Perform a second editing operation on the edited image to obtain multiple variant images; Based on the initial question and the read answer, determine the variant question and variant answer corresponding to each variant image; The variant images are verified based on the preset verification dimensions. If the verification is successful, a sample thought chain corresponding to each variant image is generated based on each variant image and its corresponding variant question and variant answer. Using each variant image as a sample image, the variant question as a sample question, and the variant answer as a sample answer, and combining the sample thought chain, the sample dataset is determined.

[0231] The second editing operation is an editing action determined by the free image attributes of the image being edited. It is used to generate variant images with diverse appearances while keeping the fixed image attributes unchanged. Specifically, the second editing operation differs from the first editing operation. The first editing operation acts on fixed image attributes and changes the answer, while the second editing operation keeps the fixed image attributes unchanged and only edits the free image attributes, causing significant changes in appearance while keeping the answer unchanged. For example, performing the second editing operation on a counting image can change the background, lighting, and viewpoint without changing the number of objects. Furthermore, the second editing operation can be determined by the planner after generating multiple sets of values ​​for the free image attributes, and described in the form of diversified editing instructions or structured value schemes.

[0232] Variant images are images obtained by performing a second editing operation on an edited image, and are used to constitute sample images in the sample dataset. Specifically, variant images maintain the same fixed image attributes as the edited image, while the free image attributes change. For example, the number, category, and key spatial relationships of objects in the edited image remain unchanged, while the background scene, lighting conditions, shooting angle, and rendering style change. Performing multiple sets of second editing operations on the edited image can yield multiple variant images, which appear very different but have the same answer. Furthermore, variant images need to meet the constraints of natural photographic distribution, such as presenting a believable real photograph rather than an illustration or surreal style; variant images also need to pass the verification of a preset verification dimension.

[0233] Variant questions are question texts corresponding to variant images, used to describe reasoning tasks that require answers based on the content of the variant image. Specifically, variant questions can be determined based on the initial question and maintain a correspondence with the visual content in the variant image. For example, if the initial question is "How many cartoon birds are in the picture?", the variant question with the corresponding subject replacement could be "How many blue cars are in the picture?". Variant questions maintain the same logical structure and examination method as the initial question, only changing elements such as the subject and the supporting surface. Furthermore, variant questions can be obtained through template instantiation, for example, by replacing the slots in the question template with the subject values ​​corresponding to the variant image, thus obtaining variant questions that match the variant image; there is a one-to-one correspondence between variant questions, variant images, and variant answers.

[0234] Variant answers are the answer text corresponding to variant questions, used together with variant images and variant questions to form the answer portion of the sample data. Specifically, variant answers can be determined based on the read answer, which is the answer obtained from reading the edited image and the initial question. The variant answer maintains the same value as the read answer; for example, if the read answer is 6, the variant answer for each variant image is 6. Variant answers also correspond to the subject replacement of the variant question; for example, if the subject of the question is replaced with a car instead of a bird, the answer value remains the same. Furthermore, variant answers can be obtained by instantiating an answer template, or they can directly use the value or category of the read answer to ensure that the answers for samples within the same variant image family remain consistent.

[0235] The preset verification dimensions are a set of verification items used to check the quality and validity of variant images, performing quality control before the variant images are included in the sample dataset. Specifically, the preset verification dimensions may include verification items such as correct target object category, preservation of key spatial relationships, clear identification of test point areas, and credibility of natural photographs. Each verification item can be judged as yes or no by a visual language verifier. Furthermore, the variant image is only accepted if all verification items pass; if any verification item fails, the variant image can be discarded or regenerated. The preset verification dimensions may also include rule-based verification items such as frozen predicate verification.

[0236] In practical applications, analyzing and editing images to determine their fixed and free image attributes can be achieved through various methods.

[0237] One alternative approach is to reuse the partitioning results obtained from the attribute analysis of the initial image in the first stage. Specifically, during the initial image analysis in the first stage, the attribute state labels on each semantic axis were determined based on the key points of the question, and fixed image attributes and free image attributes were partitioned accordingly. The first editing operation only applies to the fixed image attributes, while the free image attributes retain their original state after editing. Therefore, this partitioning result can be directly used as the fixed and free image attributes for the edited image. It can be modified based on the type of the first editing operation; for example, when surface attributes are involved, the relevant attributes can be added as fixed image attributes. Furthermore, the frozen predicates and hold lists from the first stage can also be reused simultaneously to maintain consistency in the diverse constraint boundaries of the second stage.

[0238] Another alternative approach is to re-perform image attribute decomposition and state determination on the edited image. Specifically, image attribute decomposition can be re-performed on the edited image according to a preset semantic axis system, extracting attribute values ​​on each semantic axis, and then re-determining the state labels of each attribute in conjunction with the test points of the initial question or variant question. For example, the correlation between attributes such as the number, category, and spatial relationship of objects in the edited image and the answer can be re-determined. The re-determination results can be compared with the division results of the first stage, and the re-determination results shall prevail if there are differences. Furthermore, re-performing the analysis can address situations where the attribute state changes during the editing process. For example, in contextual substitution editing, if background attributes become associated with the answer, the re-analysis can capture this change and adjust the state labels.

[0239] In addition, the results of the attribute analysis before and after editing can be compared, and the semantic axis where the attribute state changes can be recorded separately to verify the impact of the editing operation on the attribute state.

[0240] Given the fixed image attributes and free image attributes of the image to be edited, the fixed image attributes of the image to be edited can be kept unchanged, and a second editing operation can be performed on the free image attributes of the image to be edited.

[0241] One alternative approach is for the planner to generate multiple sets of value schemes for the free image attributes. Specifically, the fixed image attributes of the edited image can be kept unchanged, and the free image attributes can be used as the operation objects. The planner can generate multiple sets of values, for example, four sets of value schemes, each corresponding to a combination of values ​​for a set of free attributes. The differences between the sets of schemes can be maximized, for example, by maximizing the sum of the distances between any two sets of values ​​in axis space to ensure that the visual differences between the sets are large enough. At the same time, each set of schemes needs to be self-consistent, for example, a snowy background should not appear simultaneously with the strong midday desert light. Furthermore, the value schemes need to meet hard constraints, such as not touching fixed image attributes, pre-defined checks still being valid, values ​​falling within the natural photo distribution, and locked slots maintaining their default categories.

[0242] Another alternative approach is to generate a diversity signature and slot assignment for each set of value schemes. Specifically, a diversity signature consisting of 3 to 5 tags can be generated for each set of value schemes, such as tags like "outdoor snow," "morning cold light," and "telephoto close-up." The diversity signature is used for distribution monitoring and deduplication at the corpus level. Slot assignments corresponding to each scheme can also be determined. These slot assignments are used for the instantiation of subsequent question and answer templates; for example, the subject slot can be assigned the actual subject in the variant image. Furthermore, the second editing operation can integrate the value scheme, slot assignments, and diversity signature determination, and include specific modification descriptions for each free axis, such as specifying the desired attribute value or direction of change for each free axis.

[0243] In addition, the second editing operation can be determined by combining the distribution monitoring at the corpus level. For example, the emphasis of each group of values ​​can be adjusted according to the distribution of diverse signatures to avoid the homogenization of variant patterns.

[0244] Once the second editing operation is determined, it can be performed on the image to obtain multiple variant images.

[0245] One alternative approach is to rewrite the second editing operation into diverse editing instructions, executed by an instruction-based image editing model. Specifically, each set of value schemes can be rewritten as a single, continuous natural language editing instruction, with a length between 150 and 260 words. Each free axis is explicitly covered with at least one specific adjective or noun phrase, such as specifying "change to winter snow scene" for the background axis and "change to soft, cool morning light" for the lighting axis. The instructions should avoid weakening words like "slight" or "absolutely," prioritizing strengthening words like "completely," "thoroughly," and "significantly." The instructions must also meet natural photographic constraints, prohibiting illustrations and surreal styles, monochrome stage lighting, and extreme weather. Furthermore, the instructions can conclude with a positive reiteration of the preservation list, such as reiterating the content to be preserved in the phrase "the scene is still displayed," without mentioning the question or answer throughout to prevent the answer from leaking into the generation conditions.

[0246] Another alternative approach is to use an instructional image editing model to perform edits group by group and output variant images. Specifically, the editable image and each set of diverse editing instructions can be input into the instructional image editing model. The model then executes the edits group by group according to the instructions, obtaining variant images corresponding to each set of value schemes. For example, executing four sets of editing instructions can generate four variant images. During the generation process, the model maintains fixed image attributes while modifying free image attributes, outputting results that satisfy the constraints of a natural photograph. After generation, it can be checked whether there are indeed significant visual differences between the variant images, and whether fixed image attributes such as the number and category of objects are consistent with the edited image. Furthermore, variant images that fail to generate can be regenerated based on the modified editing instructions, or the set of schemes can be discarded.

[0247] In addition, batch deduplication can be performed on the generated variant images, such as removing variant images that look too similar based on diversity signatures or image similarity.

[0248] With multiple variant images available, the variant question and variant answer corresponding to each variant image can be determined based on the initial question and the read answer.

[0249] One alternative approach is to determine variant questions and answers through template instantiation. Specifically, during the initial image analysis in the first stage, question templates and answer templates can be generated. The question template includes slots for the main subject, the supporting surface, and the quantity, while the answer template contains corresponding subject descriptions. In the second stage, the slot values ​​for each solution are assigned to the templates. For example, assigning "blue car" to the main subject slot and "snow" to the supporting surface slot, the question template is instantiated to obtain variant questions. The answer template, combined with the read answer, is instantiated to obtain variant answers. For example, if the read answer is 6, the variant answer could be "6 vehicles." Template instantiation allows the same logical structure to cover different subjects; for example, the subject of the question can be changed from a bird to a car. Furthermore, the instantiated variant questions and answers need to match with the variant image to ensure that the question is answerable on the variant image.

[0250] Another alternative approach is to directly rewrite and generate variant questions and answers based on the initial question and the read answer. Specifically, the semantic structure and testing method of the initial question can be kept unchanged, while elements such as the subject and supporting surface in the question can be replaced with elements corresponding to the variant image. For example, "How many cartoon birds are in the picture?" can be rewritten as "How many blue cars are in the picture?". The variant answer can be based on the read answer, adjusting the expression according to the replaced subject. For example, if the read answer is 6, the variant answer can be expressed as "6 cars". The rewriting can be performed by a text model or automatically according to slot assignment rules. Furthermore, the correspondence between variant questions and variant answers can be verified, such as checking whether the variant question is answerable on the variant image and whether the visual content of the variant answer is consistent with the variant image.

[0251] In addition, the semantic rationality of slot assignment can be combined to determine variant questions and variant answers. For example, the template can be instantiated only after verifying that the main slot assignment is consistent with the content of the variant image.

[0252] After obtaining the variant questions and variant answers, each variant image can be verified based on preset verification dimensions.

[0253] One alternative approach is for a visual language validator to perform yes / no judgments across multiple verification items. Specifically, preset verification dimensions may include verification items such as correct target object category, preservation of key spatial relationships, clear identification of test point areas, and credibility of the natural photograph. The visual language validator can answer yes / no questions for each variant image item, such as whether the target object category in the variant image is correct, whether key spatial relationships are preserved, whether test point areas are clearly identifiable, and whether the overall image presents a credible, realistic photograph. If the judgment result for each verification item is "yes," the variant image passes verification. Furthermore, the validator can employ a model with visual understanding capabilities, or multiple validators can be used to perform judgments independently to improve the reliability of the verification.

[0254] Another alternative approach is to combine frozen predicates with rule-based validation for a comprehensive judgment. Specifically, in addition to visual validation, frozen predicate validation can be performed on the variant image. This includes checking, for example, whether the fixed image attributes are consistent between the edited and variant images, including the number of objects, categories, and key spatial relationships. Rule-based checks can also be performed, such as checking for duplicates between variant images and whether the diversity signature meets distribution requirements. If all validation items pass, the variant image is accepted. If any item fails, the variant image can be discarded or regenerated; for example, the diversity editing instructions can be adjusted and a second editing operation can be performed. Furthermore, the validated variant images and their corresponding variant questions and answers are then used in the subsequent sample thought chain generation stage.

[0255] In addition, it can record and classify the reasons for failed verification of variant images, such as distinguishing failure types such as category error, relationship disruption and insufficient readability, to guide the adjustment of various editing instructions.

[0256] Once the verification is successful, a sample thought chain corresponding to each variant image can be generated based on each variant image and its corresponding variant question and variant answer.

[0257] One alternative approach is to use a model with visual understanding capabilities to generate sample thought chains by combining variant images, variant questions, and variant answers. Specifically, variant images, variant questions, and variant answers can be input into a model with visual understanding capabilities. The model observes the content of the variant images and reasons step-by-step about the variant questions, generating a reasoning process text that includes overall observation, key element identification, task-oriented reasoning, and conclusions. The reasoning process text is organized in a natural writing style, for example, using 3 to 6 coherent sentences to describe the image structure, key objects and their attributes and counts, spatial relationship determination process, and the final answer; the conclusion can be consistent with the variant answer. Furthermore, during generation, ambiguity resolution soft cues from the first-stage analysis results can be incorporated to help the model accurately identify visual elements related to the variant questions.

[0258] Another alternative approach is to perform a consistency check after the sample thought chain is generated. Specifically, this involves checking whether the objects, quantities, and spatial relationships described in the sample thought chain are consistent with the variant image; for example, checking whether the object categories and quantities mentioned in the thought chain match the visual content in the variant image. It also involves checking whether the reasoning conclusions of the sample thought chain are consistent with the variant answer; for example, whether the numerical values ​​in the conclusion section are the same as the variant answer. Sample thought chains that pass the check, along with the variant image, variant question, and variant answer, constitute the sample data. Sample thought chains that fail the check can be regenerated. Furthermore, multiple candidate thought chains can be generated for the same variant image, and the one that best fits the variant image and has a complete reasoning process can be selected as the sample thought chain.

[0259] In addition, the generated sample thought chain can be checked for length and format, such as whether the number of sentences is within the preset range and whether it ends with a fixed marker. If it does not meet the requirements, it can be regenerated.

[0260] By generating sample thought chains corresponding to each variant image, each variant image can be used as a sample image, the variant question as a sample question, and the variant answer as a sample answer. Combined with the sample thought chains, the sample dataset can be determined.

[0261] One alternative approach is to aggregate sample datasets in the form of variant data quadruples. Specifically, each variant image can be used as a sample image, the corresponding variant question as a sample question, the corresponding variant answer as a sample answer, and the corresponding sample thought chain as a reference reasoning chain to constitute sample data. For example, the same variant image family can generate multiple sample datasets, with different image appearances but the same answers. Multiple sample datasets are aggregated into a sample dataset, which can be used for training multimodal large language models. Furthermore, each sample data in the sample dataset can also be accompanied by metadata, such as question type, question focus, and diversity signature, for data organization and statistical analysis during the training process.

[0262] Another alternative approach is to perform sample-level quality checks before determining the sample dataset. Specifically, quality checks can be performed on each sample data point before aggregation. For example, this includes checking the consistency between the sample image, sample question, sample answer, and sample thought process chain; checking if the sample image passes the verification of preset dimensions; and checking if the sample data is duplicated from existing samples. Diversity signatures can be used to perform corpus-level deduplication and distribution monitoring, such as removing highly similar variant samples to ensure a balanced diversity distribution in the sample dataset. Sample data that passes the checks is added to the sample dataset, while sample data that fails can be discarded or regenerated. Furthermore, the size of the sample dataset can be configured based on the target training data volume and continuously expanded as data construction capabilities increase.

[0263] In addition, corpus-level distribution statistics can be performed after aggregating the sample dataset, such as counting the number of samples for each question type and difficulty level, and generating additional samples based on the distribution.

[0264] In this embodiment, fixed image attributes and free image attributes are determined by analyzing and editing images. The fixed attributes are kept unchanged, and a second editing operation is determined for the free attributes. After execution, multiple variant images are obtained, and the variant questions and variant answers corresponding to each variant image are determined. After verification by a preset verification dimension, a sample thought chain is generated, and finally the sample dataset is determined. This allows a group of variant samples with the same answer but different appearances to be constructed in batches. The model needs to actually observe the image content to maintain the correct answer across the entire set of samples. The second editing operation expands the free image attributes, allowing the samples to obtain sufficient visual diversity while maintaining the logic of the answer. The preset verification dimension removes unqualified results before the variant images enter the sample dataset, providing a visually diverse, answer-consistent, and reliable sample dataset foundation for the training of multimodal large language models.

[0265] In one optional embodiment of this specification, the sample thought chain includes overall observation content, attribute identification content, task reasoning content, and reasoning conclusion content; Based on the variant images and their corresponding variant questions and answers, a sample thought chain is generated for each variant image, including: Based on each variant image and its corresponding variant title, overall observation content and attribute recognition content are generated. The overall observation content is used to describe the overall structure and object scale of each variant image, and the attribute recognition content is used to characterize the image attributes related to the variant title. Based on the image attributes included in the attribute recognition content, task reasoning content is generated, wherein the task reasoning content includes the reasoning operation to be performed corresponding to the variant question; Based on the variant answers, generate the reasoning conclusion.

[0266] The overall observation content is the information used in the sample thought chain to describe the overall structure and object scale of the variant image. Specifically, the overall observation content can describe the scene type, subject distribution, and object scale presented in the image. For example, it can describe the image as an outdoor scene, containing multiple objects, with a large number of objects that are scattered. The overall observation content is expressed in natural language, without using a numbered list format, and can briefly point out the image areas related to the variant question. Furthermore, the overall observation content can be generated jointly based on the variant image and the variant question, with the observation scope centered on the objects and areas of interest in the variant question, providing background and location information for subsequent attribute recognition and task reasoning.

[0267] Attribute recognition content refers to the image attributes used in the sample thought chain to represent those related to the variant question. Specifically, attribute recognition content can list the objects and their attributes related to the variant question item by item, such as the object's color, shape, size, and material, and can also include spatial relationships and positional information between objects. The recognition results are organized in coherent sentences, such as "There are 3 blue birds on the left side of the picture, two of which are perched on branches." Furthermore, the recognition scope is based on the elements examined in the variant question; objects unrelated to the question can be briefly described. Attribute recognition content provides identified visual elements for task reasoning, giving subsequent reasoning operations a clear focus.

[0268] The task reasoning content is the information within the sample thought chain used to describe the reasoning operations to be performed, corresponding to the variant questions. Specifically, the task reasoning content can include concrete operations such as counting, spatial relationship determination, and conditional filtering. For example, it might involve counting identified objects one by one, comparing the relative positions of objects, or filtering target objects by color or location. These reasoning operations are progressively applied to the identified visual elements within the attribute recognition content, ensuring that each step of the reasoning is supported by visual evidence. Furthermore, the task reasoning content can be organized using 3 to 6 natural sentences and can briefly indicate distractors and distinguishing features, such as pointing out that objects with similar shapes but different colors should not be counted.

[0269] The reasoning conclusion is the content used to output the final answer in the sample thought chain, corresponding to the variant answer. Specifically, the reasoning conclusion can be determined based on the variant answer, converging the reasoning result obtained from the task reasoning content into a clear answer statement, such as "Therefore, there are 6 birds in the picture"; the conclusion part can use a fixed marker to end, such as ending with "FinalAnswer: 6". During training, this format can be converted to " <answer> 6< / answer> In the form of "", the reasoning conclusion is consistent with the attribute identification content and the task reasoning content, and the numerical value or category in the conclusion matches the variant answer, forming a complete reasoning chain from observation to reasoning to conclusion.

[0270] In practical applications, generating overall observation content and attribute recognition content based on each variant image and its corresponding variant title can be achieved through various methods.

[0271] One alternative approach is to directly generate the image using a visual language model with visual understanding capabilities, combined with variant images and variant questions. Specifically, the variant images and variant questions can be input into the visual language model. The model first performs an overall perception of the scene, generating holistic observation content describing the scene type, object size, and distribution. Then, based on the elements examined in the variant questions, it identifies relevant objects and their attributes item by item, such as the object's color, shape, size, and material, generating attribute recognition content. The two parts can be merged into a single continuous text, connected by natural language. Furthermore, ambiguity resolution soft cues from the analysis results can be used to guide the model to focus on easily confused areas, avoiding the inclusion of visually similar but unrelated objects in the recognition scope.

[0272] Another alternative approach is to first extract the structured elements of the image and then organize them into natural text using a text model. Specifically, object detection and attribute annotation can be performed on the variant image beforehand to obtain structured elements such as object category, quantity, location, and color. For example, three objects may be detected, all of which are blue. These structured elements are then compared with the elements examined in the variant question to filter out objects and their attributes relevant to the question. These are then organized by a text model into overall observation content and attribute recognition content. Furthermore, the consistency between the generated content and the variant image can be verified, such as comparing the number of objects with the detection results. If discrepancies are found, the description can be regenerated or revised.

[0273] In addition, the overall observation content can be generated by combining the image description model and the attribute recognition content can be generated by combining the target detection results. The two can then be organized into the starting part of the thought chain.

[0274] Once the overall observation content and attribute recognition content are generated, task reasoning content can be generated based on the image attributes included in the attribute recognition content.

[0275] One alternative approach is to generate reasoning operations by mapping the image attributes in the attribute recognition content to the reasoning requirements in the variant questions. Specifically, the examination method of the variant questions can be analyzed, such as determining whether the question requires counting, judging spatial relationships, or filtering by conditions. Then, the corresponding image attributes are extracted from the attribute recognition content to perform the operation, such as counting the identified objects one by one, comparing the relative positions between objects, or filtering target objects by color or location. Each step of the operation is applied to the identified visual elements. Furthermore, the reasoning operations can be developed step by step using natural language, and briefly pointing out distractors and distinguishing features, such as explaining that objects with similar shapes but different colors should not be included.

[0276] Another alternative approach is for a visual language model to perform multi-step reasoning based on attribute recognition and record intermediate judgments. Specifically, variant images, variant questions, and attribute recognition content can be input into the visual language model. The model sequentially performs operations such as counting, relation determination, and condition filtering to generate task reasoning content that includes intermediate judgment processes. For example, it might first generate the judgment "The total number of birds in the picture is 6," and then generate the judgment "Of them, 3 are standing on the branches." The latter conclusion is built upon the former conclusion and the image evidence. Furthermore, the quantity and spatial relationship descriptions in the task reasoning content can be verified to ensure they match the variant images; if inconsistencies are found, the data can be regenerated.

[0277] In addition, task reasoning content can be generated by combining preset templates of reasoning rules. For example, a counting reasoning template can be applied to counting problems, and a relationship judgment template can be applied to spatial relationship problems.

[0278] When generating task reasoning content, reasoning conclusions can be generated based on variant answers.

[0279] One alternative approach is to organize variant answers into conclusion statements and end them with a fixed marker. Specifically, variant answers can be embedded into conclusion sentences; for example, when the variant answer is 6, the conclusion could be generated as "Therefore, there are 6 birds in the picture." Alternatively, the conclusion could be organized based on the question format of the variant question. A fixed marker could be appended to the conclusion, such as ending with "Final Answer: 6." During training, this marker can be uniformly converted to "...". <answer> 6< / answer> The format is "". Furthermore, it can verify the consistency between the conclusion and the task's reasoning content. For example, if the numerical value in the conclusion matches the numerical value obtained during the reasoning process, and they do not match, the conclusion should be corrected or the reasoning should be restarted.

[0280] Another alternative approach is to use a visual language model to generate the conclusion by combining variant answers with the reasoning process. Specifically, variant images, variant questions, attribute recognition content, task reasoning content, and variant answers can be input into the visual language model. The model then connects the variant answers with the aforementioned content to generate a conclusion that flows naturally with the reasoning process, such as "Therefore, there are a total of 6 that meet the color condition." The conclusion not only provides the variant answer but also briefly reiterates the key judgments made in the reasoning process. Furthermore, multiple candidate conclusions can be generated, and the best one, which best fits the variant question, variant answer, and reasoning process, can be selected as the final reasoning conclusion.

[0281] In addition, consistency checks can be performed on the reasoning conclusion and the variant answer, such as comparing whether the numerical value or category in the conclusion is consistent with the variant answer. If they are inconsistent, the conclusion content can be regenerated.

[0282] In this embodiment, the sample thought chain is organized into overall observation content, attribute identification content, task reasoning content, and reasoning conclusion content, and generated sequentially. This ensures that the sample thought chain fully covers the entire process from image observation, element identification, task-oriented reasoning to conclusion output, with each stage interconnected. The task reasoning content is grounded in the identified visual elements, providing clear visual evidence to support the reasoning process. The reasoning conclusion content remains consistent with the variant answers and concludes with a fixed marker. This provides a structurally complete and visually anchored reference reasoning chain for the scoring model to evaluate the prediction thought chain based on the sample thought chain, providing a reliable basis for the process scoring stage.

[0283] In one optional embodiment of this specification, the answer is read based on the initial question and the edited image to obtain the read answer, including: Using a first and second independent verification model, answers are read based on the edited image and the initial question, respectively, to obtain the first and second read answers. If the first read answer and the second read answer are consistent, and the output confidence of the first validation model and / or the second validation model meets the preset confidence threshold, then the read answer is determined.

[0284] The first verification model is a visual verification model used to read answers based on edited images and initial questions, and it is independent of the second verification model. Specifically, the first verification model can receive edited images and initial questions, and output the read answers and corresponding output confidence levels based solely on the image content and question information, such as outputting the answer "6" with a confidence level of "high". During the reading process, the correctness of the designed answer (i.e., the expected answer predetermined by the data construction process for the editing operation) is not disclosed; the model can only read the answer from the image. For example, it can be implemented using an open-source visual language model with visual understanding capabilities. Furthermore, the first and second verification models use different model instances to avoid sharing the same set of perceptual biases, providing independent sources of interpretation for dual-path comparison.

[0285] The second verification model is an independent visual verification model used to read answers based on edited images and initial questions. Specifically, the second verification model takes the edited image and initial questions as input and outputs the read answer and confidence level. The reading method is the same as the first verification model, i.e., the correctness of the designed answer is not disclosed; the answer is given solely based on the image content. The second verification model can use different model instances than the first verification model, such as visual language models with different parameter scales or from different sources, making the two reading results independent in perception and reasoning. Furthermore, the first and second verification models can be uniformly scheduled by the verifier, executing the reading independently and outputting the results to the acceptance criteria.

[0286] The first read answer is the answer obtained by the first verification model based on the edited image and the initial question. Specifically, the first read answer reflects what the first verification model actually observes in the edited image. For example, if the number of objects in the edited image is 6, the first read answer can be "6". The first read answer is directly output by the first verification model and can be given in text form or accompanied by structured representations such as numerical values ​​or categories. Furthermore, there is no hint association between the first read answer and the designed answer; its value depends entirely on the visual content in the edited image and the way the initial question is presented, constituting an independent read result in the dual-path comparison.

[0287] The second reading answer is the answer obtained by the second verification model based on the edited image and the initial question. Specifically, the second reading answer is independently output by the second verification model, reflecting what the model observes from the edited image. For example, if the number of objects in the edited image is 6, the second reading answer can be 6, or it can be different from the first reading answer. The consistency between the two reading results is used to measure the clarity of the answer in the edited image. Furthermore, the second reading answer and the first reading answer can be compared using semantic equivalence judgment, such as ignoring superficial differences like numerical format and capitalization to determine whether the two answers are consistent.

[0288] Output confidence is a measure of the credibility of the answer provided by the validation model when it outputs the read answer; it is directly output by the validation model. Specifically, output confidence can be divided into multiple levels, such as low, medium, and high. The validation model determines the confidence level based on its observation of the content of the edited image. For example, high confidence is given when the object is clearly visible and its quantity is clear, while low confidence is given when the image is blurry or the object is partially occluded. Confidence does not require additional calculation; it is directly provided by the model based on the input. Furthermore, the output confidence is compared with a preset confidence threshold to determine whether the read result reaches an acceptable level of credibility.

[0289] A pre-set reliability threshold is a benchmark value used to determine whether the output confidence of the validation model reaches an acceptable level. Specifically, the pre-set reliability threshold can be set to no less than a medium confidence level, meaning that reads with high or medium confidence are considered to meet the threshold, while reads with low confidence are considered to not meet the threshold. The threshold can be flexibly adjusted according to data quality requirements; for example, if the requirements are more stringent, it can be set to only accept high confidence levels. Furthermore, the pre-set reliability threshold and the consistency judgment of the two read answers together constitute the acceptance criterion. The read answer is determined if the two answers are consistent and at least one of them meets the confidence threshold.

[0290] In practical applications, the first and second verification models, which are independent of each other, are used to read the answers based on the edited image and the initial question, respectively, and the first and second reading answers are obtained accordingly. This can be achieved through various methods.

[0291] One alternative approach is to input the edited image and the initial question into a first verification model and a second verification model, respectively, and have each model perform blind reading independently. Specifically, the edited image and the initial question can be copied as two sets of inputs and fed into the first and second verification models respectively. The two readings do not communicate with each other and are unaware of each other's results. Each model outputs the reading answer and confidence level based solely on the image content and question information; for example, the first verification model outputs "6" with high confidence, and the second verification model outputs "6" with medium confidence. During the reading process, the correctness of the designed answer (i.e., the expected answer predetermined by the data construction process for the editing operation) is not disclosed. Furthermore, the two readings can be executed in parallel or sequentially, with the results recorded as the first reading answer and the second reading answer, respectively.

[0292] Another alternative approach is to have two models sequentially read and output confidence scores. Specifically, the edited image and the initial question can be input into a first validation model to obtain the first read answer and its output confidence score. Then, the edited image and the initial question can be input into a second validation model to obtain the second read answer and its output confidence score. The two models use independent model instances; for example, they can use visual language models that emphasize fine-grained visual perception and those that emphasize counting accuracy, or they can use visual language models from different sources, ensuring that the two interpretations are independent of each other in terms of perceptual bias. Furthermore, the reading results are recorded in a structured format, such as recording the answer text and confidence level, for use in receiving the judgment criteria.

[0293] In addition, the two validation models can be configured to read answers using different input organization methods. For example, one validation model can receive a concatenated image and question input, while the other validation model can receive a separate image and question input, thereby enhancing the independence of the two reading results.

[0294] Once the first and second read answers are obtained, the read answer can be determined based on the first read answer, the second read answer, and the output confidence score.

[0295] One alternative approach is to use a combination of dual-path consistency and confidence threshold criteria to determine the answer to be read. Specifically, a consistency check can be performed on the first and second read answers, for example, using semantic equivalence to ignore superficial differences such as numerical format and case sensitivity. Then, the confidence scores of the two outputs are checked. If the two answers are consistent and at least one confidence score is not lower than a preset confidence threshold, the consistent answer is determined as the read answer. For example, if both answers are 6 and one has a high confidence score, the read answer is determined to be 6. Furthermore, if any criterion is not met, the record is discarded. For example, if the two answers are inconsistent or both confidence scores are lower than the threshold, the image can be regenerated, edited, and read again.

[0296] Another alternative approach is to combine the change objective and frozen attribute constraints to comprehensively determine the answer to be read. Specifically, based on the consistency of the two read answers and the achievement of the confidence threshold, it is also possible to check whether the read answer conforms to the change objective for the initial answer. For example, if the change objective is to decrease the quantity by one, the read answer should differ from the initial answer and reflect the change in quantity. A summary check can also be performed to see if the frozen attributes of the edited image are maintained, such as whether the key objects and spatial relationships carrying the answer logic are intact after editing. If the check passes, the read answer is determined. Furthermore, records that fail any check are discarded to prevent data with insufficient credibility from entering subsequent processes.

[0297] In addition, the answer to the reading can be determined by combining the difference in confidence between the two readings. For example, if the two results are consistent but the confidence is low, the reading or editing process can be re-executed.

[0298] In an optional embodiment of this specification, the verification results of the verification model can also be used in the training process. Specifically, the verification results can be used as the basis for selecting training samples, or they can be used as part of the training signal to participate in model training, so as to further improve the quality of training data and the training effect of the model. This optional implementation can be selected and enabled according to actual training needs.

[0299] In this embodiment, by using two independent verification models—a first and a second—to read answers based on the edited image and the initial question, respectively, and determining the read answer only when the two read answers are consistent and at least one output confidence level meets a preset confidence threshold, the interpretation of the edited image's answer is expanded from a single source to two mutually corroborating independent sources. Only when the two results are consistent and the confidence level meets the threshold is the read answer accepted, ensuring its credibility. The confidence level is directly output by the verification model, making the criterion simple and easy to implement. Combined with the summary check of the changed target and frozen attribute constraints, unqualified edit records are removed before entering subsequent processes, providing a reliable answer foundation for the construction of the sample dataset.

[0300] In an optional embodiment of this specification, after obtaining the trained visual reasoning model, the method further includes: obtaining an evaluation dataset, wherein the evaluation dataset includes multiple evaluation samples, each evaluation sample includes an evaluation image, an evaluation question, and an evaluation answer, and the visual rules in the evaluation image conflict with the preset common sense prior. Using the trained visual reasoning model, reasoning is performed based on the evaluation images and evaluation questions to obtain the predicted evaluation answers for each evaluation sample; The evaluation result of the trained visual reasoning model is determined based on the matching results between the predicted evaluation answer and the corresponding evaluation answer.

[0301] The evaluation dataset is a collection of samples used to evaluate the reasoning ability of a visual reasoning model. Specifically, the evaluation dataset includes multiple evaluation samples, each consisting of an evaluation image, an evaluation question, and an evaluation answer. The visual rules in the evaluation images conflict with pre-set common sense priors, preventing the model from answering correctly based solely on linguistic priors. The evaluation dataset can be constructed using a process consistent with that of the sample dataset, such as seed collection, variant generation, and quality verification. The number of evaluation samples can be configured according to evaluation needs. Furthermore, the evaluation dataset is completely isolated from the training data source. Evaluation samples are constructed and stored in an independent pipeline branch and are never accessed during training, ensuring evaluation independence.

[0302] Evaluation samples are the basic data units in the evaluation dataset. Specifically, an evaluation sample consists of three parts: an evaluation image, an evaluation question, and an evaluation answer. The evaluation image presents counterintuitive visual rules, the evaluation question is an open-ended reasoning question designed based on the evaluation image, and the evaluation answer is the standard answer to that question. The model performs reasoning based on the evaluation image and the evaluation question, and the output result is compared with the evaluation answer to measure the model's performance on that sample. Furthermore, each evaluation sample can include difficulty stratification information, such as reasoning complexity level and visual interference intensity level, for statistical evaluation results stratified by difficulty.

[0303] The evaluation image is the visual portion of the evaluation sample, and the visual rules presented in the image conflict with pre-set common sense priors. Specifically, the evaluation image can be generated following a process consistent with the construction of the training sample images. The only difference is that the verification stage does not constrain the generated result to be a natural image, or it constrains it to be an unnatural image. For example, it can generate counterintuitive scenes such as a lion with eight legs. The rules in the image constitute the unique correct basis, and purely linguistic priors will inevitably lead to an incorrect answer. Furthermore, the fixed image attributes of the evaluation image are consistent with those of the seed image, and the logical structure of the same question remains unchanged across all visual variations.

[0304] The evaluation question is the question text in the evaluation sample, which is used to describe the reasoning task that needs to be answered based on the content of the evaluation image. Specifically, the evaluation question is designed according to the visual rules in the evaluation image. For example, it can put forward a question in the form of "How many legs in the image step on the white floor". The question adopts open-ended generation for answering, provides no options or auxiliary prompts, and eliminates guesswork and option elimination methods; the evaluation question can be obtained by instantiating seed questions through templates, and the same logical structure corresponds to multiple variants of evaluation images, so that the performance difference can be attributed to visual understanding rather than question difficulty drift. Furthermore, the evaluation questions cover multiple disciplines such as mathematics, physics and engineering, reasoning and logic, perception, and natural science.

[0305] The evaluation answer is the standard answer text in the evaluation sample. Specifically, the evaluation answer is determined according to the visual rules in the evaluation image, which is different from the answer given by common sense prior. For example, for a lion with 8 legs, the number of legs that actually contact the white floor in the image is the evaluation answer. The evaluation answer is used for matching judgment with the predicted evaluation answer. Furthermore, the evaluation answer can be manually determined by annotators according to the evaluation image, and can also be automatically derived after combining the answer effect of editing operators and then verified by annotators. For example, when an object that meets the counting condition is removed, the answer decreases accordingly.

[0306] The preset common sense prior is an inherent knowledge expectation formed for common situations. Specifically, the preset common sense prior includes the general understanding of common object attributes and relations in the real world, for example, lions usually have 4 legs and birds usually fly in the sky. Models also carry such priors after being trained on large-scale corpora; the visual rules in the evaluation image conflict with the preset common sense prior, so that models that simply rely on the prior to answer will definitely give wrong answers. For example, common sense will answer 4 legs, while the actual number in the image is 5. Furthermore, by creating conflicts between visual rules and common sense priors, it can be distinguished whether the model reasons based on the image content or answers based on the text prior in memory.

[0307] The predicted evaluation answer is the answer result output by the visual reasoning model for the evaluation sample. Specifically, using the trained visual reasoning model, reasoning is performed based on the visual content of the evaluation image and the evaluation question. The model outputs the answer content after generating the reasoning process text, and this answer is the predicted evaluation answer, which is used for matching judgment with the evaluation answer to measure the answering correctness of the model on the evaluation sample. Furthermore, the predicted evaluation answer can be in multiple forms such as numerical value, category or text. During matching judgment, superficial differences such as numerical format and case are ignored. For example, "4" and "four" are regarded as equivalent.

[0308] The evaluation results are a comprehensive measure of the visual reasoning model's performance on the evaluation dataset. Specifically, the evaluation results can be determined based on the matching results between the predicted evaluation answers and the actual evaluation answers for each evaluation sample. For example, the overall score can be calculated as the proportion of successfully matched samples to the total number of evaluation samples, or the matching success rate for each difficulty level can be calculated as a stratified score to observe the changes in model behavior with task complexity. Furthermore, the evaluation results can be output in report form, including the overall score, stratified scores, and representative examples, for reference in model release, version comparison, and iterative training.

[0309] In practical applications, evaluation datasets can be obtained through various methods.

[0310] One alternative approach is to construct the evaluation dataset following a process consistent with that used for the sample dataset. Specifically, STEM (Science, Technology, Engineering, and Mathematics) seed questions are collected from open educational resources, covering subject families such as mathematics, physics and engineering, reasoning and logic, perception, and natural sciences, as well as dozens of subfields. Multiple-choice questions are uniformly converted into open-ended questions. The same pipeline used in constructing the sample dataset is employed to generate evaluation images and corresponding questions. For example, an analyzer distinguishes between fixed and free image attributes, a planner generates value schemes, and an image editing model performs the generation. Evaluation questions and answers are obtained through template instantiation. The only difference from constructing the training data is that the validation phase does not constrain the evaluation images to be natural images, or it may constrain them to be non-natural images. Furthermore, multiple counterintuitive scene variations can be generated and aggregated into evaluation samples.

[0311] Another alternative approach is to determine the evaluation dataset through difficulty stratification and quality control. Specifically, difficulty is explicitly controlled along two dimensions: reasoning complexity and visual interference intensity. Reasoning complexity is determined by the number of frozen dimensions; more frozen dimensions mean more constraints need to be maintained. Visual interference intensity is determined by the magnitude of change in the free dimensions; the greater the magnitude of change, the stronger the interference. The intersection of these two dimensions forms a difficulty matrix, generating evaluation samples of multiple difficulty levels for each seed question. Furthermore, a dual quality check is performed, both manually and automatically. Each evaluation sample is verified by at least two independent annotators for correct answer, visual reasonableness, and retention of frozen attributes. Samples with discrepancies are discarded. The evaluation samples that pass the quality check are organized according to difficulty stratification to form the evaluation dataset.

[0312] In addition, manual verification can be used to conduct random checks on the evaluation dataset. For example, a predetermined ratio of evaluation samples can be drawn to verify the correctness of the answers and the rationality of the images. The dataset can only be put into use after the random check passes.

[0313] Once the evaluation dataset is available, the trained visual reasoning model can be used to reason based on the evaluation images and evaluation questions to obtain the predicted evaluation answers for each evaluation sample.

[0314] One alternative approach is to input the evaluation image and evaluation question into a trained visual reasoning model. The model then generates a reasoning response in an end-to-end manner and extracts the predicted evaluation answer from it. Specifically, the evaluation image is segmented into multiple image patches and encoded into a visual feature sequence. The evaluation question is lexicalized and concatenated with the visual feature sequence. This concatenation is then input into the model decoder to generate reasoning text word by word. The reasoning text includes multiple reasoning steps and the final answer. The answer content is extracted from the response based on fixed markers, such as responses ending with "Final Answer" or "...". <answer>When the "mark ends, extract the content after the mark as the predicted evaluation answer. Further, the inference process is automatically completed by the model, and all predicted evaluation answers can be obtained by performing inference sequentially on each evaluation sample.

[0315] Another alternative method is that the predicted evaluation answer can be determined through multiple sampling and consistency processing. Specifically, multiple inference samplings are performed for the same evaluation sample, for example, the sampling times are set to 4 or 8 times, to obtain multiple candidate answers; statistics and consistency analysis are performed on each candidate answer, for example, the answer with the highest occurrence frequency is selected as the predicted evaluation answer, and all multiple sampling results can also be retained for subsequent comprehensive determination according to the majority consistency criterion. Further, generation parameters such as decoding temperature can be set during the sampling process, for example, a lower temperature is set to obtain stable answer output.

[0316] In addition, the consistency of multiple samplings can also be combined to determine the predicted evaluation answer. For example, multiple inference samplings are performed on the same evaluation sample, and the answer with the highest occurrence frequency is taken as the predicted evaluation answer.

[0317] When the predicted evaluation answers corresponding to each evaluation sample are obtained, the evaluation result of the trained visual reasoning model can be determined based on the matching result of the predicted evaluation answers and the corresponding ground truth evaluation answers.

[0318] An alternative method is that semantic equivalence matching can be used to determine the answer correctness of each evaluation sample, and the overall score is counted. Specifically, for each evaluation sample, semantic equivalence judgment is performed between the predicted evaluation answer and the ground truth evaluation answer, and surface differences such as numerical format, uppercase and lowercase are ignored. For example, "4" and "four", full-width and half-width forms are all regarded as equivalent; a judged matching sample is recorded as one hit, the proportion of the number of hit samples to the total number of evaluation samples is counted as the overall score, and the hit rate of each difficulty level is counted separately in combination with difficulty layering information. Further, the overall score and the layered scores together constitute the evaluation result, which supports horizontal comparison of the visual reasoning capabilities of different models.

[0319] Another alternative method is that the evaluation result can be determined by combining the quality of the inference process on the basis of answer matching. Specifically, in addition to answer matching judgment, the inference process output by the model can also be evaluated, for example, checking whether the reasoning steps are derived based on the visual evidence in the evaluation image, and whether the reasoning chain corresponds to the content in the image; the answer matching result and the quality of the inference process are integrated, for example, the answer accuracy rate and the reasoning basis accuracy rate are counted respectively, forming a multi-dimensional evaluation result. Further, the evaluation result can be output as an evaluation report, including the overall score, layered scores and representative sample analysis, for reference in model selection and iterative optimization.

[0320] In addition, it can output detailed analysis information of the evaluation results, such as the answer records and representative error examples of each evaluation sample, for reference in model iteration and optimization.

[0321] In the embodiments of this specification, an evaluation dataset containing evaluation images, evaluation questions, and evaluation answers is acquired. A trained visual reasoning model is used to reason based on the evaluation images and evaluation questions to obtain predicted evaluation answers. The evaluation result is then determined based on the matching results between the predicted evaluation answers and the actual evaluation answers. This causes the visual rules in the evaluation images to conflict with the preset common sense priors. The model cannot rely on language priors or template matching to guess the correct answer. It needs to reason based on the image content to obtain the correct result. This allows for an accurate distinction between whether the model relies on visual evidence for reasoning or on textual priors based on memory. The evaluation results support reporting methods for overall scores and scores stratified by difficulty, which facilitates the observation of the changes in model behavior with task complexity and provides a reference for model selection, version comparison, and iterative training.

[0322] The following is in conjunction with the appendix Figure 3a To be continued Figure 3b Taking the construction process of the sample dataset as an example, this document further explains the training data construction method provided. Figure 3a This specification shows a flowchart of the first stage of a training data construction method according to an embodiment of the present specification; Figure 3b A flowchart illustrating the second-stage processing of a training data construction method provided in one embodiment of this specification is shown.

[0323] Specifically, Figure 3a The diagram illustrates the first phase of the data construction process. First, visual understanding analysis is performed to obtain the initial image, initial question, and initial answer, determining the global reasoning focus. This global reasoning focus includes the task type, scope, target entity, and reasoning requirements, and is understood in the order of observing the overall image, identifying the target entity, and performing reasoning. Next, visual attributes are categorized into fixed image attributes and free image attributes. Fixed image attributes may include, for example, object identity, location, spatial relationships, and raw semantic counts. Free image attributes may include, for example, lighting, viewpoint, color and texture, background, and text. Then, editing operations are determined. Editing operations can include types such as adding, removing, modifying, and associating. The diagram uses the adding operation as an example. The editing operation determines the target object and constraints based on the changed target of the initial answer, while keeping the fixed image attributes unchanged, and updates the initial answer to the target answer corresponding to the changed target. Subsequently, controlled image editing is performed on the initial image to obtain the edited image. Finally, the answer is verified and output. The edited image is verified, including whether the answer has changed, whether the scene is still reasonable, and whether the fixed image attributes are retained. After the verification is passed, the edited seed sample is output. The edited seed sample includes the edited image, the initial question, the read answer, and the preliminary thought chain outline.

[0324] Specifically, Figure 3b The diagram illustrates the second-stage data construction process. The first half involves axial diversification and variant image generation. While maintaining the fixed image attributes of the edited image, the free image attributes are adjusted. These free image attributes can include, for example, lighting, weather, viewpoint, background, color, style, object appearance, and scene context, to introduce diverse and reasonable variants and expand the coverage of the key visual axis. Multiple variant images are then generated, such as those depicting a bright morning, a rainy day, a wider viewpoint, a warm dusk, different background environments, and rearranged objects. Each variant image has a different appearance but the same variant answer. The second half involves constructing sample thought chains and outputting sample datasets. It sequentially generates overall observation content, attribute recognition content, task reasoning content, and reasoning conclusion content, using variant answers as responses. Finally, a sample dataset is output, with each sample data point including a sample image, a sample question, a sample answer, and a sample thought chain.

[0325] In the embodiments of this specification, by means of Figure 3a and Figure 3b The training data construction process shown in the diagram involves two stages. In the first stage, controlled editing is performed on the initial image based on the changed target. The verification stage confirms that the answer has changed, the scene is reasonable, and the fixed image attributes are maintained. This allows the changed answer to be mechanically derived from the editing operation, enabling large-scale production without manual annotation. In the second stage, while keeping the fixed image attributes unchanged, the attributes of the free image are diversified and extended to generate variant images with different appearances but the same answer. A four-step sample thinking chain is constructed for each variant image, so that the same logical structure can cover multiple visual scenes. This provides the model training with multimodal thinking chain training data that is visually diverse, has reliable labels, and has a complete inference chain.

[0326] This specification also provides an embodiment of a visual reasoning method. Specifically, see [link to embodiment]. Figure 4 , Figure 4 A flowchart of a visual reasoning method according to an embodiment of this specification is shown, such as... Figure 4 As shown, the method specifically includes the following steps.

[0327] Step 402: Obtain the target image and target question to be reasoned; Step 404: Using a visual reasoning model, reason based on the target image and the target question to obtain the target answer and the corresponding target thought chain. The visual reasoning model is trained according to the model training method described above.

[0328] The target image is the image input for the user's reasoning. Specifically, the target image can be any image uploaded by the user or selected from the application scenario, such as an illustration in the text and image content of a community platform, an illustration for a question in an educational scenario, or a screen to be reviewed in a content review task. The image content contains visual evidence needed to answer the target question, and the target image and the target question are interconnected, jointly constituting the input of the visual reasoning model. Furthermore, the target image can come from various channels, such as being taken by the user, retrieved from a database, or pushed by an upstream system, and the image format and size can be flexibly adjusted.

[0329] The target question is an unanswerable question posed by the user in response to a target image. Specifically, the target question describes a reasoning task that requires an answer based on the content of the target image. Examples include questions about the number of objects in the image / text, math word problems in educational scenarios, and auditing inquiries requiring judgment based on visual information. The target question is expressed in natural language and corresponds to the key content of the target image; the information tested can be found visually in the target image. Furthermore, the target question can be directly input by the user or automatically generated or rewritten by the application system based on business needs.

[0330] The visual reasoning model is a reasoning model trained using the aforementioned model training methods. Specifically, the visual reasoning model is a trained visual reasoning model that possesses the ability to reason step-by-step based on image evidence. During the reasoning process, it progressively references specific content from the image as the basis for deduction, first outputting the text of the reasoning process, and then outputting the final answer. The reasoning chain maintains a correspondence with the actual content of the image. The model can be deployed on application servers, such as image-text question-and-answer services, educational problem-solving services, or content moderation assistance services. Furthermore, the visual reasoning model can be trained using the aforementioned model training methods, or multiple trained model instances can form a service cluster to support high-concurrency reasoning requests.

[0331] The target answer is the response output by the visual reasoning model to the target image and the target question. Specifically, the target answer is the final result given by the model after completing the reasoning process. It can be expressed in the form of numerical values, categories, or textual descriptions, such as the calculation result of a math problem, a judgment on the content of an image, or a brief answer to a user's question. The target answer follows the target thought chain and together with the target thought chain, constitutes the model's complete response. Furthermore, the target answer can be presented directly to the user or used as a processing basis for downstream applications, such as providing the answer to a content moderation system to perform subsequent judgments.

[0332] The target thought chain is the reasoning process text generated by the visual reasoning model before outputting the target answer. Specifically, the target thought chain includes multiple reasoning steps, such as observing the overall structure of the image, identifying key elements, step-by-step reasoning towards the target question, and the final conclusion. Each step is presented in natural language, progressing layer by layer, fully recording the model's process of deriving the target answer from image evidence. The reasoning conclusion can be found in the image. Furthermore, the target thought chain can be presented to the user along with the target answer to help understand the origin of the answer, or it can be extracted separately for tracing and analyzing the reasoning process.

[0333] In practical applications, the target image and target question to be reasoned can be obtained through various methods.

[0334] One alternative approach is for users to directly provide the target image and the target question. Specifically, the application interface receives the user-uploaded image file and the input question text. For example, in a text-based Q&A application, it receives the user-uploaded image and question; in an educational problem-solving application, it receives a photo of the question and the solution requirements taken by the user. After receiving the input, format validation and preprocessing are performed, such as standardizing image size, resolution, and text encoding, to obtain the target image and the target question. Furthermore, semantic analysis can be performed on the target question to identify key information being tested, facilitating the reasoning process to focus on relevant visual content in the image.

[0335] Another alternative approach is to automatically acquire target images and questions from the application scenario. Specifically, this involves extracting the images and questions to be reasoned from business data; for example, extracting accompanying images and corresponding text questions from text and image content on a community platform, extracting questions and their accompanying images from a knowledge point question bank on an educational platform, and obtaining images and review inquiries to be reviewed from content review tasks. The images and questions are then linked as input pairs, and deduplication and filtering are performed to remove inputs with missing content or substandard quality, thus obtaining the target images and target questions. Furthermore, automatic acquisition can be performed in batches, supporting continuous inference services for large amounts of text and image content.

[0336] In addition, target images and target questions can be automatically obtained in combination with business scenarios. For example, the application system can automatically select and push image-text pairs to be reasoned from the data source according to business needs.

[0337] Once the target image and target question are obtained, a visual reasoning model can be used to reason based on the target image and target question to obtain the target answer and the corresponding target thought chain.

[0338] One alternative approach is to input the target image and target question into a visual reasoning model, which then generates the target thought chain and target answer end-to-end. Specifically, the target image is encoded as a visual feature sequence, the target question is lexically converted and concatenated with the visual feature sequence, and then input into the model decoder to progressively generate reasoning text. The reasoning text begins with the content of the target thought chain and ends with the target answer. Responses are segmented based on fixed markers; for example, content before the "Final Answer" marker is identified as the target thought chain, and content after the marker is identified as the target answer. Furthermore, during the generation process, the model progressively verifies the visual evidence in the image to ensure that each reasoning step corresponds to the image content.

[0339] Another alternative approach is to generate the target thought chain and target answer in stages and perform post-processing. Specifically, the model first generates the target thought chain based on the target image and target question, and then determines the target answer based on the conclusion of the target thought chain. Subsequently, post-processing is performed on the output, such as standardizing the answer expression, removing redundant markers, verifying the consistency between the target answer and the conclusion of the target thought chain, and triggering re-reasoning or prompting the user to supplement information if post-processing fails. Furthermore, post-processing can be performed by rule-based procedures or assisted by a lightweight model, and the processed target answer and target thought chain are output to downstream applications.

[0340] In addition, the level of detail in the reasoning output can be controlled according to user needs. For example, a concise thought process can be output when a user needs a brief answer, and a complete reasoning process can be output when a detailed explanation is required.

[0341] In a specific application scenario, the visual reasoning method provided in the embodiments of this specification can also be applied to content review assistance scenarios. For example, the text and image content to be reviewed can be input into the visual reasoning model as the target image and target question. The model can then reason based on the image content and the review question, and output a judgment result containing the reasoning basis. This helps reviewers verify the relationship between the image content and the question, thereby improving review efficiency and the interpretability of the results.

[0342] In the embodiments of this specification, by acquiring the target image and target question to be reasoned about, and using the visual reasoning model trained according to the above-described model training method, reasoning is performed based on the target image and target question to obtain the target answer and the corresponding target thought chain. This allows users to directly obtain reasoning results for specific image content. The reasoning process is presented completely in the form of a target thought chain, and the origin of the answer is clear and traceable. Since the visual reasoning model is trained through reinforcement learning with image evidence as anchor, the reasoning steps correspond to the actual content of the image, ensuring the correctness and interpretability of the target answer. Application scenarios such as image-text question answering, mathematical problem solving, and content review assistance can obtain visualized and verifiable reasoning services, improving the credibility of visual reasoning results and user experience.

[0343] Corresponding to the above method embodiments, this specification also provides embodiments of a model training apparatus. Specifically, see... Figure 5 , Figure 5 A schematic diagram of a model training apparatus according to one embodiment of this specification is shown. Figure 5 As shown, the model training device includes: The first acquisition module 502 is configured to acquire a sample dataset, wherein each sample data in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain; The first reasoning module 504 is configured to use the visual reasoning model to be trained to reason based on sample images and sample questions to obtain the predicted answer and the corresponding predicted thought chain. Evaluation module 506 is configured to use a pre-trained scoring model to evaluate whether the conclusions of each reasoning step in the predicted thinking chain are derived from the preceding reasoning steps and visual evidence in the sample image, based on the sample thinking chain, and obtain a logical validity score. Training module 508 is configured to train the visual reasoning model to be trained based on logical validity scoring, and obtain the trained visual reasoning model.

[0344] The first acquisition module is a module in the model training device used to acquire sample datasets. Specifically, the first acquisition module can be configured to read sample datasets from local storage media, distributed file systems, or cloud databases, or it can receive newly generated sample data from the data construction pipeline. Each sample dataset includes sample images, sample questions, sample answers, and sample thought chains. Furthermore, the first acquisition module can also perform integrity checks on the sample data, or filter sample data according to question type or data source to obtain sample datasets that meet the training requirements.

[0345] The first inference module is the module in the model training device that calls the visual inference model to be trained to perform inference. Specifically, the first inference module can acquire sample images and sample questions, segment the sample images into image blocks and encode them as visual features, and input them into the visual inference model along with the text content of the sample questions. The model then generates a predicted thought chain and the final predicted answer in an autoregressive manner. Furthermore, the first inference module can also segment the model response into inference steps and answer parts, which are provided to the evaluation module and the training module.

[0346] The evaluation module, located within the model training device, is used to assess the quality of the predictive thought chain reasoning. Specifically, the evaluation module can invoke a pre-trained scoring model, using the sample thought chain as a benchmark, to check step-by-step whether the conclusions of each reasoning step in the predictive thought chain are derived from preceding reasoning steps and visual evidence in the sample image, summarizing the results to obtain a logical validity score. Furthermore, the evaluation module can organize the sample image, sample thought chain, predictive thought chain, and scoring instructions into a scoring prompt input to the scoring model, which then outputs scores for each step; for example, scores can range from 0 to 1.

[0347] The training module is a component of the model training device used to update model parameters based on the logic validity score. Specifically, the training module can use the logic validity score as a training signal to update the parameters of the visual reasoning model to be trained, or it can weight the logic validity score with other reward components to determine a comprehensive training signal before updating the model parameters. For example, it can use reinforcement learning algorithms such as group relative policy optimization to update the parameters. Furthermore, the training module can train in multiple iterations, and the completed visual reasoning model is determined after satisfying preset convergence conditions.

[0348] Specifically, the first acquisition module can perform the function of acquiring the sample dataset in the above method embodiment, acquiring multiple sample data from sources such as storage systems or data construction pipelines and aggregating them into a sample dataset; the sample dataset can be collaboratively constructed by functional modules such as analyzer, director, generator, planner and validator, and the model training device can also include analysis module, editing module and verification module, etc., to perform analysis, editing generation and quality verification.

[0349] The first reasoning module can perform the steps of reasoning using the visual reasoning model to be trained in the above method embodiment. It inputs the sample image and sample question into the visual reasoning model, and the model generates the predictive thought chain and predictive answer in an end-to-end autoregressive manner. Alternatively, it can first extract the visual features of the sample image and then combine them with the sample question to perform multi-step reasoning, and output the reasoning result to the evaluation module.

[0350] The evaluation module can perform the function of evaluating the logical validity using the pre-trained scoring model in the above method embodiment. Based on the sample thinking chain, it evaluates step by step whether the conclusion of each reasoning step in the prediction thinking chain is derived from the preceding reasoning steps and the visual evidence in the sample image, and obtains a logical validity score. For example, it can use methods such as premise-conclusion comparison, organizing scoring prompts, or generating explanations of derivation relationships.

[0351] The training module can perform the function of training based on logical validity scores in the above method embodiments. It uses logical validity scores as reward signals to train the visual reasoning model to be trained. For example, it can directly use logical validity scores as reinforcement learning reward signals to update model parameters, or it can combine answer matching rewards and format rewards to determine a comprehensive training signal and then update model parameters. After multiple rounds of iteration to meet the preset convergence conditions, the trained visual reasoning model is determined.

[0352] The model training device provided in this specification acquires a sample dataset containing sample images, sample questions, sample answers, and sample thought chains through a first acquisition module. A first inference module infers the predicted answer and the predicted thought chain. An evaluation module evaluates the logical validity of each inference step based on the sample thought chain. A training module updates the parameters of the visual inference model based on the logical validity score, enabling fine-grained supervision of the quality of each step of the inference process. This guides the model to anchor the inference steps to visual evidence in the sample images, improving the reliability and interpretability of the inference process. This device can be deployed on a cloud server cluster or a local training platform to meet the model training needs in various scenarios.

[0353] The above is an illustrative scheme of a model training device according to this embodiment. It should be noted that the technical solution of this model training device and the technical solution of the above-described model training method belong to the same concept. For details not described in detail in the technical solution of the model training device, please refer to the description of the technical solution of the above-described model training method.

[0354] Corresponding to the above method embodiments, this specification also provides embodiments of a visual reasoning device. Specifically, see... Figure 6 , Figure 6 A schematic diagram of the structure of a visual reasoning device according to one embodiment of this specification is shown. Figure 6 As shown, the visual reasoning device includes: The second acquisition module 602 is configured to acquire the target image and target question to be reasoned; The second reasoning module 604 is configured to use a visual reasoning model to reason based on the target image and the target question to obtain the target answer and the corresponding target thought chain. The visual reasoning model is trained according to the model training method described above.

[0355] The second acquisition module is a module in the visual reasoning device used to acquire the target image and target question to be reasoned about. Specifically, the second acquisition module can receive images and questions uploaded by users, or it can read corresponding target images and target questions from business systems or storage locations, such as images and questions from text and image content on a community platform. Furthermore, the second acquisition module can perform preprocessing on the target image and target question, such as decoding, scaling, and format conversion of the image, and cleaning the question text, before providing it to the second reasoning module.

[0356] The second reasoning module is a component of the visual reasoning device used to perform visual reasoning based on the target image and the target question. Specifically, the second reasoning module can call the visual reasoning model trained according to the aforementioned model training method, input the target image and the target question into the model, and the model generates the target thought chain and the final target answer. Furthermore, the second reasoning module can segment the model response, extract the target thought chain and the target answer, and return them together to the requester for them to view the reasoning basis.

[0357] Specifically, the second acquisition module can perform the function of acquiring the target image and target question to be inferred in the above method embodiment, and acquire the corresponding target image and target question from the image acquisition device, user terminal, business system or cloud storage. The acquisition method may include real-time reception and batch import. After acquisition, the image and question are preprocessed such as decoding, scaling and cleaning, and the processing results are passed to the second inference module.

[0358] The second reasoning module can perform the steps of reasoning using a visual reasoning model in the above method embodiment. It inputs the target image and target question into the visual reasoning model, and the model reasones step by step based on the visual evidence in the target image, outputting the target thought chain and the target answer. The visual reasoning model is trained according to the above model training method and has the ability to reason step by step based on image evidence. The reasoning result is returned to the user terminal or business system for use.

[0359] The visual reasoning device provided in this specification acquires a target image and a target question through a second acquisition module, and then the second reasoning module calls the visual reasoning model trained according to the above-mentioned model training method to perform reasoning, thereby obtaining the target answer and the corresponding target thought chain. This enables the device to quickly output a reasoning result containing the complete reasoning process. Since the reasoning steps are anchored to visual evidence in the target image, the target thought chain output by the device is clear, coherent, and traceable, making it easy to verify the reasoning basis. This device can be deployed on cloud server clusters or terminal devices to meet the visual reasoning needs in various scenarios.

[0360] The above is an illustrative scheme of a visual reasoning device according to this embodiment. It should be noted that the technical solution of this visual reasoning device and the technical solution of the above-described visual reasoning method belong to the same concept. For details not described in detail in the technical solution of the visual reasoning device, please refer to the description of the technical solution of the above-described visual reasoning method.

[0361] Figure 7 A structural block diagram of a computing device 700 according to one embodiment of this specification is shown. The components of the computing device 700 include, but are not limited to, a memory 710 and a processor 720. The processor 720 is connected to the memory 710 via a bus 730, and a database 750 is used to store data.

[0362] The computing device 700 also includes an access device 740, which enables the computing device 700 to communicate via one or more networks 760. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 740 may include one or more of any type of wired or wireless network interface (e.g., a network interface controller (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Worldwide Interoperability for Microwave Access (WiMAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0363] In one embodiment of this specification, the above-described components of the computing device 700 and Figure 7 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 7 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0364] The computing device 700 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 700 can also be a mobile or stationary server.

[0365] The processor 720 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above method.

[0366] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computing device can be referred to the description of the technical solution of the above method.

[0367] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0368] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the storage medium can be referred to the description of the technical solution of the above method.

[0369] An embodiment of this specification also provides a computer program product, including computer-executable instructions that, when executed by a processor, implement the steps of the above-described method.

[0370] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the above method belong to the same concept, and all details not described in detail in the technical solution of the computer program product can be referred to in the description of the technical solution of the above method.

[0371] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0372] Computer-executable instructions include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. Computer-readable storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in computer-readable media can be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0373] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0374] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0375] The preferred embodiments disclosed above are merely illustrative of this specification. Optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.< / answer> < / answer>

Claims

1. A model training method, characterized in that, include: Obtain a sample dataset, wherein each sample data item in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought process chain; Using the visual reasoning model to be trained, reasoning is performed based on the sample images and sample questions to obtain the predicted answer and the corresponding predicted thought chain; Using a pre-trained scoring model, based on the sample thought chain, the conclusions of each reasoning step in the predicted thought chain are derived from the preceding reasoning steps and the visual evidence in the sample image, and a logical validity score is obtained. Based on the logical validity score, the visual reasoning model to be trained is trained to obtain a trained visual reasoning model.

2. The method according to claim 1, characterized in that, Before training the visual reasoning model to be trained based on the logical validity score to obtain the trained visual reasoning model, the method further includes: Using the scoring model, based on the semantic matching results between the reasoning steps in the sample thinking chain and the predicted thinking chain, the coverage of the predicted thinking chain to each reference reasoning step in the sample thinking chain is determined, and a reasoning completeness score is obtained. Using the scoring model, a process consistency score is obtained by scoring based on whether there is a logical contradiction between adjacent reasoning steps in the predictive thought chain. The step of training the visual reasoning model to be trained based on the logical validity score to obtain the trained visual reasoning model includes: The logical validity score, the reasoning completeness score, and the process consistency score are weighted and aggregated to obtain the reasoning process reward. Based on the reward for the reasoning process, the visual reasoning model to be trained is trained to obtain a trained visual reasoning model.

3. The method according to claim 2, characterized in that, Before training the visual reasoning model to be trained based on the reward of the reasoning process to obtain the trained visual reasoning model, the method further includes: The answer reward is determined based on the predicted answer and the sample answer, and the format reward is determined based on the predicted answer and the predicted thought chain. The step of training the visual reasoning model to be trained based on the reward for the reasoning process to obtain the trained visual reasoning model includes: The total reward is determined based on the answer reward, the reasoning process reward, and the format reward, and the visual reasoning model to be trained is trained based on the total reward to obtain the trained visual reasoning model.

4. The method according to claim 3, characterized in that, The determination of the total reward based on the answer reward, the reasoning process reward, and the format reward includes: Determine the answer weight corresponding to the answer reward, the reasoning process weight corresponding to the reasoning process reward, and the format weight corresponding to the format reward, wherein the answer weight is greater than the reasoning process weight, and the reasoning process weight is greater than the format weight; Based on the answer weight, the reasoning process weight, and the format weight, the answer reward, the reasoning process reward, and the format reward are weighted and summed to obtain the total reward.

5. The method according to claim 3, characterized in that, The step of training the visual reasoning model to be trained based on the total reward to obtain the trained visual reasoning model includes: Using a group relative strategy optimization algorithm, a set of candidate responses is generated based on the sample images and the sample questions, using the visual reasoning model to be trained; Based on the total reward, determine the relative strength of each candidate response within the set of candidate responses; Based on the aforementioned relative advantages, the parameters of the visual reasoning model to be trained are updated to obtain the trained visual reasoning model.

6. The method according to any one of claims 1-5, characterized in that, Before obtaining the sample dataset, the following is also included: Obtain an initial dataset, wherein the initial dataset includes an initial image, an initial question, and an initial answer; Based on the change target of the initial answer, the initial image is edited to obtain an edited image; Based on the initial question and the edited image, the answer is retrieved to obtain the retrieved answer; If the read answer meets the change target, the sample dataset is determined based on the edited image, the initial question, and the read answer.

7. The method according to claim 6, characterized in that, The step of editing the initial image based on the change target of the initial answer to obtain an edited image includes: Analyze the initial image to determine its fixed image attributes and free image attributes; Based on the change target for the initial answer, a first editing operation is determined for the fixed image attributes of the initial image; Perform the first editing operation on the initial image to obtain the edited image.

8. The method according to claim 7, characterized in that, The analysis of the initial image to determine its fixed and free image attributes includes: Based on multiple preset semantic axes, the initial image is decomposed into image attributes, and combined with the test points of the initial question, the attribute state labels corresponding to the image attributes on each preset semantic axis are determined. The attribute state labels include frozen state, free state, and condition-dependent state. The image attributes in the frozen state and the condition-dependent state are determined as the fixed image attributes of the initial image, and the image attributes in the free state are determined as the free image attributes of the initial image.

9. The method according to claim 6, characterized in that, The process of determining the sample dataset based on the edited image, the initial question, and the read answer includes: Analyze the edited image to determine its fixed image attributes and free image attributes; While keeping the fixed image attributes of the edited image unchanged, determine a second editing operation for the free image attributes of the edited image; Perform the second editing operation on the image to obtain multiple variant images; Based on the initial question and the read answer, determine the variant question and variant answer corresponding to each variant image; The variant images are verified based on a preset verification dimension. If the verification is successful, a sample thought chain corresponding to each variant image is generated based on each variant image and its corresponding variant question and variant answer. Using each of the variant images as sample images, the variant questions as sample questions, and the variant answers as sample answers, the sample thought chain is combined to determine the sample dataset.

10. The method according to claim 9, characterized in that, The sample thought chain includes overall observation content, attribute identification content, task reasoning content, and reasoning conclusion content; The step of generating sample thought chains corresponding to each variant image based on each variant image and its corresponding variant questions and variant answers includes: Based on each variant image and its corresponding variant title, the overall observation content and attribute recognition content are generated, wherein the overall observation content is used to describe the overall structure and object scale of each variant image, and the attribute recognition content is used to characterize the image attributes related to the variant title; Based on the image attributes included in the attribute recognition content, the task reasoning content is generated, wherein the task reasoning content includes reasoning operations to be executed corresponding to the variant question; Based on the variant answer, the reasoning conclusion is generated.

11. The method according to claim 6, characterized in that, The step of reading the answer based on the initial question and the edited image to obtain the read answer includes: Using a first verification model and a second verification model that are independent of each other, answers are read based on the edited image and the initial question, respectively, to obtain the first read answer and the second read answer. The answer to be read is determined when the first read answer and the second read answer are consistent, and the output confidence of the first verification model and / or the second verification model meets the preset confidence threshold.

12. The method according to any one of claims 1-5, characterized in that, After obtaining the trained visual reasoning model, the following is also included: Obtain an evaluation dataset, wherein the evaluation dataset includes multiple evaluation samples, each evaluation sample includes an evaluation image, an evaluation question, and an evaluation answer, and the visual rules in the evaluation image conflict with the preset common sense priors; Using the trained visual reasoning model, reasoning is performed based on the evaluation image and the evaluation question to obtain the predicted evaluation answer corresponding to each evaluation sample; Based on the matching results between the predicted evaluation answer and the corresponding evaluation answer, the evaluation result of the trained visual reasoning model is determined.

13. A visual reasoning method, characterized in that, include: Obtain the target image and target question to be reasoned; Using a visual reasoning model, reasoning is performed based on the target image and the target question to obtain the target answer and the corresponding target thought chain, wherein the visual reasoning model is trained according to the model training method described in any one of claims 1-12.

14. A model training device, characterized in that, include: The first acquisition module is configured to acquire a sample dataset, wherein each sample data in the sample dataset includes a sample image, a sample question, a sample answer, and a sample thought chain; The first reasoning module is configured to use a visual reasoning model to be trained to reason based on the sample image and the sample question to obtain a predicted answer and a corresponding predicted thought chain. The evaluation module is configured to use a pre-trained scoring model to evaluate whether the conclusions of each reasoning step in the predicted thinking chain are derived from the preceding reasoning steps and the visual evidence in the sample image, based on the sample thinking chain, and obtain a logical validity score. The training module is configured to train the visual reasoning model to be trained based on the logical validity score, so as to obtain the trained visual reasoning model.

15. A visual reasoning device, characterized in that, include: The second acquisition module is configured to acquire the target image and target question to be reasoned. The second reasoning module is configured to use a visual reasoning model to reason based on the target image and the target question to obtain the target answer and the corresponding target thought chain, wherein the visual reasoning model is trained according to the model training method described in any one of claims 1-12.

16. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 13.

17. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

18. A computer program product, characterized in that, It includes computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images