Positioning method and device based on multi-image reasoning

Abstract

The invention provides a positioning method and device based on multi-image reasoning, and relates to the technical field of artificial intelligence. The method comprises the following steps: rewriting original multi-graph data into a sample triple; preprocessing the sample triple to construct a training data set; screening the training data set to construct a cold start set; utilizing the training data set to supervise and finely adjust the multi-modal visual language model to obtain an initial multi-image reasoning model; utilizing the cold start set to supervise and finely adjust the initial multi-image reasoning model to obtain a basic multi-image reasoning model; performing reinforcement learning on the basic multi-image reasoning model by utilizing the training data set according to a preset learning value dynamic filtering mechanism and / or a difficulty perception course current candidate pool mechanism to obtain a target multi-image reasoning model; and inputting the to-be-processed multi-image data into the target multi-image reasoning model to obtain a reasoning positioning result, thereby realizing stable, complete, good-interpretability and generalizable positioning in a multi-image and multi-target output scene.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, specifically to a localization method and apparatus based on multi-image reasoning. Background Technology

[0002] Currently, multimodal large models have made significant progress in visual understanding and language reasoning. However, real-world applications increasingly require models to handle multi-image inputs: such as multi-view photography, continuous frame sequences, comparison of different viewpoints of the same scene, and cross-image referencing and contrastive localization across multiple images. In such scenarios, the system must not only "answer correctly" but also output executable pixel-level / region-level localization results, ensuring that each localization result corresponds to the correct image number or image label. This type of task belongs to Multi-Image Reasoning Localization (MRG), and its core challenge lies in the fact that the model must establish evidentiary associations between multiple images, complete reasoning decisions, and stably map the reasoning conclusions to the spatial localization output.

[0003] Existing technical approaches mainly fall into two categories: end-to-end direct prediction and supervised fine-tuning / hinting engineering incorporating thought chains. End-to-end direct prediction is prone to drift problems in multi-graph localization, where the semantics are correct but the localization is incorrect. While incorporating thought chains can improve reasoning performance to some extent, in the absence of verifiable constraints, the reasoning text may exhibit redundancy, drift, or pseudo-reasoning, or even prematurely reveal the conclusion in the reasoning section, thereby weakening the stability and interpretability of localization. Summary of the Invention

[0004] Based on this, this application provides a localization method and apparatus based on multi-image reasoning, which achieves high stability and interpretability of multi-image reasoning localization.

[0005] According to one aspect of this application, a localization method based on multi-image reasoning is proposed, comprising: rewriting the original multi-image data into sample triples, wherein the sample triples include an image set, a cue word, and the actual localization result; preprocessing the sample triples to generate valid sample triples, and using the valid sample triples as training samples to construct a training dataset; filtering the training dataset using a pre-constructed distillation cue template and a pre-constructed multimodal visual language model to construct a cold-start set; using the training dataset, supervising fine-tuning the multimodal visual language model to obtain an initial multi-image reasoning model; using the cold-start set, supervising fine-tuning the initial multi-image reasoning model to obtain a basic multi-image reasoning model; using the training dataset, according to a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, performing reinforcement learning on the basic multi-image reasoning model until a preset termination condition is reached to obtain a target multi-image reasoning model; and inputting the multi-image data to be processed into the target multi-image reasoning model to obtain the reasoning localization result.

[0006] According to some embodiments, sample triples are preprocessed to generate valid sample triples, and the valid sample triples are used as training samples to construct a training dataset. This includes: using pre-set screening and quality control rules to remove or repair abnormal sample triples in the sample triples to obtain a first sample triple, and using sample triples that fail to be repaired as sample triples to be processed; using a preset multimodal model to assist in the generation or structured rewriting of the sample triples to be processed to obtain a second sample triple; using the first and second sample triples as valid sample triples; using the valid sample triples as training samples, and constructing a training dataset based on the training samples.

[0007] According to some embodiments, a cold start set is constructed by filtering the training dataset using a pre-built distillation cue template and a pre-built multimodal visual language model. This includes: extracting samples to be detected from the training dataset; inputting the samples to be detected into the multimodal visual language model to obtain a first localization result; inputting the distillation cue template and the samples to be detected into the multimodal visual language model to obtain an inference trajectory and a second localization result; calculating the localization accuracy of the first localization result as the first accuracy, and calculating the localization accuracy of the second localization result as the second accuracy; storing the samples to be detected in the inference enhancement precise sample set if the first accuracy reaches a correct threshold and the difference between the second accuracy and the first accuracy is greater than a first preset difference; storing the samples to be detected in the inference error correction sample set if the first accuracy does not reach a correct threshold but the second accuracy does; storing the samples to be detected in the inference significantly improved sample set if neither the first nor the second accuracy reaches a correct threshold and the difference between the second accuracy and the first accuracy is greater than a second preset difference; and constructing a cold start set based on the inference enhancement precise sample set, the inference error correction sample set, and the inference significantly improved sample set.

[0008] According to some embodiments, the distillation prompt template includes role definition, input specifications, task instructions, output format, anti-leakage constraints, anti-leakage filtering rules, and / or anti-pseudo-reasoning filtering rules.

[0009] According to some embodiments, using a training dataset, a basic multi-image reasoning model is subjected to reinforcement learning based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism until a preset termination condition is reached to obtain a target multi-image reasoning model, including: S1401: Selecting the current input sample from the training dataset; S1402: Based on the basic multi-image reasoning model, sampling the current input sample multiple times to generate multiple candidate outputs to construct a candidate output group; S1403: Calculating the reward value of multiple candidate outputs in the candidate output group according to a preset reward function; S1404: Calculating the learning value of the candidate output group based on the reward value of multiple candidate outputs and a pre-constructed learning value index. S1405: If the learning value is greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the high learning value sample set; S1406: If the learning value is not greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the low learning value sample set; S1407: Based on the high learning value sample set and / or the low learning value sample set, update the parameters of the basic multi-image inference model using a preset loss function; S1408: Repeat steps S1401-S1407 until the preset termination condition is met, and output the basic multi-image inference model updated in the last iteration as the target multi-image inference model.

[0010] According to some embodiments, using a training dataset, a basic multi-image reasoning model is subjected to reinforcement learning based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism until a preset termination condition is reached to obtain a target multi-image reasoning model. This includes: S1411: Calculating the current localization accuracy of the basic multi-image reasoning model using a pre-constructed validation set, and updating the current training stage, its difficulty range, and the value threshold for value filtering based on the current localization accuracy; S1412: Constructing a current candidate pool based on the training dataset according to the difficulty range corresponding to the current training stage; S1413: Selecting the current input sample from the current candidate pool; S1414: Sampling the current input sample multiple times based on the basic multi-image reasoning model. Multiple candidate outputs are generated to construct candidate output groups; S1415: Calculate the reward values ​​of multiple candidate outputs according to the preset reward function; S1416: Repeat steps S1413-S1415 until the training samples in the current candidate pool are traversed; S1417: Calculate the group average reward of the candidate output groups of the training samples in the current candidate pool and sort them according to the group average reward to obtain the sorting result; S1418: According to the sorting result, extract training samples and their output groups from the current candidate pool, and update the basic multi-image inference model using the preset loss function; S1419: Repeat steps S1411-S1418 until the preset termination condition is reached, and output the basic multi-image inference model after the last iteration update as the target multi-image inference model.

[0011] According to some embodiments, using a training dataset, a basic multi-image reasoning model is subjected to reinforcement learning based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism until a preset termination condition is reached to obtain a target multi-image reasoning model, including: S1421: Calculating the current positioning accuracy of the basic multi-image reasoning model using a pre-constructed validation set, and updating the current training stage, its difficulty range, and the value threshold for value filtering based on the current positioning accuracy; S1422: Constructing a current candidate pool based on the training dataset according to the difficulty range corresponding to the current training stage; S1423: Selecting the current input sample from the current candidate pool; S1424: Sampling the current input sample multiple times based on the basic multi-image reasoning model to generate multiple candidate outputs to construct a candidate output group; S1425: Calculating the candidate output group according to a preset reward function. S1426: Calculate the learning value of the candidate output group based on the reward values ​​of the multiple candidate outputs and the pre-constructed learning value index; S1427: If the learning value is greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the high learning value sample set; S1428: If the learning value is not greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the low learning value sample set; S1429: Update the parameters of the basic multi-image inference model using a preset loss function based on the high learning value sample set and / or the low learning value sample set; S1430: Repeat steps S1421-S1429 until the preset termination condition is met, and output the basic multi-image inference model after the last iteration update as the target multi-image inference model.

[0012] According to some embodiments, the parameters of the basic multi-image inference model are updated using a preset loss function based on a set of high-learning-value samples and / or a set of low-learning-value samples. This includes: updating the parameters of the basic multi-image inference model using a preset loss function based on a set of high-learning-value samples; or updating the parameters of the basic multi-image inference model using a preset loss function based on a set of high-learning-value samples, a set of low-learning-value samples, a preset update weight for the set of high-learning-value samples, and update weight for the set of low-learning-value samples.

[0013] According to some embodiments, based on a basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs to construct a candidate output group, including: extracting visual features and prompt word features of the current input sample; fusing visual features and prompt word features to obtain fused features; and based on the basic multi-image reasoning model, sampling the fused features multiple times to generate multiple candidate outputs to construct a candidate output group.

[0014] According to some embodiments, the preset reward function includes a format reward function and an accuracy reward function, and the accuracy reward function includes an image identification reward function and an IoU reward function.

[0015] According to one aspect of this application, a localization device based on multi-image reasoning includes: a first data unit for rewriting original multi-image data into sample triples, wherein the sample triples include an image set, a prompt word, and an actual localization result; and a second data unit for preprocessing the sample triples to generate valid sample triples, and using the valid sample triples as training samples to construct a training dataset. The third data unit is used to filter the training dataset using pre-built distillation prompt templates and a pre-built multimodal visual language model to construct a cold start set. The first fine-tuning unit is used to perform supervised fine-tuning of the multimodal visual language model using the training dataset to obtain an initial multi-image inference model. The second fine-tuning unit is used to perform supervised fine-tuning of the initial multi-image inference model using the cold start set to obtain a basic multi-image inference model. The reinforcement learning unit is used to perform reinforcement learning on the basic multi-image inference model using the training dataset, based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, until a preset termination condition is reached to obtain a target multi-image inference model. The model inference unit is used to input the multi-image data to be processed into the target multi-image inference model to obtain the inference and localization results.

[0016] According to one aspect of this application, an electronic device is provided, comprising: one or more processors; a storage device for storing one or more programs; and, when the one or more programs are executed by the one or more processors, causing the one or more processors to implement the method as described above.

[0017] According to one aspect of this application, a computer-readable medium is provided that stores a computer program or instructions thereon, which, when executed by a processor, implement the method as described above.

[0018] Through the embodiments provided in this application, during the training phase, the original multi-image data is rewritten and preprocessed to construct a training dataset, ensuring that the samples in the training dataset meet the structured requirements of the multi-image reasoning and localization task, providing a stable data foundation for subsequent supervised training and post-training phases. The training dataset is then filtered using distilled cue templates and a multimodal visual language model to construct a cold-start set. The multimodal visual language model is then fine-tuned under supervision using the training dataset, enabling the model to establish format recognition and fine-grained visual perception capabilities for the multi-image localization task. The initial multi-image reasoning model obtained in the first stage is then fine-tuned under supervision using the cold-start set, enabling the model to transition from direct localization to reasoning-driven localization. Finally, the basic multi-image reasoning model obtained in the second stage is reinforced using the training dataset, based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, to further strengthen the causal coupling between reasoning trajectory and localization accuracy, resulting in the target multi-image reasoning model. In the application phase, the multi-image data to be processed is input into the trained target multi-image inference model, and the model outputs the inference localization result, achieving stable, complete, interpretable and generalizable localization in multi-image and multi-target output scenarios. Attached Figure Description

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings, without exceeding the scope of protection claimed by this application.

[0021] Figure 1 A flowchart illustrating the localization method based on multi-image reasoning provided in this application embodiment;

[0022] Figure 2 A flowchart for constructing a training dataset provided in an embodiment of this application; Figure 3 A flowchart for constructing a cold start set provided in an embodiment of this application; Figure 4 A flowchart for constructing candidate output groups provided in an embodiment of this application; Figure 5 A block diagram of a positioning device based on multi-image reasoning provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0027] It should be understood that although the terms first, second, third, etc., may be used herein to describe various components, these components should not be limited by these terms. These terms are used to distinguish one component from another. Therefore, the first component discussed below may be referred to as the second component without departing from the teachings of this application. As used herein, the term "and / or" includes all combinations of any one and more of the associated listed items.

[0028] For specific implementation details, please refer to the following examples.

[0029] In recent years, rule-based reward-based reinforcement learning post-training (such as Group Relative Policy Optimization, GRPO) has been used to incentivize model inference capabilities. However, directly applying these methods to multi-graph inference and localization encounters more severe training instability: 1) Insufficient cross-image evidence association leads to localization drift. The key to multi-image tasks lies in establishing a consistent "evidence-conclusion" mapping across images. That is, the model needs to locate candidates in the image set, compare fine-grained attributes, eliminate interference items, and then transform the inference conclusion into bounding boxes in the corresponding images. End-to-end direct prediction often lacks an explicit cross-image memory and verification mechanism, which easily leads to the phenomenon that is semantically understandable but spatially misaligned. That is, the inference conclusion is inconsistent with the localized region, which manifests as localization drift or target confusion.

[0030] 2) Reasoning text relying solely on prompts is prone to drift and redundancy. To improve reasoning ability, a common practice is to require the model to "think step-by-step" in the prompts. However, in multi-image localization tasks, if the reasoning text lacks verifiable constraints, problems may arise such as: inconsistencies between the reasoning content and image evidence, repetitive and lengthy statements, or even premature disclosure of the final conclusion during the reasoning stage. This not only affects interpretability but also interferes with the localization output, leading to unstable formatting, incomplete multi-target output, and cross-image correspondence errors.

[0031] 3) Mixed training with multiple tasks and varying difficulties leads to instability in reinforcement learning. In recent years, rule-based reward-based reinforcement learning post-training (such as group relative policy optimization methods) has shown outstanding performance in large-scale model alignment. However, when this paradigm is directly applied to multi-image inference and localization, it encounters difficulties due to data-level heterogeneity and sample-level bias in learning value: different task types (comparison, association, tracking, region matching, etc.) have different requirements for model capabilities; even within the same task, the difficulty can vary significantly due to differences in the number of images, the number of targets, the degree of occlusion, and resolution. In this situation, if the same sampling and update strategy is used for all samples, problems such as vanishing advantage values, drastic fluctuations in gradient updates, and low training efficiency can easily occur.

[0032] 4) Significant heterogeneity in difficulty and learning value. Multi-image tasks cover various sub-tasks such as visual comparison, semantic association, spatial relationships, temporal tracking, and multi-view consistency. Different samples vary greatly in the number of images, targets, occlusion complexity, resolution, and scene interference. A uniform sampling and update strategy can easily lead to gradient oscillations and convergence difficulties. In intra-group sampling, many samples' candidate outputs may be homogeneous (with very small reward differences), causing the advantage value to approach zero, resulting in the disappearance of the advantage and a significant decrease in training efficiency. Conversely, a small number of highly dissimilar samples may lead to over-updates.

[0033] 5) Insufficient reward design and output format constraints lead to decreased usability. Multi-image localization tasks typically require structured output results (e.g., a list of bounding boxes containing image labels and multiple targets). If the reward only focuses on localization accuracy while ignoring the correctness of the format and image labels, the model may output unresolvable results, miss targets, or provide high IoU boxes on incorrect images, making it difficult to deploy in engineering.

[0034] Based on this, this application proposes a localization method and apparatus based on multi-image reasoning.

[0035] Figure 1 A flowchart illustrating the localization method based on multi-image reasoning provided in this application embodiment. Figure 1 As shown, the method includes steps S110-S170.

[0036] In step S110, the original multi-image data is rewritten into sample triples, wherein the sample triples include the image set, the prompt words, and the actual location results.

[0037] The original multi-image data includes data from at least two related images.

[0038] According to the example embodiment, the image data in the original multi-image data includes spatial association, temporal association and / or semantic association, and a set of two or more image data. For example, multi-view shooting, continuous frame sequence, comparison of different viewpoints of the same scene, and cross-image referencing and comparative positioning on multiple images, etc.

[0039] Furthermore, the image data in the original multi-image data can be obtained directly through image acquisition equipment or collected from existing resources, and this application does not impose any restrictions on this.

[0040] The image data in the original multi-image data can be data that has not been structured and annotated for a specific visual localization task, or it can be data that has been structured and annotated for a specific visual localization task. This application does not impose any restrictions on this.

[0041] In the process of rewriting raw multi-image data into sample triples, if the raw multi-image data contains high-quality natural language descriptions, these can be directly cleaned and normalized, and task intent and output format requirements can be added as prompts. Ground truth structured localization results, including bounding box and image label fields, can also be added as the actual localization results. If the raw multi-image data contains structured annotations for a specific visual localization task, the annotations can be organized, and the parts related to the task intent and output format requirements can be extracted as prompts. Ground truth structured localization results, including at least bounding box and image label fields, can also be extracted and organized as the actual localization results.

[0042] In the specific implementation process, the original multi-graph data is uniformly standardized into sample triples. Where I is the image set, p is a prompt containing the task intent and output format requirements, and a is the ground truth structured localization result, which includes at least bounding box and image label fields.

[0043] In step S120, the sample triples are preprocessed to generate valid sample triples, which are then used as training samples to construct the training dataset.

[0044] Preprocessing is performed on the sample triples, including data screening and quality control.

[0045] In some embodiments, preset rules are used to achieve data filtering and quality control.

[0046] According to the example implementation, data filtering and quality control includes: removing or repairing the following data: samples with abnormal number of images, resolution exceeding limits, missing key fields, and redundant dialogue rounds.

[0047] According to the example implementation, data filtering and quality control also include: format translation and field alignment of the data, such as unifying key names and coordinate order, supplementing image_id (image index), and standardizing label text, to ensure parsability and matchability.

[0048] The preprocessed sample triples are denoted as valid sample triples and used as training samples to construct the training dataset.

[0049] In step S130, the training dataset is filtered using a pre-built distillation cue template and a pre-built multimodal visual language model to construct a cold start set.

[0050] Distillation cue templates are pre-built according to task type to induce reasoning trajectories across image evidence retrieval, comparison, and disambiguation, thereby constructing cold-start thought chain data.

[0051] Furthermore, the distillation prompt template can also be designed to impose anti-leakage constraints on the reasoning section (e.g., the final location coordinates or equivalent answer information must not be leaked), and to perform anti-leakage and anti-false reasoning filtering on the generated results.

[0052] In the process of constructing the cold start thinking chain data, the localization effect of "direct prediction without reasoning" and "prediction with reasoning trajectory" is compared for the same sample in the training dataset, and only the samples that can bring substantial localization improvement are retained as the cold start set.

[0053] In practice, based on the same multimodal visual language model, two different inference protocols are executed on the same sample in the training dataset, and the accuracy of their localization output is compared. Samples whose accuracy meets preset conditions are included as cold-start thought chain data to construct the cold-start set. It's important to explain that the comparison essentially involves controlling the inference protocol variables on the same teacher model to evaluate the actual gain of explicit inference trajectories on localization accuracy. The core purpose of this design is to select samples whose inference trajectories genuinely contribute positively to localization, rather than including them in the cold-start set simply because the model generates inference text.

[0054] The cold start set is used for supervised fine-tuning so that the model has a stable inference-answer output format and inference-localization consistency before reinforcement learning, thereby significantly reducing the risk of format collapse, homogenized output and loss of advantage in subsequent reinforcement learning.

[0055] It should be noted that the multimodal visual language model can be selected according to the actual situation, such as Qwen2.5-VL or similar multimodal large language models, and this application does not impose any restrictions on it.

[0056] Furthermore, in some embodiments, a multimodal input processing mechanism is configured for the multimodal visual language model, which concatenates or weights and fuses the feature representations of each source, and finally inputs the fused visual features and textual question features into the language model for output decoding.

[0057] In step S140, the multimodal visual language model is supervised and fine-tuned using the training dataset to obtain the initial multi-image reasoning model.

[0058] Using the training dataset, a multimodal visual language model is subjected to supervised fine-tuning (SFT) for multi-image fine-grained perception to obtain an initial multi-image inference model. This stage uses the training dataset as the foundation for training data, standardizes the input and output formats using structured output templates, and enables the basic model to establish format recognition and fine-grained visual perception capabilities for multi-image localization tasks through standard supervised fine-tuning.

[0059] It is important to emphasize that this stage does not require the model to generate any inference trajectories, nor does it involve the generation of inference trajectories. The model outputs structured localization results in an end-to-end direct prediction manner.

[0060] The structured output template standardizes the output format for multi-image localization tasks, enabling the model to develop stable expectations and format compliance with multi-object structured outputs when encountering large-scale multi-image localization data (training dataset). Its design focuses on format constraints, ensuring that the model output can be parsed and evaluated in a rule-based manner.

[0061] According to the example implementation, the standardized output format includes a numerical representation of the bounding box coordinates (bbox_2d), a JSON structure of the target label and the image index (image_id).

[0062] According to the example implementation, the structured output template includes a task description, input images and question text, and a standardized JSON output format definition.

[0063] In step S150, the initial multi-image inference model is supervised and fine-tuned using the cold start set to obtain the basic multi-image inference model.

[0064] Using the cold-start set, the initial multi-image reasoning model is activated by cold-start data-based reasoning capability activation (Cold-start SFT) to obtain the basic multi-image reasoning model.

[0065] This stage uses the cold start set obtained through distillation template and dual filtration process as training data, and then analyzes the data within the cold start set... <think> ...< / think> and <answer> ...< / answer> Supervised fine-tuning with structured annotations injects the ability to jointly generate explicit inference trajectories and precise localization outputs into the model, completing the leap from direct localization to inference-driven localization, and providing a stable format protocol and policy initialization foundation for subsequent reinforcement learning stages.

[0066] In step S160, using the training dataset, reinforcement learning is performed on the basic multi-image reasoning model according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism until the preset termination condition is reached, thereby obtaining the target multi-image reasoning model.

[0067] Using the training dataset, the basic multi-image reasoning model is trained through FG-DAPO reinforcement learning to obtain the target multi-image reasoning model. This stage uses the model obtained in the second stage as the starting point for policy initialization. Based on a pre-set dynamic filtering mechanism of learning value and / or a difficulty-aware course current candidate pool mechanism, the model policy is optimized online through a pre-set fine-grained reward mechanism, further strengthening the causal coupling between reasoning trajectory and localization accuracy.

[0068] The preset termination conditions can be set according to the actual situation, and this application does not impose any restrictions on them.

[0069] According to the example implementation, the preset learning value dynamic filtering mechanism is used to: filter multiple candidate outputs for the same input, prioritizing the retention of sample groups with more significant differences in rewards within the group and providing effective comparative learning signals, thereby improving training stability and sample efficiency. The reward is calculated for the filtered output, and advantages are updated based on the reward results. The policy model is then iteratively optimized under the reference model constraint (KL Loss) to suppress policy drift and maintain output protocol stability.

[0070] The difficulty-aware course current candidate pool mechanism is used to: redistribute and select samples based on the reward distribution, realize training scheduling from easy to difficult, and further alleviate the convergence fluctuations caused by the heterogeneity of difficulty in multi-graph tasks.

[0071] In step S170, the multi-image data to be processed is input into the target multi-image inference model to obtain the inference and localization results.

[0072] In the application phase, the multi-image data to be processed is input into the trained target multi-image inference model, and the model outputs the inference and localization results.

[0073] Through the embodiments provided in this application, during the training phase, the original multi-image data is rewritten and preprocessed to construct a training dataset, ensuring that the samples in the training dataset meet the structured requirements of the multi-image reasoning and localization task, providing a stable data foundation for subsequent supervised training and post-training phases. The training dataset is then filtered using distilled cue templates and a multimodal visual language model to construct a cold-start set. The multimodal visual language model is then fine-tuned under supervision using the training dataset, enabling the model to establish format recognition and fine-grained visual perception capabilities for the multi-image localization task. The initial multi-image reasoning model obtained in the first stage is then fine-tuned under supervision using the cold-start set, enabling the model to transition from direct localization to reasoning-driven localization. Finally, the basic multi-image reasoning model obtained in the second stage is reinforced using the training dataset, based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, to further strengthen the causal coupling between reasoning trajectory and localization accuracy, resulting in the target multi-image reasoning model. In the application phase, the multi-image data to be processed is input into the trained target multi-image inference model, and the model outputs the inference localization result, achieving stable, complete, interpretable and generalizable localization in multi-image and multi-target output scenarios.

[0074] According to some embodiments, refer to Figure 2 In step S120, the sample triples are preprocessed to generate valid sample triples, which are then used as training samples to construct a training dataset. This can be achieved through steps S210-S240.

[0075] In step S210, abnormal sample triples in the sample triples are removed or repaired using pre-set screening and quality control rules to obtain the first sample triples, and the sample triples that fail to be repaired are taken as sample triples to be processed.

[0076] A rule-based screening and quality control process is implemented for sample triplets. In practice, pre-set screening and quality control rules are used to screen and control sample triplets, eliminating or repairing abnormal triplets. The resulting set is designated as the first sample triplet, ensuring that the samples meet the structured requirements of the multi-graph reasoning and localization task and reducing noise. Simultaneously, sample triplets that fail to be repaired are designated as pending samples.

[0077] Furthermore, in some embodiments, anomalous sample triples include: abnormal number of images, resolution exceeding limits, missing key fields, and redundant dialogue turns. Correspondingly, the rule processing includes question type filtering, image number filtering, field error filtering, and dialogue turn filtering.

[0078] Simultaneously, format translation and field alignment are performed, such as standardizing key names and coordinate order, padding image_id (image index), and normalizing label text. In other words, rule processing also includes format translation and adaptation.

[0079] In step S220, a second sample triplet is obtained by using a preset multimodal model to assist in the generation or structured rewriting of the triplet to be processed.

[0080] The triplet to be processed includes fields that are difficult to repair according to rules. Based on rule processing, this application further introduces model-based enhancement and repair. A preset multimodal model is introduced to assist in the generation or structured rewriting of the triplet to be processed. The multimodal large model is used to generate or complete key field information (such as target "label" text, image identifier "image id"), and the output format is restated and normalized so that the sample meets the requirements of reinforcement learning and verifiable evaluation in terms of field completeness, parsability and consistency. The obtained assisted generation or structured rewriting result is recorded as the second triplet.

[0081] Furthermore, in some embodiments, during the assisted generation or structured rewriting process, bucketing and hierarchical sampling are performed according to dimensions such as task type, number of images, and number of targets to control the training distribution.

[0082] In step S230, the first sample triplet and the second sample triplet are used as valid sample triplets.

[0083] The first sample triplet and the second sample triplet are used together as the effective sample triplet.

[0084] In step S240, the effective sample triples are used as training samples, and a training dataset is constructed based on the training samples.

[0085] The embodiments of this application obtain high-quality multi-graph localization refined data through a two-stage processing of "rule constraints + model repair", providing a stable data foundation for subsequent supervised training and post-training stages.

[0086] According to some embodiments, refer to Figure 3 In step S130, the training dataset is filtered using a pre-built distillation cue template and a pre-built multimodal visual language model to construct a cold start set, which can be implemented through steps S310-S380.

[0087] In step S310, samples to be detected are extracted from the training dataset.

[0088] Select at least one training sample from the training dataset as the sample to be detected.

[0089] In step S320, the sample to be detected is input into the multimodal visual language model to obtain the first localization result.

[0090] The first localization result is obtained by "direct prediction without reasoning" of the extracted samples to be detected based on a multimodal visual language model.

[0091] It should be noted that "direct prediction without reasoning" means directly inputting the image and question into the multimodal visual language model, allowing it to directly output the localization result (bounding box) in an end-to-end manner, without attaching any thought chain generation instructions.

[0092] In step S330, the distillation prompt template and the sample to be detected are input into the multimodal visual language model to obtain the inference trajectory and the second localization result.

[0093] Based on a multimodal visual language model, the extracted samples to be detected are "predicted with inference trajectory" to obtain the inference trajectory and the second localization result.

[0094] It should be noted that "inference trajectory prediction" refers to inputting the image, question, and task-type distillation template together into a multimodal visual language model, allowing it to generate... <think>The reasoning trajectory, and then in <answer>The segment outputs the positioning result (referred to as the second positioning result).

[0095] In step S340, the positioning accuracy of the first positioning result is calculated as the first accuracy, and the positioning accuracy of the second positioning result is calculated as the second accuracy.

[0096] Calculate the positioning accuracy of the first positioning result, denoted as the first accuracy, and calculate the positioning accuracy of the second positioning result, denoted as the second accuracy.

[0097] According to the example embodiment, the positioning accuracy is set to IoU (Intersection over Union) accuracy.

[0098] The localization accuracy (i.e., first accuracy and second accuracy) of the localization outputs when applying two different inference protocols to the same sample is compared. Essentially, this comparison involves controlling the inference protocol variables on the same teacher model and evaluating the actual gain of explicit inference trajectories on localization accuracy. The core purpose of this design is to select samples whose inference trajectories genuinely contribute positively to localization, rather than including them in the cold start set simply because the model generates inference text. Only when the prediction containing the inference trajectories meets the localization accuracy improvement threshold condition specified in the embodiments of this application compared to the direct prediction is the CoT (CoT) annotation of the sample considered valid and allowed to enter the final cold start set.

[0099] The design motivation for this comparison mechanism stems from a core problem: inference texts guided solely by prompts are prone to redundancy, drift, and even spurious inference when lacking verifiable constraints, thus interfering with the location output. Therefore, using improved location accuracy as a verifiable objective signal to verify the validity of the inference for each distillation sample is essentially to establish whether the inference truly serves the location as a hard criterion for data quality, thereby ensuring a genuine causal gain relationship between inference and location in cold-start data, rather than merely superficial format conformity.

[0100] By using a verifiable admission criterion centered on improving positioning accuracy, for any sample in the training dataset, let its positioning accuracy (i.e., the first positioning accuracy) under direct prediction without inference be denoted as . The positioning accuracy (i.e., the second positioning accuracy) with inference trajectory prediction is: The threshold conditions for improving positioning accuracy for selecting "samples that can bring substantial positioning improvement" are as shown in steps S350-S370.

[0101] In step S350, if the first precision reaches the correct threshold and the difference between the second precision and the first precision is greater than the first preset difference, the sample to be detected is stored in the inference enhancement precise sample set.

[0102] When the direct prediction itself is correct (i.e.) (reaching the correct threshold), and with the introduction of inference trajectories, the positioning accuracy is further improved, satisfying... If the first preset difference is true, the sample to be detected belongs to the inference enhancement precise sample set, and the sample to be detected is stored in the inference enhancement precise sample set.

[0103] These samples demonstrate that inference trajectories can still bring quantifiable accuracy gains on top of existing correct localization, and that the introduction of inference has substantial value in improving fine-grained accuracy.

[0104] According to the example embodiment, the first preset difference is set to 10%.

[0105] In step S360, if the first precision does not reach the correct threshold and the second precision reaches the correct threshold, the sample to be detected is stored in the inference error correction sample set.

[0106] In direct prediction error (i.e. The prediction did not reach the correct threshold, but it became correct after the inference trajectory was introduced. If the correct threshold is reached, the sample to be detected belongs to the inference error correction sample and is stored in the inference error correction sample set.

[0107] These samples demonstrate that the reasoning trajectory has a clear error correction capability, which is a direct manifestation of the functioning of cross-image evidence association and disambiguation mechanisms.

[0108] In step S370, if neither the first precision nor the second precision reaches the correct threshold, and the difference between the second precision and the first precision is greater than the second preset difference, the sample to be detected is stored in the inference significantly improved sample set.

[0109] Even after introducing inference trajectories, the predictions were still not entirely accurate despite direct prediction errors, but the improvement in positioning accuracy met the requirements. If the sample to be tested is a sample with significantly improved reasoning, then the sample to be tested will be stored in the set of samples with significantly improved reasoning.

[0110] These samples demonstrate that although the reasoning trajectory has not achieved complete error correction, it has brought about significant improvements in localization and is therefore valuable for retention.

[0111] According to the example embodiment, the second preset difference is set to 20%.

[0112] In step S380, a cold start set is constructed based on the inference reinforcement accurate sample set, the inference error correction sample set, and the inference significantly improved sample set.

[0113] The three types of samples are collectively referred to as "samples that can bring substantial improvement in positioning". The sets corresponding to the three types of samples are merged to construct the cold start set.

[0114] This application embodiment completes the secondary construction of multi-map positioning data and the distillation cold start of thought chain demonstration data to reduce format noise and establish a stable output form of "reasoning first and then positioning".

[0115] According to some embodiments, the distillation prompt template includes role definition, input specifications, task instructions, output format, anti-leakage constraints, anti-leakage filtering rules, and / or anti-pseudo-reasoning filtering rules.

[0116] Understandably, the distillation template serves the second stage of cold-start inference activation. Its core objective is to further activate the potential inference capabilities of the model that already possesses basic localization capabilities, and to induce the model to generate cross-image inference trajectories that are deeply coupled with the localization task.

[0117] This template uses a multimodal visual language model (e.g., Qwen2.5-VL-72B) as the teacher model, and guides it to generate high-quality CoT (CoT) labeled data through a carefully designed prompt structure, and finally builds a cold start set for policy initialization in the subsequent reinforcement learning stage.

[0118] Furthermore, the distillation prompt template is designed for specific task types.

[0119] According to the example implementation, for cross-image evidence retrieval and inference trajectory guidance, at the distillation template design level, multi-image instances are divided into four categories according to task attributes: visual comparison analysis, spatial perception, temporal perception, and semantic / logical association. For each category, a targeted distillation prompt template is designed to induce the model to generate inference operators that match the task semantics.

[0120] Specifically, visual comparison tasks are injected with operators such as comparison and observation to guide the model to perform difference retrieval across image colors, textures, and shapes; spatial perception tasks are injected with searching and observation operators to guide region correspondence localization under viewpoint changes; temporal perception tasks are injected with tracking and searching operators to enhance cross-frame target identity consistency tracking; and semantic / logical association tasks are injected with association and reasoning operators to promote the inference of abstract semantic and functional relationships.

[0121] In terms of template structure, each distillation prompt consists of four elements: role definition, input specifications, task instructions, and output format. The role definition positions the model as a visual expert to establish a professional reasoning identity; the input specifications explicitly provide multiple images, question text, and truth answers (used only for verification and not explicitly referenced in inference); the task instructions require the model to analyze visual and textual cues step by step, following the order of "compare / reason first, and then draw a conclusion"; the output format mandates that the inference section be a plain text paragraph, and the answer section be a structured JSON block.

[0122] In the specific use of the template, in the operational implementation of cross-image evidence retrieval, the template explicitly injects the aforementioned task-oriented keywords, forcibly guiding the model to execute a four-step operation sequence in the inference segment: First, anchor the target in the reference image and extract distinguishable visual attributes (color, texture, pose, contextual relationship, etc.); second, search for candidate regions in the target image set and compare the attributes one by one; third, complete disambiguation by eliminating interference items and establish a consistent "evidence-conclusion" mapping across images; finally, give a qualitative judgment at the end of the inference segment without revealing the coordinate information in advance.

[0123] Anti-leakage constraints are applied during the generation of inference segments, imposing strict anti-leakage requirements that prevent the final conclusion or any truth location information (such as bounding box coordinates) from being revealed prematurely. This constraint is applied collaboratively at both the hinting layer and the output protocol layer. At the hinting layer, the distillation template explicitly prohibits the inclusion of final location coordinates (in the format [x1,y1, x2, y2]) or equivalent location descriptions (such as numerical spatial representations that can be inferred from the coordinates) in the inference segments. The inference content must contain only visual attribute analysis and logical deduction of plain text, and must not contain any numerical spatial information. At the output protocol layer, the template enforces the use of... <think> ...< / think> and <answer> ...< / answer> The segmented label structure physically isolates the reasoning process from the final localization result. Coordinate information is only allowed to appear in... <answer>The JSON block of the segment is structurally designed to prevent the leakage of the answer in the reasoning segment, ensuring the true explanatory nature of the reasoning trajectory rather than the restatement of the answer.

[0124] The post-processing filtering mechanism for anti-leakage and anti-spurious inference is used to execute a two-step post-processing filtering pipeline after distillation to ensure data quality.

[0125] Anti-leakage filtering includes: applying rule-based detection to generated samples to eliminate "pseudo-reasoning" where the final conclusion (location coordinates or equivalent answer) appears at the beginning or middle of the reasoning section. This rule detects... <think>Does the segment contain a coordinate format string (such as a regular expression matching \[\d+,\s)? \d+,\s \d+,\s Automatic identification and elimination are achieved by using text fragments that highly overlap with the true answer (\d+\]) or text segments that are highly overlapping with the true answer, thus ensuring the purity of the reasoning content at the structural level.

[0126] Anti-spurious inference filtering includes comparing the localization accuracy of "direct prediction without inference" and "prediction with inference trajectory" for the same sample, and verifying the actual effectiveness of the inference trajectory based on the improvement in localization accuracy. Specifically, only samples that meet any of the localization accuracy improvement threshold conditions are retained, and samples that do not meet any of the above conditions (i.e., samples where inference does not bring about localization improvement or even causes degradation) are not included in the cold start set, fundamentally eliminating the negative impact of spurious inference samples that are "redundant and invalid inference" or "correct inference but incorrect localization" on subsequent reinforcement learning stages.

[0127] After the above complete pipeline processing, the final high-quality cold start set ensures that each sample meets three conditions: the inference trajectory is real and valid, there is no answer leakage, and it has a verifiable positive contribution to the localization accuracy. This provides a stable format protocol and inference-localization consistency foundation for the subsequent FG-DAPO reinforcement learning stage.

[0128] According to some embodiments, in step S160, the basic multi-image reasoning model is reinforced by using the training dataset and according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism until the preset termination condition is reached, and the target multi-image reasoning model is obtained. Specifically, this can be achieved through steps S1401-S1408.

[0129] In step S1401, the current input sample is selected from the training dataset.

[0130] Select at least one training sample from the training dataset as the current input sample.

[0131] In step S1402, based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs in order to construct a candidate output group.

[0132] In the post-training phase of reinforcement learning, this embodiment applies the following to each current input sample: Perform in-group sampling to generate multiple (e.g., K) candidate outputs and construct candidate output groups.

[0133] In step S1403, the reward values ​​of multiple candidate outputs in the candidate output group are calculated according to the preset reward function.

[0134] Based on the preset reward function, the verifiable reward is calculated for each candidate output in the candidate output group, and the reward value is obtained.

[0135] In step S1404, the learning value of the candidate output group is calculated based on the reward values ​​of multiple candidate outputs and the pre-built learning value index.

[0136] To address the vanishing advantage and training instability issues commonly found in multi-graph task training, this application introduces a dynamic filtering mechanism for learning value in the proposed FG-DAPO algorithm.

[0137] Specifically, by using a pre-built learning value index, the learning value index is calculated for the reward value of each candidate output in the candidate output group, thus obtaining the learning value of the candidate output group.

[0138] According to the example implementation, the learning value metric is set as the within-group reward variance. : .

[0139] in, For variance operators, Indicates the first k The reward value of each candidate output. This indicates the number of candidate outputs in the candidate output group.

[0140] In step S1405, if the learning value is greater than the value threshold corresponding to the current training stage, the current input sample and its candidate output group are saved to the high learning value sample set.

[0141] It should be explained that the embodiments of this application divide the training process into two main stages. The distinction between the two stages is mainly based on a threshold. A dynamic monitoring mechanism for determining the benchmark enables adaptive switching.

[0142] Specifically, the model's localization accuracy on the validation set during training (e.g., accuracy based on IoU) is used as the core monitoring signal for stage switching. Let the current training step... The positioning accuracy of the time model is threshold If the preset accuracy threshold is used, the formal distinction between the two stages is as follows.

[0143] Early stage.

[0144] Stable Stage of Training.

[0145] That is, stage attribution is determined by whether the model's current positioning accuracy crosses a threshold. The number of training steps or epochs is determined dynamically, rather than being predetermined by a fixed number of training steps or epochs.

[0146] The core significance of this design is that the convergence speed of the model varies under different sample batches and different task distributions. A fixed number of steps will cause the timing of stage switching to be out of sync with the actual capability of the model. However, using an accuracy threshold as an admission condition can ensure that stage switching occurs when the model's capability has truly reached a stable level.

[0147] The essential difference between the two stages lies not only in the judgment conditions, but also in the composition and weighting of the reward mechanism. Early stage ( The training focus is on establishing stable format compliance and basic inference protocols. At this stage, the model has not yet reached a stable state. <think> ...< / think> and <answer> ...< / answer> Segmented generation capability: Introducing high-weighted localization accuracy rewards too early will cause the model to disrupt the format structure in pursuit of localization scores, leading to training instability. Therefore, this stage assigns higher weight to format compliance rewards and adopts a relatively lenient evaluation standard for localization accuracy rewards, prioritizing the structural stability of the model's output. Training stabilization stage ( The training focus shifts to optimizing the deep coupling between inference quality and localization accuracy. At this stage, the model has a stable format compliance capability, the marginal value of format compliance rewards decreases, and the dominance of training signals shifts to fine-grained localization accuracy rewards and inference trajectory quality rewards, driving the model to further strengthen the causal relationship between cross-image inference and precise localization while maintaining format stability.

[0148] Different value thresholds are set for different stages. In the early stages, a higher value threshold is used to prioritize the retention of samples with high learning value, ensuring sufficient contrast differences within the group to form effective advantage signals and promote exploration and error correction. As training stabilizes, the threshold is gradually lowered in the stable training stage to include more samples and improve coverage and generalization.

[0149] Based on the value threshold corresponding to the current training phase Perform binary filtering and judgment.

[0150] When the learning value is greater than the value threshold corresponding to the current training stage In the case of [missing information], the current input sample and its candidate output groups are saved to a set of high-learning-value samples.

[0151] In step S1406, if the learning value is not greater than the value threshold corresponding to the current training stage, the current input sample and its candidate output group are saved to the low learning value sample set.

[0152] If the learning value is less than or equal to the value threshold corresponding to the current training stage, the current input sample and its candidate output group are saved to the low learning value sample set.

[0153] In step S1407, the parameters of the basic multi-image inference model are updated using a preset loss function based on the set of high-learning-value samples and / or the set of low-learning-value samples.

[0154] The extracted high-learning-value sample set refers to the sample group with more obvious intra-group reward differences and that can provide effective comparative learning signals. Advantages are updated based on the high-learning-value sample set and / or the low-learning-value sample set, and the basic multi-image inference model is iteratively optimized under the constraints of the reference model (i.e., the loss function, such as KL Loss) to suppress policy drift and maintain the stability of the output protocol.

[0155] According to the example implementation, only the set of high-learning-value samples is used for advantage updates to improve training stability and sample efficiency.

[0156] In step S1408, steps S1401-S1407 are repeated until the preset termination condition is met, and the basic multi-image reasoning model after the last iteration update is output as the target multi-image reasoning model.

[0157] Repeat steps S1401-S1407 until the preset termination condition is met, and output the target multi-image reasoning model.

[0158] After each round of parameter updates, the current positioning accuracy is evaluated on the validation set. ,like This triggers a phase switch and synchronously updates the dynamic threshold for value filtering. .

[0159] The preset termination conditions can be set according to the actual situation, and this application does not impose any restrictions on them.

[0160] This application embodiment independently uses a preset learning value dynamic filtering mechanism to filter multiple candidate outputs for the same input, prioritizing the retention of sample groups with more significant differences in rewards within each group and providing effective comparative learning signals, thereby improving training stability and sample efficiency. Rewards are calculated for the filtered outputs, and advantages are updated based on the reward results. The policy model is then iteratively optimized under reference model constraints (KL Loss) to suppress policy drift and maintain output protocol stability.

[0161] According to some embodiments, in step S160, the basic multi-image reasoning model is reinforced by using the training dataset and according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism until the preset termination condition is reached, and the target multi-image reasoning model is obtained. This can be specifically achieved through steps S1411-S1419.

[0162] In step S1411, the current localization accuracy of the basic multi-image inference model is calculated using a pre-built validation set, and the current training stage, its difficulty range, and the value threshold for value filtering are updated based on the current localization accuracy.

[0163] The determination of the training phase can be referred to the above embodiments, and the embodiments of this application will not be repeated here.

[0164] The core function of the course candidate pool is to dynamically construct a candidate sample pool with a specific difficulty distribution from the complete training set before the start of each training batch, based on the difficulty range corresponding to the current training stage.

[0165] The course schedule follows a gradual progression from easy to difficult. Early training ( Prioritize building a candidate pool from low-difficulty samples (e.g., samples with difficulty less than or equal to a preset difficulty threshold) to ensure that the model does not diverge during training due to overly difficult samples when format compliance and basic reasoning ability are not yet stable; as training progresses to a stable stage ( The difficulty range of the candidate pool gradually shifts towards a higher difficulty range (e.g., samples with difficulty greater than a preset difficulty threshold), driving the model to continuously face challenging cross-image reasoning scenarios in order to break through the upper limit of its capabilities.

[0166] In other words, if the current training phase is in the early stage, it corresponds to the low difficulty range; if the current training phase is in the stable stage, it corresponds to the high difficulty range.

[0167] The determination of the value threshold can be referred to the above embodiments, and the embodiments of this application will not be repeated here.

[0168] Based on current positioning accuracy With threshold The relationship determines the current training phase, and based on this, the difficulty range and value threshold are selected.

[0169] In step S1412, the current candidate pool is constructed based on the training dataset according to the difficulty range corresponding to the current training stage.

[0170] Based on the determined difficulty range corresponding to the current training stage, the current candidate pool is dynamically constructed from the training dataset. .

[0171] The sample difficulty can be defined according to a pre-set difficulty index. First, the difficulty index value of the training samples in the training dataset is calculated. Based on the difficulty range corresponding to the current training stage, samples whose difficulty index values ​​meet the difficulty range are extracted to obtain the current candidate pool.

[0172] According to the example embodiment, the sample difficulty is determined by the group average reward. Operational definition.

[0173] in, Let be the sample index, and i be the candidate output index. The number of times each sample is sampled, i.e., the number of candidate outputs in the candidate output group for each sample.

[0174] The average reward per group is inversely correlated with the sample difficulty. .

[0175] In this embodiment, the group average reward of the training samples in the training dataset is first calculated. The implementation steps can be referred to as steps S1413-S1415. The reward values ​​of multiple candidate outputs of each training sample in the training dataset are calculated, thereby calculating the group average reward of the candidate output group of each training sample in the training dataset, and thus determining its difficulty.

[0176] It should be noted that the size of the candidate pool needs to be larger than that of the training batch, and its size can be several times that of the training batch.

[0177] In step S1413, the current input sample is selected from the current candidate pool.

[0178] Select at least one training sample from the training dataset as the current input sample.

[0179] In step S1414, based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs in order to construct a candidate output group.

[0180] This application embodiment applies to each current input sample Perform in-group sampling to generate multiple (e.g., G) candidate outputs and construct candidate output groups.

[0181] In step S1415, the reward values ​​of multiple candidate outputs are calculated according to a preset reward function.

[0182] Based on the preset reward function, the verifiable reward is calculated for each candidate output in the candidate output group, and the reward value is obtained.

[0183] In step S1416, steps S1413-S1415 are repeated until all training samples in the current candidate pool are traversed.

[0184] Repeat steps S1413-S1415 to calculate the reward values ​​for multiple candidate outputs of all training samples in the current candidate pool.

[0185] In step S1417, the average reward of the candidate output groups of the training samples in the current candidate pool is calculated and sorted according to the average reward to obtain the sorting result.

[0186] Sample difficulty is determined by the average reward of the groups. Operational definition.

[0187] in, Let be the sample index, and i be the candidate output index. The number of times each sample is sampled, i.e., the number of candidate outputs in the candidate output group.

[0188] The candidate pool is sorted according to the average reward of each group to obtain the ranking result.

[0189] In step S1418, training samples and their output groups are extracted from the current candidate pool according to the sorting results, and the basic multi-image inference model is updated using a preset loss function.

[0190] The underlying multi-image inference model is iteratively optimized under reference model constraints (i.e., loss functions, such as KL Loss) by sorting within the candidate pool and extracting samples in a phased order for updates.

[0191] In step S1419, steps S1411-S1418 are repeated until the preset termination condition is met, and the basic multi-image reasoning model after the last iteration update is output as the target multi-image reasoning model.

[0192] Repeat steps S1411-S1418 until the preset termination condition is met, and output the target multi-image reasoning model.

[0193] After each round of parameter updates, the current positioning accuracy is evaluated on the validation set. ,like This triggers a phase switch, synchronously updating the difficulty range and dynamic threshold for value filtering in the course candidate pool. .

[0194] The preset termination conditions can be set according to the actual situation, and this application does not impose any restrictions on them.

[0195] This application embodiment independently uses the difficulty-aware course current candidate pool mechanism to build a candidate pool with a size larger than the training batch in each training cycle or every few steps. The average reward of the candidate pool samples is calculated and sorted to form a local "easy to difficult" scheduling. The model is more likely to obtain a stable advantage estimate within its current capability range, reducing the drastic gradient fluctuations caused by the early introduction of difficult samples, and improving sample utilization efficiency.

[0196] According to some embodiments, in step S160, the basic multi-image reasoning model is reinforced by using the training dataset and according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism until the preset termination condition is reached, and the target multi-image reasoning model is obtained. This can be specifically achieved through steps S1421-S1430.

[0197] In step S1421, the current localization accuracy of the basic multi-image inference model is calculated using a pre-built validation set, and the current training stage, its difficulty range, and the value threshold for value filtering are updated based on the current localization accuracy.

[0198] For specific implementation steps, please refer to step S1411, which will not be elaborated upon in this application.

[0199] In step S1422, the current candidate pool is constructed based on the training dataset according to the difficulty range corresponding to the current training stage.

[0200] For specific implementation steps, please refer to step S1412, which will not be elaborated upon in this application.

[0201] In step S1423, the current input sample is selected from the current candidate pool.

[0202] Select at least one training sample from the training dataset as the current input sample.

[0203] In step S1424, based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs in order to construct a candidate output group.

[0204] This application embodiment applies to each current input sample Perform in-group sampling to generate multiple (e.g., K) candidate outputs and construct candidate output groups.

[0205] In step S1425, the reward values ​​of multiple candidate outputs in the candidate output group are calculated according to the preset reward function.

[0206] Based on the preset reward function, the verifiable reward is calculated for each candidate output in the candidate output group, and the reward value is obtained.

[0207] In step S1426, the learning value of the candidate output group is calculated based on the reward values ​​of multiple candidate outputs and the pre-built learning value index.

[0208] For specific implementation steps, please refer to step S1404, which will not be elaborated upon in this application.

[0209] In step S1427, if the learning value is greater than the value threshold corresponding to the current training stage, the current input sample and its candidate output group are saved to the high learning value sample set.

[0210] For specific implementation steps, please refer to step S1405, which will not be elaborated upon in this application.

[0211] In step S1428, if the learning value is not greater than the value threshold corresponding to the current training stage, the current input sample and its candidate output group are saved to the low learning value sample set.

[0212] For specific implementation steps, please refer to step S1406, which will not be elaborated upon in this application.

[0213] In step S1429, the parameters of the basic multi-image inference model are updated using a preset loss function based on the set of high-learning-value samples and / or the set of low-learning-value samples.

[0214] For specific implementation steps, please refer to step S1407, which will not be elaborated upon in this application.

[0215] According to the example implementation, only a set of high-learning-value samples is used. Calculate the gradient loss of the FG-DAPO policy from the samples. And perform parameter updates.

[0216] .

[0217] in, For the expectation operator, For the clipping function, The cropping threshold, The KL divergence coefficient is... Let KL divergence be the KL divergence. For the new strategy to be optimized in the model, The old strategy before model optimization. For the model's observations / actions, For the conditions / states of the model, The importance sampling ratio, This is an advantage estimate based on within-group reward normalization.

[0218] In step S1430, steps S1421-S1429 are repeated until the preset termination condition is met, and the basic multi-image reasoning model after the last iteration update is output as the target multi-image reasoning model.

[0219] Repeat steps S1421-S1429 until the preset termination condition is met, and output the target multi-image reasoning model.

[0220] After each round of parameter updates, the current positioning accuracy is evaluated on the validation set. ,like This triggers a phase switch, synchronously updating the difficulty range and dynamic threshold for value filtering in the course candidate pool. .

[0221] The preset termination conditions can be set according to the actual situation, and this application does not impose any restrictions on them.

[0222] In this embodiment, the learning value dynamic filtering mechanism and the difficulty-aware course current candidate pool mechanism are not independent parallel modules, but constitute a two-layer nested online training scheduling architecture. The course candidate pool is responsible for the dynamic control of sample difficulty distribution in the outer layer, and the learning value dynamic filtering is responsible for the accurate screening of effective gradient samples in the inner layer. The two are coupled in series to jointly determine the final set of effective samples participating in parameter updates.

[0223] It is important to emphasize that the fundamental value of this cascaded coupling lies in the precise control of the training signal from two orthogonal dimensions: sample distribution and gradient effectiveness. The course candidate pool addresses the question of "what difficulty distribution of samples should be exposed to the model during the current training phase," while the dynamic filtering of learning value addresses the question of "which samples in the candidate pool can provide effective learning gradients at the current model capability level." The former ensures the asymptotic stability of training at a macro level, while the latter eliminates invalid gradient noise at a micro level. Together, they enable FG-DAPO to achieve significantly better training efficiency and final performance than the standard DAPO baseline in multi-graph inference and localization tasks. The model can continuously obtain effective advantageous signals in complex multi-graph, multi-object scenarios, achieving synergistic optimization of improved inference ability and improved localization accuracy.

[0224] According to some embodiments, in step S1429 or step S1407, the parameters of the basic multi-image inference model are updated using a preset loss function based on the set of high-learning-value samples and / or the set of low-learning-value samples. Specifically, this can be achieved through step S410 or step S420.

[0225] In step S410, the parameters of the basic multi-image inference model are updated based on the set of high-learning-value samples using a preset loss function.

[0226] In this embodiment, model parameter updates are performed only on samples in the high-learning-value sample set, and groups with learning value below the threshold (i.e., samples in the low-learning-value sample set) are discarded.

[0227] In step S420, the parameters of the basic multi-image inference model are updated using a preset loss function based on the high-learning-value sample set, the low-learning-value sample set, the pre-set update weights of the high-learning-value sample set and the low-learning-value sample set.

[0228] In this embodiment of the application, when updating model parameters, in addition to samples in the high-learning-value sample set, the groups with learning values ​​below the threshold (i.e., samples in the low-learning-value sample set) are updated with reduced weights, thereby reducing invalid gradients and oscillations.

[0229] According to some embodiments, refer to Figure 4 In step S1402, step S1414 or step S1424, based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs in order to construct a candidate output group. Specifically, this can be achieved through steps S510-S530.

[0230] In step S510, the visual features and prompt word features of the current input sample are extracted.

[0231] In step S520, visual features and cue word features are fused to obtain fused features.

[0232] In step S530, based on the basic multi-image inference model, the fused features are sampled multiple times to generate multiple candidate outputs in order to construct a candidate output group.

[0233] In this embodiment, a multimodal input processing mechanism is configured for the model, which concatenates or weights the feature representations of each source, and finally inputs the fused visual features and prompt word features into the language model for output decoding.

[0234] During the output decoding process, multiple different outputs are generated for the fusion features corresponding to the same current input sample, serving as multiple candidate outputs to construct a candidate output group.

[0235] According to some embodiments, the preset reward function includes a format reward function and an accuracy reward function, and the accuracy reward function includes an image identification reward function and an IoU reward function.

[0236] Unlike rewards based solely on a single accuracy metric, this application's embodiments employ tiered, verifiable rewards to ensure project availability and prevent speculation.

[0237] The total reward consists of a format reward and an accuracy reward, with the accuracy reward further broken down into an image labeling reward and an IoU reward. The core idea is to first verify that the output is parsable and conforms to the protocol, then verify that the localization occurs on the correct image, and finally evaluate the accuracy of the bounding box. This avoids the model obtaining unreasonable rewards by using degenerate bounding boxes on incorrect images.

[0238] Among them, the format reward function Used to verify that the labels in the reasoning / answer section appear in pairs, are in the correct order, the structured blocks can be parsed, and the fields are complete; image-based reward function. The image_id (image index) used to verify that the prediction matches the ground truth (optimal matching can be used in multi-object scenarios); IoU reward function The calculation is performed only when an image labeling reward is obtained; otherwise, it is set to zero to form a hierarchical constraint.

[0239] To further illustrate the localization method based on multi-image reasoning, two specific embodiments are given below.

[0240] In one embodiment, multi-image data from the reconstructed MGrounding-630K is received synchronously, and the instruction prompt "locate the target position corresponding to the reference target in the first image in the Kth image" is received simultaneously, which is then sent to the strategy model for in-group sampling generation.

[0241] In this embodiment, the method implementation process includes the following steps: 1) Generate G candidate outputs, each of which contains an inference segment and an answer segment.

[0242] 2) First, verify the parsability of the output labels and structured results, and provide a format reward.

[0243] 3) Matching is performed based on the image identifier field in the structured results. If the image identifier is incorrect, the candidate is directly determined to have a low reward and the IoU calculation is terminated. For candidates with correct image identifiers, the IoU reward is calculated, and the total reward is synthesized.

[0244] 4) Calculate the learning value within the group and filter it according to the current stage threshold, retaining the set of samples with high learning value for updating.

[0245] The updated model exhibits the following characteristics: the inference segment focuses more on cross-image distinguishable attributes (color, texture, pose, contextual relationships, etc.), and the answer segment more consistently outputs correct image labels and accurate bounding boxes.

[0246] In another embodiment, multi-map data from the reconstructed MGrounding-630K is received synchronously, and the instruction prompt "locate the position of all pointers to the corresponding target in all maps" is received at the same time, and the output supports the one-time output of a structured list.

[0247] In this embodiment, the method implementation process includes the following steps: 1) Force "reasoning section first, answer section second", and require the answer section to output a structured list containing multiple elements.

[0248] 2) Whether there are missing targets, redundant targets, and whether the structured data is parsable are included in the format bonus and accuracy bonus.

[0249] 3) Match multiple targets and calculate image identification reward and IoU reward separately, and sum them up according to the target to form the total reward.

[0250] 4) Dynamic filtering maintains a sufficient number of differentiated candidates within the group, preventing the advantage from disappearing due to homogeneity among all candidates.

[0251] After training, the model can output a complete list of multi-objectives in a single generation, reducing omissions and inconsistencies caused by polling-based reasoning.

[0252] The localization method based on multi-image reasoning provided in this application is applicable to input formats such as multiple images, multiple viewpoints, and video frames. It can also be extended from bounding boxes to spatial output formats such as segmentation masks. Through the inference segment / answer segment protocol and hierarchical verifiable rewards, it ensures that the output is structured and parsable, and that the localization occurs on the correct image with high accuracy. The learning value dynamic filtering continuously maintains effective advantage signals and reduces homogeneous invalid updates and advantage vanishing. The course candidate pool alleviates gradient oscillations caused by difficulty heterogeneity. The output is a structured and parsable result, which is convenient for downstream systems to consume directly. The reward rules are clear and can be reused on different datasets and tasks.

[0253] The following describes an apparatus embodiment of this application, which can be used to perform the method embodiment of this application. For details not disclosed in the apparatus embodiment of this application, please refer to the method embodiment of this application.

[0254] Figure 5 A block diagram of a positioning device based on multi-image reasoning according to an exemplary embodiment is shown.

[0255] Figure 5 The apparatus shown can perform the aforementioned localization method based on multi-image reasoning according to the embodiments of this application.

[0256] like Figure 5 As shown, a localization device based on multi-image reasoning may include: See Figure 5 Referring to the preceding description, the first data unit 610 is used to rewrite the original multi-image data into sample triplets, wherein the sample triplets include an image set, prompt words, and actual location results.

[0257] The second data unit 620 is used to preprocess the sample triples to generate valid sample triples, and uses the valid sample triples as training samples to construct the training dataset.

[0258] The third data unit 630 is used to filter the training dataset using pre-built distillation cue templates and a pre-built multimodal visual language model to construct a cold start set.

[0259] The first fine-tuning unit 640 is used to perform supervised fine-tuning of the multimodal visual language model using the training dataset to obtain the initial multi-image inference model.

[0260] The second fine-tuning unit 650 is used to perform supervised fine-tuning of the initial multi-image inference model using the cold start set, so as to obtain the basic multi-image inference model.

[0261] The reinforcement learning unit 660 is used to perform reinforcement learning on the basic multi-image reasoning model using the training dataset, based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, until a preset termination condition is reached, thus obtaining the target multi-image reasoning model.

[0262] The model inference unit 670 is used to input the multi-image data to be processed into the target multi-image inference model to obtain the inference and localization results.

[0263] The device performs functions similar to those described above; other functions are described in the preceding descriptions and will not be repeated here.

[0264] This application discloses an electronic device, including: a processor; and a memory storing a computer program, which, when executed by the processor, causes the processor to execute the above-described instruction generation method.

[0265] For example, refer to Figure 6 , Figure 6 The illustrated electronic device 700 includes a processor 701 and a memory 703. The processor 701 and the memory 703 are connected, for example, via a bus 702. Optionally, the electronic device 700 may also include a transceiver 704. It should be noted that in practical applications, the transceiver 704 is not limited to one type, and the structure of this electronic device 700 does not constitute a limitation on the embodiments of this application.

[0266] Processor 701 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in this application. Processor 701 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0267] Bus 702 may include a pathway for transmitting information between the aforementioned components. Bus 702 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. Bus 702 can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 6 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0268] The memory 703 may be a ROM (Read Only Memory) or other type of static storage device capable of storing static information and instructions, RAM (Random Access Memory) or other type of dynamic storage device capable of storing information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other storage medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0269] The memory 703 is used to store application code that executes the solution of this application, and its execution is controlled by the processor 701. The processor 701 is used to execute the application code stored in the memory 703 to implement the content shown in the foregoing method embodiments.

[0270] Figure 6 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0271] This application discloses a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the processor to execute an instruction generation method.

[0272] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0273] The above are only some embodiments of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this application, and these improvements and modifications should also be considered within the scope of protection of this application.< / think> < / answer> < / answer> < / think>

Claims

1. A localization method based on multi-image reasoning, characterized in that, include: The original multi-image data is rewritten into sample triples, wherein the sample triples include an image set, a prompt word, and the actual location result; The sample triples are preprocessed to generate valid sample triples, which are then used as training samples to construct a training dataset. The training dataset is filtered using pre-built distillation cue templates and a pre-built multimodal visual language model to construct a cold start set; Using the training dataset, the multimodal visual language model is fine-tuned under supervision to obtain an initial multi-image inference model; Using the cold start set, the initial multi-image inference model is subjected to supervised fine-tuning to obtain the basic multi-image inference model; Using the training dataset, reinforcement learning is performed on the basic multi-image reasoning model according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism until the preset termination condition is reached, and the target multi-image reasoning model is obtained. The multi-image data to be processed is input into the target multi-image inference model to obtain the inference and localization results.

2. The method according to claim 1, characterized in that, The sample triples are preprocessed to generate valid sample triples, which are then used as training samples to construct a training dataset, including: Using pre-set screening and quality control rules, abnormal sample triplets in the sample triplets are removed or repaired to obtain the first sample triplet, and the sample triplets that fail to be repaired are taken as sample triplets to be processed. The sample triplets to be processed are assisted in generating or rewriting the structure using a preset multimodal model to obtain a second sample triplet. The first sample triplet and the second sample triplet are taken as valid sample triplets; The effective sample triples are used as training samples, and a training dataset is constructed based on the training samples.

3. The method according to claim 1, characterized in that, The training dataset is filtered using pre-built distillation cue templates and a pre-built multimodal visual language model to construct a cold start set, including: Extract the samples to be detected from the training dataset; The sample to be detected is input into the multimodal visual language model to obtain the first localization result; The distillation prompt template and the sample to be detected are input into the multimodal visual language model to obtain the inference trajectory and the second localization result; Calculate the positioning accuracy of the first positioning result as the first accuracy, and calculate the positioning accuracy of the second positioning result as the second accuracy; If the first precision reaches the correct threshold and the difference between the second precision and the first precision is greater than the first preset difference, the sample to be detected is stored in the inference enhancement precise sample set. If the first precision does not reach the correct threshold and the second precision reaches the correct threshold, the sample to be detected is stored in the inference error correction sample set. If neither the first precision nor the second precision reaches the correct threshold, and the difference between the second precision and the first precision is greater than the second preset difference, the sample to be detected will be stored in the inference significantly improved sample set. A cold start set is constructed based on the inference reinforcement accurate sample set, the inference error correction sample set, and the inference significantly improved sample set.

4. The method according to claim 1, characterized in that, The distillation prompt template includes role definition, input specifications, task instructions, output format, anti-leakage constraints, anti-leakage filtering rules, and / or anti-pseudo-reasoning filtering rules.

5. The method according to claim 1, characterized in that, Using the training dataset, and based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, reinforcement learning is performed on the basic multi-image reasoning model until a preset termination condition is reached, resulting in a target multi-image reasoning model, including: S1401: Select the current input sample from the training dataset; S1402: Based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs, so as to construct a candidate output group; S1403: Calculate the reward value of the multiple candidate outputs in the candidate output group according to the preset reward function; S1404: Calculate the learning value of the candidate output group based on the reward values ​​of the multiple candidate outputs and the pre-constructed learning value index; S1405: If the learning value is greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the high learning value sample set. S1406: If the learning value is not greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the low learning value sample set. S1407: Based on the set of high-learning-value samples and / or the set of low-learning-value samples, update the parameters of the basic multi-image inference model using a preset loss function; S1408: Repeat steps S1401-S1407 until the preset termination condition is met, and output the basic multi-image reasoning model after the last iteration update as the target multi-image reasoning model.

6. The method according to claim 1, characterized in that, Using the training dataset, and based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, reinforcement learning is performed on the basic multi-image reasoning model until a preset termination condition is reached, resulting in a target multi-image reasoning model, including: S1411: Using a pre-built validation set, calculate the current localization accuracy of the basic multi-image inference model, and update the current training stage, its difficulty range, and the value threshold for value filtering based on the current localization accuracy. S1412: Based on the training dataset, construct the current candidate pool according to the difficulty range corresponding to the current training stage; S1413: Select the current input sample from the current candidate pool; S1414: Based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs, so as to construct a candidate output group; S1415: Calculate the reward value of the multiple candidate outputs according to the preset reward function; S1416: Repeat steps S1413-S1415 until all training samples in the current candidate pool have been traversed; S1417: Calculate the group average reward of the candidate output groups of the training samples in the current candidate pool and sort them according to the group average reward to obtain the sorting result; S1418: Based on the sorting results, extract training samples and their output groups from the current candidate pool, and update the basic multi-image inference model using a preset loss function; S1419: Repeat steps S1411-S1418 until the preset termination condition is met, and output the basic multi-image reasoning model after the last iteration update as the target multi-image reasoning model.

7. The method according to claim 1, characterized in that, Using the training dataset, and based on a preset learning value dynamic filtering mechanism and / or a difficulty-aware course current candidate pool mechanism, reinforcement learning is performed on the basic multi-image reasoning model until a preset termination condition is reached, resulting in a target multi-image reasoning model, including: S1421: Using a pre-built validation set, calculate the current localization accuracy of the basic multi-image inference model, and update the current training stage, its difficulty range, and the value threshold for value filtering based on the current localization accuracy. S1422: Based on the training dataset, construct the current candidate pool according to the difficulty range corresponding to the current training stage; S1423: Select the current input sample from the current candidate pool; S1424: Based on the basic multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs, so as to construct a candidate output group; S1425: Calculate the reward value of the multiple candidate outputs in the candidate output group according to the preset reward function; S1426: Calculate the learning value of the candidate output group based on the reward values ​​of the multiple candidate outputs and the pre-constructed learning value index; S1427: If the learning value is greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the high learning value sample set. S1428: If the learning value is not greater than the value threshold corresponding to the current training stage, save the current input sample and its candidate output group to the low learning value sample set. S1429: Based on the set of high-learning-value samples and / or the set of low-learning-value samples, update the parameters of the basic multi-image inference model using a preset loss function; S1430: Repeat steps S1421-S1429 until the preset termination condition is met, and output the basic multi-image reasoning model after the last iteration update as the target multi-image reasoning model.

8. The method according to claim 5 or 7, characterized in that, Based on the set of high-learning-value samples and / or the set of low-learning-value samples, the parameters of the basic multi-image inference model are updated using a preset loss function, including: Based on the set of high-learning-value samples, the parameters of the basic multi-image inference model are updated using a preset loss function; or Based on the set of high-learning-value samples, the set of low-learning-value samples, the pre-set update weights of the set of high-learning-value samples and the set of low-learning-value samples, the parameters of the basic multi-image inference model are updated using a preset loss function.

9. The method according to any one of claims 5-7, characterized in that, Based on the aforementioned fundamental multi-image reasoning model, the current input sample is sampled multiple times to generate multiple candidate outputs, thereby constructing a candidate output group, including: Extract the visual features and prompt word features of the current input sample; The visual features and the cue word features are fused to obtain the fused features; Based on the aforementioned basic multi-image reasoning model, the fused features are sampled multiple times to generate multiple candidate outputs, thereby constructing a candidate output group.

10. The method according to any one of claims 5-7, characterized in that, The preset reward function includes a format reward function and an accuracy reward function. The accuracy reward function includes an image identification reward function and an IoU reward function.

11. A positioning device based on multi-image reasoning, characterized in that, include: The first data unit is used to rewrite the original multi-image data into sample triples, wherein the sample triples include an image set, prompt words, and actual location results; The second data unit is used to preprocess the sample triples to generate valid sample triples, and use the valid sample triples as training samples to construct a training dataset. The third data unit is used to filter the training dataset using a pre-built distillation cue template and a pre-built multimodal visual language model to construct a cold start set. The first fine-tuning unit is used to perform supervised fine-tuning of the multimodal visual language model using the training dataset to obtain an initial multi-image inference model. The second fine-tuning unit is used to perform supervised fine-tuning of the initial multi-image inference model using the cold start set to obtain the basic multi-image inference model. The reinforcement learning unit is used to perform reinforcement learning on the basic multi-image reasoning model using the training dataset, according to the preset learning value dynamic filtering mechanism and / or the difficulty-aware course current candidate pool mechanism, until the preset termination condition is reached, and the target multi-image reasoning model is obtained. The model inference unit is used to input the multi-image data to be processed into the target multi-image inference model to obtain the inference and localization results.

12. An electronic device, characterized in that, include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the method as described in any one of claims 1-10.

Images