Exclusion contrast reasoning method and system based on multi-modal large model, device

Abstract

The application provides a kind of based on multimodal large model's exclusion comparison reasoning method and system, equipment, the method includes: based on preset interference pool and candidate category set carries out adversarial thinking chain construction, obtains confusion interference item pool;Confusion interference item pool is based on and carries out fine granularity identification, obtains optimal confusion interference item;Based on optimal confusion interference item and true answer carries out quality verification, obtains verified optimal confusion interference item, based on verified optimal confusion interference item obtains preset exclusion prompt word;Based on preset exclusion prompt word, group relative strategy optimization algorithm and preset reward function are aligned to optimize processing to small-scale multimodal large language model, guide small-scale multimodal large language model when reasoning, compare input image data with optimal confusion interference item and exclude optimal confusion interference item.The discrimination ability is activated by introducing adversarial comparison reasoning construction and discrimination logic activation, to effectively solve the problem of semantic reasonable but insufficient discrimination.

Description

Technical Field

[0001] This application belongs to the field of computer natural language processing technology, and relates to an exclusionary comparative reasoning method, system, and device based on a multimodal large model. Background Technology

[0002] Multimodal Large Language Models (MLLMs) for image classification. Visual classification has shifted from dual-encoder VLMs (Vision-Language Models) similar to CLIP (Contrastive Language-Image Pre-training) to generative MLLMs that perform classification through instruction-tuned inference, offering greater flexibility and interpretability. While early MLLMs lagged behind dedicated VLMs in zero-shot accuracy, contemporary models now meet or surpass state-of-the-art baselines.

[0003] Multimodal large language models have demonstrated strong generalization capabilities across various vision-language tasks. By integrating Chain-of-Thought (CoT) reasoning, explicit decision-making can be achieved, thereby improving their performance in few-shot visual classification. While standard CoT performs well on visually distinct samples, it fails when candidate categories have high visual similarity to the ground truth labels. This limitation stems from a systematic bias acquired during pre-training on large-scale, diverse data: the model tends to prioritize high-frequency visual features that are general across scenarios, rather than task-specific discriminative features.

[0004] To mitigate the illusion caused by this systematic bias, recent research has focused on fine-tuning MLLMs using reinforcement learning. Some methods, such as Visual Reinforcement Fine-Tuning (FRT), explicitly incentivize the generation of longer inference chains to reveal details, but this often leads to "overthinking," where the model generates lengthy descriptions that gradually detach from visual evidence. Conversely, to avoid inference-induced illusions, Classification Reinforcement Learning (CLS-RL) methods completely abandon the inference process in pursuit of simplicity. While this approach reduces the illusion to some extent, it sacrifices interpretability and the ability to handle complex logic. Existing reinforcement learning methods struggle to balance this trade-off because they fail to address the fundamental problem of "semantically plausible but lacking discriminative power"—that is, the model can fluently describe images but cannot logically anchor the decisive evidence needed to eliminate distractors. Summary of the Invention

[0005] This application provides an exclusionary comparative reasoning method, system, and device based on a multimodal large model, which addresses the problem that existing technologies perform poorly when processing complex sample instances due to excessive focus on general features while neglecting discriminative details.

[0006] In a first aspect, this application provides an exclusionary comparative reasoning method based on a multimodal large-scale model, comprising: constructing an adversarial thought chain based on a preset interference pool and the candidate category set to obtain a pool of confusing interference items; the preset interference pool includes input image data, a candidate category set, and the true answer; the candidate category set includes several candidate interference items; the pool of confusing interference items includes several confusing interference items; performing fine-grained identification based on the pool of confusing interference items to obtain the optimal confusing interference item; performing quality verification based on the optimal confusing interference item and the true answer to obtain the verified optimal confusing interference item, and obtaining the corresponding preset exclusionary prompt word based on the verified optimal confusing interference item; performing alignment optimization processing on a small-scale multimodal large-scale language model based on the interference pool, the preset exclusionary prompt word, a group relative strategy optimization algorithm, and a preset reward function, guiding the small-scale multimodal large-scale language model to compare the input image data with the optimal confusing interference item and exclude the optimal confusing interference item during reasoning.

[0007] In one implementation of the first aspect, constructing an adversarial thought chain based on a preset interference pool and the candidate category set, and obtaining a pool of confusing interference items includes: extracting semantic embedding vectors from the input image data using a large-scale multimodal language model to obtain semantic embedding vectors of the input image data; extracting semantic embedding vectors from the candidate category set using the large-scale multimodal language model to obtain semantic embedding vectors of each candidate interference item; calculating semantic similarity between the semantic embedding vectors of the input image data and the semantic embedding vectors of each candidate interference item to obtain semantic similarity between the input image data and each candidate interference item; and selecting several candidate interference items with high semantic similarity that are not the true answer to construct a pool of confusing interference items based on the semantic similarity between the input image data and each candidate interference item.

[0008] In one implementation of the first aspect, fine-grained identification based on the pool of obfuscated interference items to obtain the optimal obfuscated interference item includes: fine-grained multimodal feature extraction based on each obfuscated interference item and the true answer to obtain corresponding multimodal features; cross-modal aligned encoding based on each of the multimodal features, and mapping the encoding results to a unified joint embedding space; calculating the visual similarity, semantic similarity, and discriminative difference difficulty corresponding to each obfuscated interference item based on the representation of each obfuscated interference item in the joint embedding space and the representation of the true answer in the joint embedding space; performing obfuscation quantification calculation based on the visual similarity, semantic similarity, and discriminative difference difficulty of each obfuscated interference item to obtain the obfuscation score of each obfuscated interference item; sorting the obfuscation scores of each obfuscated interference item according to their obfuscation scores, and selecting the obfuscated interference item with the highest score as the optimal obfuscated interference item.

[0009] In one implementation of the first aspect, quality verification based on the optimal obfuscation item and the true answer to obtain the verified optimal obfuscation item includes: calculating the confidence level based on the optimal obfuscation item and the true answer to obtain the corresponding confidence score; performing an exclusion comparison judgment based on the confidence score and a preset confidence threshold; if the confidence score is greater than or equal to the preset confidence threshold, it indicates that the judgment is passed, and the current optimal obfuscation item is taken as the verified optimal obfuscation item; if the confidence score is less than the preset confidence threshold, it indicates that the judgment is not passed, and the other obfuscation items in the obfuscation item pool are sequentially subjected to quality verification to obtain the verified optimal obfuscation item.

[0010] In one implementation of the first aspect, calculating the confidence score based on the optimal obfuscation item and the true answer to obtain the corresponding confidence score includes: calculating the semantic similarity between the optimal obfuscation item and the true answer to obtain the semantic similarity of the optimal obfuscation item; calculating the visual semantic alignment between the optimal obfuscation item and the true answer to obtain the visual semantic alignment of the optimal obfuscation item; calculating the visual distinguishability between the optimal obfuscation item and the true answer to obtain the visual distinguishability of the optimal obfuscation item; evaluating the validity of the discrimination reasoning between the optimal obfuscation item and the true answer to obtain the validity of the discrimination reasoning of the optimal obfuscation item; and performing a weighted fusion calculation based on the semantic similarity, visual semantic alignment, visual distinguishability, and validity of the discrimination reasoning to obtain the confidence score of the optimal obfuscation item; wherein, the confidence score is expressed as: The weights are configured as follows: , The function `clamp(x,0,1)` is defined as: `clamp(x,0,1)=min(max(0,1),1)`. sem S represents semantic similarity; vsa S represents visual semantic alignment. vd Indicates visual distinguishability; q R This indicates the validity of the reasoning used to make the judgment.

[0011] In one implementation of the first aspect, alignment optimization processing is performed on a small-scale multimodal large language model based on the interference pool, preset exclusionary prompts, a group relative strategy optimization algorithm, and a preset reward function. Guiding the small-scale multimodal large language model to compare the input image data with the optimal obfuscation term and exclude the optimal obfuscation term during inference includes: obtaining an input question based on the input image data and preset exclusionary prompts; generating multiple candidate prediction results for the input question based on the current small-scale multimodal large language model; calculating a scalar reward corresponding to each candidate prediction result based on the preset reward function; and calculating a scalar reward based on each candidate prediction result. The scalar reward corresponding to each candidate prediction result is used to calculate the advantage estimation function for each candidate prediction result; based on the advantage estimation function and the difference between the current small-scale multimodal large language model and the reference small-scale multimodal large language model, an objective function is constructed; alignment optimization is performed based on the objective function, and the parameters of the current small-scale multimodal large language model are updated through the aligned and optimized objective function to obtain the final small-scale multimodal large language model; the final small-scale multimodal large language model has an exclusion comparison logic, which compares the input image data with the optimal confusion interference term and excludes the optimal confusion interference term during inference.

[0012] In one implementation of the first aspect, the preset reward function includes an accuracy reward function and a format reward function, wherein the accuracy reward function is: The format reward function is: The preset reward function is: R i =R acc +R format , where R i R represents the preset reward function. acc R represents the accuracy reward function. format The format reward function is represented by the objective function: ,in, Denotes the objective function of GRPO; E represents the expected data. This represents the current small-scale multimodal large language model. Compared to the old small-scale multimodal large language model Regarding the current candidate prediction result o i The probability variation is assigned, where q represents the input problem; Ai This indicates the degree of change between the current small-scale multimodal large language model and the old small-scale multimodal large language model; This represents the cropped ratio; `clip` indicates the cropping function. Hyperparameters that control the magnitude of model updates; This represents the KL divergence between the new small-scale multimodal large language model and the reference small-scale multimodal large language model. This represents a pre-defined strategy model. This represents the hyperparameter controlling the KL divergence.

[0013] In one implementation of the first aspect, guiding the small-scale multimodal large language model to compare the input image data with the optimal obfuscation interference item and eliminate the optimal obfuscation interference item during inference includes: analyzing the input image data to obtain image features; eliminating candidate interference items in the preset interference pool that have significantly different features from the image features based on the image features to obtain an eliminated interference pool; eliminating obfuscation interference items in the eliminated interference pool that have significantly easily confused features with the image features based on preset exclusion prompt words to obtain a classification result; the classification result includes a prediction output result and an inference process.

[0014] Secondly, this application provides an exclusionary comparison reasoning system based on a multimodal large model, comprising: a confusion / interference item pool construction module, configured to construct a confusion / interference item pool based on a preset interference pool and the candidate category set; the preset interference pool includes input image data, a candidate category set, and a true answer; the candidate category set includes several candidate interference items; the confusion / interference item pool includes several confusion / interference items; a fine-grained recognition module, configured to perform fine-grained recognition based on the confusion / interference item pool to obtain the optimal confusion / interference item; a quality verification module, configured to perform quality verification based on the optimal confusion / interference item and the true answer to obtain the verified optimal confusion / interference item, and obtain the corresponding preset exclusionary prompt word based on the verified optimal confusion / interference item; and an alignment optimization module, configured to perform alignment optimization processing on a small-scale multimodal large language model based on the preset exclusionary prompt word, a group relative strategy optimization algorithm, and a preset reward function, guiding the small-scale multimodal large language model to compare the input image data with the optimal confusion / interference item and exclude the optimal confusion / interference item during reasoning.

[0015] Thirdly, this application provides an electronic device, the electronic device comprising: a memory for storing a computer program; and a processor for running the computer program to implement the exclusionary comparative reasoning method based on a multimodal large model as described above.

[0016] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described exclusionary comparative reasoning method based on a multimodal large model.

[0017] As described above, the exclusionary comparative reasoning method, system, and device based on a multimodal large model described in this application have the following beneficial effects:

[0018] This application utilizes a large-scale multimodal language model to construct an adversarial thought chain process, initially identifying candidate confusing distractors from difficult-to-distinguish categories. Subsequently, it employs Group Relative Policy Optimization (GRPO) technology, using a designed discriminative logic activation mechanism to align the small-scale multimodal language model. This mechanism integrates confidence-aware optimization and backoff strategies, injecting validated distractors as cognitive constraints into the reasoning path, thereby effectively stimulating complex elimination logic.

[0019] This application activates discriminative capabilities by introducing adversarial contrastive reasoning construction and discriminative logic activation. By reconstructing the reasoning process from a simple forward description to a dual "exclusion-contrast" paradigm, the model is forced to explicitly exclude easily confused negation terms and compare and verify verified confusing interference terms. This effectively solves the bottleneck problem and illusion phenomenon of "semantically reasonable but insufficient discriminative ability," and also realizes the bridging role between general perception and fine discriminative ability. Attached Figure Description

[0020] Figure 1 The diagram shown is a schematic representation of the hardware structure of the electronic terminal described in an embodiment of this application.

[0021] Figure 2 The diagram shown is a flowchart illustrating the exclusionary comparative reasoning method based on a multimodal large model as described in the embodiments of this application.

[0022] Figure 3 The diagram shows the complete flowchart of the exclusionary comparative reasoning method based on a multimodal large model as described in the embodiments of this application.

[0023] Figure 4 The diagram shown is a schematic diagram of the process for obtaining the obfuscation / interference item pool as described in an embodiment of this application.

[0024] Figure 5 The diagram shown is a schematic representation of the fine-grained identification process described in an embodiment of this application.

[0025] Figure 6 The diagram shown is a schematic representation of the quality verification process described in an embodiment of this application.

[0026] Figure 7 The diagram shown is a schematic representation of the alignment optimization process described in an embodiment of this application.

[0027] Figure 8 The diagram shown is a flowchart of the training process for the exclusionary comparative reasoning method described in this application embodiment.

[0028] Figure 9 The diagram shown is a structural schematic of the exclusionary comparative reasoning system based on a multimodal large model as described in an embodiment of this application.

[0029] Figure 10 The diagram shown is a structural schematic of the electronic device described in an embodiment of this application. Detailed Implementation

[0030] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0031] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the drawings only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0032] The following embodiments of this application provide an exclusionary comparative reasoning method, system, and device based on a multimodal large model, which solves the problem that the prior art performs poorly when processing complex sample instances because it focuses too much on general features and ignores discrimination details.

[0033] Reinforcement Learning in Multimodal Language Models. Reinforcement learning has become a key technology for enabling multilingual models to conform to human preferences and logical correctness. Early research attempted to apply RLHF (Reinforcement Learning from Human Feedback) to the vision domain, but its reliance on PPO (Proximal Policy Optimization) and expensive human preference data has driven a shift towards automated, rule-based reward mechanisms. Research has shown that combining simple rule-based rewards with GRPO can elicit complex reasoning without criticizing the model. This approach has been applied in the multimodal domain: Visual-RFT introduces verifiable reasoning rewards, CLS-RL directly optimizes classification accuracy, and other studies have explored perceptual consistency rewards and large-scale reinforcement learning frameworks. However, methodological gaps remain in reward design. Most methods rely on complex reward engineering or dense preference data. In contrast, this application proposes a data-centric alternative: complex discriminative logic does not require meticulously designed rewards, but rather the construction of well-contrasting perturbations. By combining base accuracy rewards with distractors generated by the Adversarial Mind Chain (Ad-CoT), GRPO-activated exclusionary contrastive reasoning is employed.

[0034] Group Relative Policy Optimization (GRPO) is a reinforcement learning algorithm that does not require an evaluator. It estimates the advantage function by normalizing the mean and standard deviation of the within-group rewards. Given a problem q, GRPO is based on the current novel policy model. Sample G candidate outputs Each output o i The scalar reward r is obtained by calculating the reward function. i Its advantage estimate A i This was obtained through standardized calculations within the group.

[0035]

[0036] in and Let represent the mean and standard deviation of the rewards within each group, respectively. This strategy uses KL divergence constraints to control the model parameters.

[0037]

[0038] in, It is the objective function of GRPO; E represents the expected value of the data; Representing the new policy model Compared to the old policy model For the answer o i The probability variation is assigned, where q represents the input problem; A iThis indicates the degree of change between the current strategy and the old strategy; It is the cropped ratio (clip is the cropping function). (Hyperparameters used to control the magnitude of model updates) It calculates the KL divergence between the new strategy model and the preset strategy model. It is a pre-defined strategy model, and It is a hyperparameter that controls the KL divergence.

[0039] While the CoT (Coding of Thought) technique has advanced multimodal large language models (MLLMs) in few-shot visual classification, it often underperforms when handling complex instances due to its overemphasis on general features and neglect of discriminative details. To address this limitation, this application proposes an exclusionary contrastive reasoning method that activates discriminative capabilities by explicitly integrating confounding terms during the reasoning process. Specifically, this application first constructs an adversarial CoT flow using a large MLLM to initially identify candidate confounding terms from difficult-to-distinguish categories. Subsequently, it employs Group Relative Policy Optimization (GRPO) to align the small-scale multimodal large language model through a designed Discriminative Logic Activation (DLA) mechanism. This mechanism integrates confidence-aware optimization and backoff strategies, injecting validated confounding terms as cognitive constraints into the reasoning path, thereby effectively stimulating complex exclusionary logic.

[0040] The exclusionary comparative reasoning method based on a multimodal large model provided in this application can be run on an electronic terminal. Figure 1 For example, Figure 1 This is a hardware block diagram of an electronic terminal for running the exclusionary comparative reasoning method based on a multimodal large model. The electronic terminal 100 includes, but is not limited to, a processing unit 101 and a storage unit 102. The processing unit 101 and the storage unit 102 are connected via a bus.

[0041] Storage unit 102 is the non-transitory computer-readable storage medium provided in this application. The storage unit stores instructions executable by at least one processing unit to cause the at least one processing unit 101 to execute the exclusionary comparative reasoning method based on a multimodal large model provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to execute the exclusionary comparative reasoning method based on a multimodal large model provided in this application.

[0042] Storage unit 102 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function. The data storage area may store image data required for exclusionary comparison inference and data created based on the use of the electronic terminal determined by exclusionary comparison inference. Furthermore, storage unit 102 may include a high-speed random access memory unit and may also include non-transient storage units, such as at least one disk storage unit, flash memory device, or other non-transient solid-state storage unit. In some embodiments, storage unit 102 may optionally include storage units remotely located relative to processing unit 101, which can be connected via a network to the electronic terminal 100 determined by exclusionary comparison inference. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0043] The electronic terminal 100 also includes an input device 103 and an output device 104. The input device 103 can receive input image data, which can be stored in the storage unit 102, so that the processing unit 101 can perform exclusionary comparison reasoning based on the image data and output the reasoned classification result through the output device 104.

[0044] The input device 103 may include, but is not limited to, fisheye cameras and cameras. The output device 104 may include, but is not limited to, liquid crystal displays (LCDs), light-emitting diode displays (LEDs), plasma displays, and touch screens. This application does not limit the scope of the embodiments.

[0045] In the embodiments of this application, the above-mentioned components of the electronic terminal 100 and Figure 1 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 1 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this application. Those skilled in the art can add or replace other components as needed.

[0046] The electronic terminal 100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The electronic terminal 100 can also be a mobile or stationary server. This application does not limit the scope of the embodiments.

[0047] The technical solutions in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0048] Figure 2 The diagram shown is a flowchart illustrating the exclusionary comparative reasoning method based on a multimodal large model as described in the embodiments of this application. Figure 3 This is a schematic diagram illustrating the complete flowchart of the exclusionary comparative reasoning method based on a multimodal large model described in an embodiment of this application. Figure 2-3 As shown, this embodiment provides an exclusionary comparative reasoning method based on a multimodal large model, the method including the following steps S201 to S204.

[0049] Step S201: Construct an adversarial thought chain based on the preset interference pool and the candidate category set to obtain a confusion interference item pool; the preset interference pool includes input image data, candidate category set and real answer; the candidate category set includes several candidate interference items; the confusion interference item pool includes several confusion interference items.

[0050] In some embodiments, the preset interference pool is a pre-set interference pool, the number of datasets in the preset interference pool is much greater than the number of datasets in the obfuscation interference item pool, and all datasets in the obfuscation interference item pool are in the preset interference pool.

[0051] Step S202: Perform fine-grained identification based on the obfuscation interference item pool to obtain the optimal obfuscation interference item. The optimal obfuscation interference item is not limited to one obfuscation interference item and can be multiple obfuscation interference items; this application does not limit it to this.

[0052] Step S203: Perform quality verification based on the optimal obfuscation item and the real answer to obtain the verified optimal obfuscation item, and obtain the corresponding preset exclusionary prompt words based on the verified optimal obfuscation item.

[0053] Step S204: Based on the preset exclusionary prompt words, the group relative strategy optimization algorithm, and the preset reward function, perform alignment optimization processing on the small-scale multimodal large language model, and guide the small-scale multimodal large language model to compare the input image data with the optimal confusion interference item and exclude the optimal confusion interference item during inference.

[0054] Existing reinforcement learning methods struggle to balance the trade-off between semantically reasonable classification results and strong discriminative ability because they fail to address the fundamental problem of "semantically reasonable but insufficiently discriminative"—that is, the model can fluently describe images but cannot logically anchor the decisive evidence needed to exclude distractors. To address this limitation, this application proposes an exclusionary contrastive reasoning method, which activates discriminative ability by introducing adversarial contrastive reasoning construction and discriminative logic activation (DLA). This method reconstructs the reasoning process from simple forward description into a dual "exclusion-contrast" paradigm, forcing the model to explicitly exclude easily confused negations and to compare and verify verified distractors. This effectively solves the bottleneck problem and illusion phenomenon of "semantically reasonable but insufficiently discriminative," and also bridges the gap between general perception and fine-grained discrimination.

[0055] Figure 4 This diagram illustrates the process for obtaining the obfuscation / interference item pool as described in an embodiment of this application. Figure 4 As shown, in one embodiment of this application, constructing an adversarial thought chain based on a preset interference pool and the candidate category set, and obtaining a pool of confusing interference items includes the following steps S2011 to S2014.

[0056] Step S2011: Based on the input image data, semantic embedding vectors are extracted using a large-scale multimodal language model to obtain the semantic embedding vectors of the input image data.

[0057] Step S2012: Based on the candidate category set, semantic embedding vectors are extracted using a large-scale multimodal language model to obtain the semantic embedding vectors of each candidate interference item.

[0058] Step S2013: Calculate semantic similarity based on the semantic embedding vector of the input image data and the semantic embedding vector of each candidate interference item to obtain the semantic similarity between the input image data and each candidate interference item.

[0059] Step S2014: Based on the semantic similarity between the input image data and each candidate interference item, select several candidate interference items with high semantic similarity that are not the real answer to construct a confusion interference item pool.

[0060] In some embodiments, this application first utilizes a large-scale MLLM to construct an adversarial thought chain process, and preliminarily identifies candidate obfuscating items from difficult-to-distinguish categories based on a preset obfuscation pool. That is, in the Ad-CoT construction stage, obfuscating items (e.g., 737-500) are obtained from the preset obfuscation item pool through a large-scale MLLM.

[0061] Standard positive CoT typically generates reasoning paths that use high-frequency, non-discriminative features to prove the true answer. This tendency leads to illusions and semantically plausible but insufficiently discriminative reasoning. To overcome this bottleneck, specific and explicit samples must be introduced to force MLLM models to verify fine-grained details through deceptive samples. As a foundational step in obtaining these key constraints, this application proposes Adversarial Ad-CoT, a data-centric mechanism specifically designed to explicitly obtain obfuscation terms. Subsequently, this application defines input instances as triples. , where x represents the input image, C represents the candidate category set (i.e., the preset interference pool), and y is the true answer.

[0062] Domain-Adaptive Distractor Pool Construction (Construction of a Confusion Distractor Pool). Directly searching for the most deceptive distractors from the candidate set C often leads to context dilution: the attention of the MLLM model is scattered by clearly irrelevant options. To alleviate this problem, this application employs a Multimodal Large Language Model (MLLM) as a semantic filter to construct a smaller distractor pool from the complete candidate set C. The specific construction steps are as follows:

[0063] First, input the target image x, and extract its semantic embedding vector e using MLLM. q ;

[0064] Secondly, iterate through each candidate distractor c in the complete distractor pool and extract its semantic embedding vector e in turn. c ;

[0065] Next, calculate the semantic similarity between the input image and the candidate distractors;

[0066] Finally, select 3-5 candidate options with high semantic similarity that are not the correct answer to construct the interference pool P;

[0067] Identifying the most confusing single interference item d* in the refined low-noise environment of P. Unlike the previous coarse scan, this step performs a fine-grained comparison. The acquisition steps are as follows:

[0068] Fine-grained multimodal feature extraction. For the correct answer and each distractor, hierarchical visual features (from low-level texture to high-level semantics) are extracted, and structured attributes (color, shape, material, etc.) are decomposed and cross-modal aligned encoding is performed to construct a joint embedding space.

[0069] Confusion quantification calculation. It comprehensively calculates visual similarity, semantic similarity, and discriminative difference difficulty: the first two measure "how similar they are," and the latter measures "how difficult they are to distinguish." The three are multiplied to obtain the final confusion score, ensuring that the selected distractors are both highly similar and difficult to distinguish.

[0070] Selection of interference items: Sort by confusion score and select the interference item with the highest score as the optimal interference item d*.

[0071] This selection strategy ensures that the distractors are not random errors, but rather confusingly similar-looking items, forcing subsequent reasoning processes to abandon general descriptions and instead focus on examining discriminative details.

[0072] Figure 5 The diagram shown is a schematic representation of the fine-grained identification process described in an embodiment of this application. Figure 5 As shown, in one embodiment of this application, fine-grained identification based on the obfuscation interference item pool to obtain the optimal obfuscation interference item includes the following steps S2021 to S2025.

[0073] Step S2021: Based on each obfuscated interference item and the true answer, perform fine-grained multimodal feature extraction to obtain the corresponding multimodal features.

[0074] Step S2022: Perform cross-modal aligned encoding based on each of the multimodal features, and map the encoding results to a unified joint embedding space.

[0075] Step S2023: Based on the representations of each obfuscating interference item in the joint embedding space and the representation of the true answer in the joint embedding space, calculate and obtain the visual similarity, semantic similarity, and discriminative difference difficulty corresponding to each obfuscating interference item.

[0076] Step S2024: Perform confusion quantification calculation based on the visual similarity, semantic similarity and discriminative difference difficulty of each of the confusion interference items to obtain the confusion score of each of the confusion interference items.

[0077] Step S2025: Sort the confusion scores of each of the aforementioned confusion interference items according to their confusion scores, and select the confusion interference item with the highest score as the optimal confusion interference item.

[0078] In some implementations, the sorting method may be sorting from high to low according to their confusion scores, or sorting from low to high according to their confusion scores, and this application is not limited to this.

[0079] Figure 6 This is a schematic diagram of the quality verification process described in an embodiment of this application. Figure 6 As shown, in one embodiment of this application, quality verification is performed based on the optimal obfuscation item and the true answer to obtain the verified optimal obfuscation item, which includes the following steps S2031 to S2034.

[0080] Step S2031: Calculate the confidence score based on the optimal obfuscation item and the true answer to obtain the corresponding confidence score.

[0081] Step S2032: Based on the confidence score and the preset confidence threshold, perform an exclusion comparison judgment.

[0082] Step S2033: If the confidence score is greater than or equal to the preset confidence threshold, it means that the judgment is passed, and the current optimal obfuscation item is taken as the verified optimal obfuscation item.

[0083] Step S2034: If the confidence score is less than the preset confidence threshold, it means that the judgment has failed. Then, the other obfuscation items in the obfuscation item pool are sequentially subjected to quality verification to obtain the verified optimal obfuscation item.

[0084] In some embodiments, if there are no other obfuscation items in the obfuscation item pool with a confidence score lower than the preset confidence threshold, it indicates that there are no effective obfuscation items in the current obfuscation item pool, that is, the current sample data (preset obfuscation pool) lacks effective obfuscation items (either too obvious or ambiguous). In such cases, this application needs to retain its original real image annotation data as standard training samples and strictly avoid injecting customized exclusion prompts. This strategy prevents the model from learning to fabricate differences between indistinguishable concepts and ensures that the exclusion logic is trained only on high-quality, discriminative evidence.

[0085] In one embodiment of this application, the confidence score is calculated based on the optimal obfuscation item and the true answer to obtain the corresponding confidence score, which includes the following steps S20311 to S20315.

[0086] Step S20311: Calculate the semantic similarity between the optimal obfuscation item and the true answer to obtain the semantic similarity of the optimal obfuscation item.

[0087] Step S20312: Calculate the visual semantic alignment based on the optimal obfuscation item and the true answer to obtain the visual semantic alignment of the optimal obfuscation item.

[0088] Step S20313: Calculate the visual distinguishability based on the optimal confusion item and the true answer to obtain the visual distinguishability of the optimal confusion item.

[0089] Step S20314: Evaluate the validity of the discrimination reason based on the optimal obfuscation item and the true answer, and obtain the validity of the discrimination reason of the optimal obfuscation item.

[0090] Step S20315: Based on the semantic similarity, visual semantic alignment, visual distinguishability, and the validity of the discrimination reason, a weighted fusion calculation is performed to obtain the confidence score of the optimal confusion interference item. The confidence score is expressed as:

[0091]

[0092] The weights are configured as follows: , The function clamp(x,0,1) is defined as: clamp(x,0,1)=min(max(0,1),1); S sem S represents semantic similarity; vsa Indicates visual semantic alignment; S vd Indicates visual distinguishability; q R This indicates the validity of the reasoning used to make the judgment.

[0093] In some embodiments, this application designs and activates the discrimination logic to verify the quality of the generated interference items through quality checks. Obtaining a deceptive interference item is merely a prerequisite. Ensuring its effectiveness and effectively transferring the discrimination capability to small-scale MLLMs is crucial. This application proposes a collaborative scheme that includes quality verification, hint injection, and robustness fallback mechanisms. Table 1 below shows the confidence scoring criteria for the iterative optimization mechanism described in one embodiment of this application.

[0094] Table 1

[0095]

[0096] A confidence-aware iterative optimization mechanism is implemented. To ensure the logical distinguishability of the confusing interference item d*, this application introduces a verification module. Large-scale MLLM generates discriminative reasons for comparing y and d*, employing a strict "exclusion comparison" template: "Unlike [d*] which presents [attribute A], this [y] has features of [attribute B]." Simultaneously, the model... Figure 3 A standardized reliability score S∈[0,1] is defined to quantify the effectiveness of the visual discrimination. The function formula is as follows:

[0097]

[0098] The weight configuration is as follows: , ;

[0099] The clamp(x,0,1) function is defined as: clamp(x,0,1)=min(max(0,1),1);

[0100] S semrepresents semantic similarity: which calculates the proximity between the name of the correct answer and the name of the distractor in the semantic space.

[0101] S vsa represents visual-semantic alignment: which performs cross-modal matching between the input image and the text description of the distractor respectively, to measure "how much this image looks like the category described by the distractor". If the visual features of the image are highly aligned with the text embedding of the distractor, it indicates that the image itself has the visual features of the distractor category.

[0102] S vd represents visual distinguishability: which calculates the matching degree between the image and the correct answer, minus the matching degree between the image and the distractor, to obtain a "discrimination boundary". The smaller the boundary, the closer the distance between the image and the correct answer as well as the distractor, and the more difficult it is to distinguish them.

[0103] q R represents discriminative rationale validity: which evaluates whether the generated discriminative rationale contains an effective comparison structure, a specific visual attribute description, and factual consistency with the knowledge base. A high-quality rationale means that a truly reliable discrimination basis has been found.

[0104] Based on the above scoring functions, an iterative mechanism as a quality gate is introduced. The present application sets a strict confidence threshold T to achieve optimal selection. If S<T, the system will reject the current d* and return to the candidate pool to evaluate other options. This feedback loop ensures that the final adversarial chain-of-thought samples are both visually challenging and logically consistent.

[0105] Exclusive prompt injection. On the basis of verified high-quality distractors, the present application implements a customized injection strategy to activate the implicit discriminative logic of the model. Different task scenarios are established through domain-specific system prompts, and more importantly, the verified confusing distractor d* is directly injected into the training query as an explicit cognitive constraint. Specifically, the present application appends the instruction: "Hint: Pay special attention to distinguishing the target from its closest confusing appearance: d*. Analyze the specific visual features that exclude d*". It should be clarified that this injection strategy is only applied in the training phase; in the inference phase, only standard queries are received without distractor prompts. The performance improvement stems from the "exclusion-comparison" reasoning ability internalized by the model through GRPO alignment. As a cognitive anchor, this injection forces small-scale MLLMs to shift from the general description mode to the exclusive comparison mode. It forces the model to focus on the specific evidence verified in the refinement phase without requiring architecture modifications.

[0106] Robustness is enhanced through a fallback mechanism. Finally, to ensure system robustness, this application handles the case where the candidate pool is exhausted but has not reached the strict threshold T. This means that the samples lack effective obfuscation terms (either too obvious or ambiguous). In such cases, this application retains its original, ground-labeled data as standard training samples, strictly refraining from injecting customized exclusion prompts. This strategy prevents the model from learning to fabricate differences between indistinguishable concepts, ensuring that the exclusionary logic is trained only on high-quality, discriminative evidence.

[0107] Figure 7 The diagram shown is a schematic representation of the alignment optimization process described in an embodiment of this application. Figure 7 As shown, in one embodiment of this application, the small-scale multimodal large language model is aligned and optimized based on the preset exclusionary prompt words, the group relative strategy optimization algorithm and the preset reward function. The process of guiding the small-scale multimodal large language model to compare the input image data with the optimal confusion interference item and exclude the optimal confusion interference item during inference includes the following steps S2041 to S2046.

[0108] Step S2041: Obtain the input question based on the input image data and preset exclusionary prompt words;

[0109] Step S2042: Generate multiple candidate prediction results for the input question based on the current small-scale multimodal large language model;

[0110] Step S2043: Calculate the scalar reward corresponding to each candidate prediction result based on the preset reward function;

[0111] Step S2044: Calculate the advantage estimation function for each candidate prediction result based on the scalar reward corresponding to each candidate prediction result;

[0112] Step S2045: Based on the advantage estimation function and the difference between the current small-scale multimodal large language model and the reference small-scale multimodal large language model, construct the objective function;

[0113] Step S2046: Perform alignment optimization processing based on the objective function, update the parameters of the current small-scale multimodal large language model through the objective function after alignment optimization, and obtain the final small-scale multimodal large language model; the final small-scale multimodal large language model has an exclusion comparison logic inside, which compares the input image data with the optimal confusion interference item and excludes the optimal confusion interference item during inference.

[0114] In one embodiment of this application, the preset reward function includes an accuracy reward function and a format reward function.

[0115] The accuracy reward function is:

[0116]

[0117] The reward function for the specified format is:

[0118]

[0119] The preset reward function is:

[0120] R i =R acc +R format

[0121] Among them, R i R represents the preset reward function. acc R represents the accuracy reward function. format Representation of the reward function format;

[0122] The objective function is:

[0123]

[0124] in, Denotes the objective function of GRPO; E represents the expected data. This represents the current small-scale multimodal large language model. Compared to the old small-scale multimodal large language model Regarding the current candidate prediction result o i The probability variation is assigned, where q represents the input problem; A i This indicates the degree of change between the current small-scale multimodal large language model and the old small-scale multimodal large language model; This represents the cropped ratio; `clip` indicates the cropping function. Hyperparameters that control the magnitude of model updates; This represents the KL divergence between the new small-scale multimodal large language model and the reference small-scale multimodal large language model. This represents a pre-defined strategy model. This represents the hyperparameter controlling the KL divergence.

[0125] In one embodiment of this application, guiding the small-scale multimodal large language model to compare the input image data with the optimal obfuscation term and eliminate the optimal obfuscation term during inference includes the following steps S2051 to S2053.

[0126] Step S2051: Analyze the input image data to obtain image features;

[0127] Step S2052: Based on the image features, eliminate candidate interference items in the preset interference pool that have significantly different features from the image features, and obtain the interference pool after elimination;

[0128] Step S2053: Based on preset exclusion prompts, exclude the confusing interference items in the excluded interference pool that have obvious easily confused features with the image features, and obtain the classification result; the classification result includes the prediction output result and the reasoning process.

[0129] In the policy optimization phase, this application trains a small-scale MLLM using basic rewards via GRPO. It then injects custom cue words to inject obfuscating distractors into the model's "exclusion-contrast" logic.

[0130] Figure 8 The diagram shown is a flowchart of the training process for the exclusionary comparative reasoning method described in an embodiment of this application. Figure 8 As shown, when fine-tuning the model based on the GRPO algorithm in this application, the reward function is designed and passed as a parameter to the GRPO algorithm. Please refer to the algorithm principle of GRPO technology described above. The reward function includes an accuracy reward function and a format reward function. Here, the accuracy reward is calculated by comparing the prediction results of the existing model with the standard answer input in the dataset of this application (reference architecture). Figure 3 The input is compared to the format reward; the format reward is actually a constraint on the output format of the model's output, which includes the thought process and the output answer, for example, <think> thought process< / think> <answer> Output the answer< / answer> .

[0131] In some embodiments, to match small-scale MLLMs with constructed Ad-CoT data, this application combines GRPO with a simplified reward mechanism of accuracy and format, which is sufficient to activate exclusionary reasoning when combined with highly similar distractors. While previous methods relied on manually designed complex reward functions to teach reasoning steps, this method, by injecting customized cue words (see Table 3), learns exclusionary contrast logic with only simple accuracy and format rewards, thereby aligning small-scale models.

[0132] Accuracy reward. This binary signal is used to penalize incorrect final answers. This application validates the model output with the answer:

[0133]

[0134] Formatting Rewards. To ensure the reliability of downstream parsing, this application imposes structured constraints on model generation. This is rigorously regulated by employing a binary formatting reward mechanism. Specifically, a formatting reward is awarded if and only if the model output completely encapsulates the reasoning process and the final result within a given format. <think>and <answer>A positive reward of 1 is only awarded when the tag is within the specified range. Any deviation from this template will result in a reward of 0. The mathematical expression is:

[0135]

[0136] Total reward. The total reward r used for calculating relative advantage. i It is the unweighted sum of accuracy bonus and format bonus:

[0137] r i =R acc +R format

[0138] This ensures that the optimization objective simultaneously incentivizes the correctness of the inference result and the structural validity of the inference process. Furthermore, the accuracy reward acts as a key filter for data quality. In rare cases, when Ad-CoT might retrieve visually irrelevant distractors, the model's attempt to force a comparison may lead to incoherent inference or incorrect predictions. By penalizing these instances (R... acc =0), the optimization process can naturally eliminate noise and only strengthen the effective discrimination logic.

[0139] Unlike methods that rely on complex, hand-designed reward functions, this method demonstrates that the underlying signals are sufficiently effective when combined with high-quality data. This mechanism effectively incentivizes the model to internalize the "exclusion-comparison" logic. Therefore, the model implicitly addresses the bottleneck of "semantically reasonable but insufficiently discriminative" results, deriving the correct answer through reliable discriminative evidence rather than generalized features.

[0140] This application designs an adversarial thought chain process using large-scale MLLM. It efficiently acquires high-quality obfuscating distractors, providing a rigorous data foundation for discriminative reasoning. This application proposes an exclusionary contrastive reasoning method that combines GRPO with a novel discriminative logic activation mechanism. By injecting validated cognitive distractors as constraints, it activates exclusionary-contrast logic, effectively solving the bottleneck problem of "semantically reasonable but discriminatively insufficient" and the illusion phenomenon.

[0141] It should be noted that although the above embodiments of this application specifically use Qwen3-VL-30B-A3B-Think as a large-scale multimodal language model for image semantic understanding and obtaining the optimal obfuscation term, the technical solution of this application is not limited to this. In alternative embodiments, the multimodal language model can also be similar visual multimodal language models such as Kimi-VL-A3B, GLM-4.5-Air, Kimi-VL series, and Llama4 series.

[0142] Although the above embodiments of this application specifically use Qwen2.5-VL-7B-Instruct as the backbone for training a small-scale multimodal large language model, the technical solution of this application is not limited to this. In alternative embodiments, the small-scale multimodal large language model can also be similar to Qwen2.5-VL-72B-Instruct, DeepSeek-VL2-7B, GLM-4V-9B, etc.

[0143] The following embodiment illustrates the above-described exclusionary comparative reasoning method based on a multimodal large model. In this embodiment, the large model used to generate the obfuscating distractors is Qwen3-VL-30B-A3B-Think. The construction of the adversarial thought chain dataset is an offline, one-off process. The following are the implementation steps of the cue word technology in Ad-CoT.

[0144] You are an EXPERT_ROLE expert.

[0145] Task

[0146] Analyze the specific photo of "{answer}" and extract the distractor item "{chosen_candidates}" from the candidate list.

[0147] Execution steps (mind chain)

[0148] 1. Construct an interference pool (rough screening)

[0149] Scan the candidate list provided above.

[0150] Choose 3 to 5 categories that are semantically or visually closest to "{answer}" (e.g., {{POOL_CRITERIA}}).

[0151] Constraint: The correct answer "{answer}" must not appear in this pool.

[0152] 2. Most confusing distractors (fine-tuned selection)

[0153] The visual features of the image are compared with the categories in the interference pool.

[0154] Choose the single category (d*) that is most difficult to distinguish from the target.

[0155] 3. Extract discriminative reasons

[0156] Identify and distinguish key visual nuances between the real label and the selected distractors.

[0157] The basis for using this strict template is: unlike [d*] which presents [attribute A], this {answer} has the characteristics of [attribute B].

[0158] Constraint: [Attribute A] and [Attribute B] focus on {{VISUAL_FOCUS}}.

[0159] 4. Configuration reliability scoring

[0160] The confidence level and visual discrimination effectiveness of the selected interference items are scored.

[0161] Confidence interval:

[0162] 0.0 - 0.3: Low confidence level.

[0163] 0.3 - 0.7: Medium confidence level.

[0164] 0.7 - 1.0: High confidence (extremely deceptive).

[0165] Output format (strict XML)

[0166] You must output only the following XML blocks:

[0167] <distractor_pool> Here are 3-5 candidates.< / distractor_pool>

[0168] <confusing_distractor> Hard negative item name< / confusing_distractor>

[0169] <confidence_score> 0.0-1.0< / confidence_score>

[0170] <rationale> Unlike [d *] which appears in [attribute A], this "{answer}" has the characteristics of [attribute B].< / rationale>

[0171] Table 2 below shows the instantiation of domain-specific prompts. For domain-adaptive adversarial CoT prompt design, to avoid repetition, this application presents a general prompt template applicable to all datasets. This template contains four domain-specific variables (marked with double curly braces), which are dynamically adjusted based on dataset characteristics. The "chosen_candidates" list fully includes all candidates related to the corresponding question. In this embodiment, the specific values ​​used to populate each benchmark placeholder in the general prompt template are shown in Table 2, based on 11 datasets:

[0172] Table 2

[0173]

[0174] During the GRPO fine-tuning phase, this application uses Qwen2.5-VL-7B-Instruct as the backbone MLLM and employs LoRA (Low-Rank Adaptation) technology to update specific model parameters to improve training efficiency. Specific optimization details are as follows:

[0175] Batch size: This application uses a batch size of 8 per GPU and a gradient accumulation step size of 2, ultimately achieving a global effective batch size of 64.

[0176] GRPO constraint: KL divergence coefficient Set to 0.04 to effectively constrain the policy and update within a stable range.

[0177] Optimization threshold: For the confidence-aware optimization loop, this application empirically applies a strict confidence threshold. Set to 0.7 to filter out low-quality inference paths.

[0178] Reward Mechanism: The reward function in this application only depends on accuracy and format consistency, and does not rely on a complex reward model.

[0179] Table 3 shows the training and cue word configurations for the 11 benchmark datasets used in the evaluation. All models were trained using a strict few-shot training scheme. Cue construction: As shown in the rightmost column, a composite cue structure was used. Each input contains a dataset-specific basic question (e.g., "Aircraft type is...") and a fixed exclusion cue. The cue word is marked with {confusion term} as a dynamic placeholder. During training, this placeholder is instantiated based on the specific confusion category identified by the Ad-CoT process (e.g., when the target is "Boeing 737-200", {confusion term} is replaced with "Boeing 737-500"), thus explicitly guiding the model to perform fine-grained discriminative reasoning.

[0180] Table 3

[0181]

[0182] The scope of protection of the exclusionary comparative reasoning method based on multimodal large models described in this application is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this application is included within the scope of protection of this application.

[0183] This application also provides an exclusionary comparison reasoning system based on a multimodal large model. The exclusionary comparison reasoning system based on a multimodal large model can implement the exclusionary comparison reasoning method based on a multimodal large model described in this application. However, the implementation device of the exclusionary comparison reasoning method based on a multimodal large model described in this application includes, but is not limited to, the structure of the exclusionary comparison reasoning system based on a multimodal large model listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this application are included within the protection scope of this application.

[0184] like Figure 9 As shown, this embodiment provides an exclusionary comparison reasoning system based on a multimodal large model. The system 900 includes: a confusion interference item pool construction module 901, a fine-grained identification module 902, a quality verification module 903, and an alignment optimization module 904.

[0185] The obfuscation interference item pool construction module 901 is configured to construct an obfuscation interference item pool based on a preset interference pool and the candidate category set; the preset interference pool includes input image data, a candidate category set, and the true answer; the candidate category set includes several candidate interference items; and the obfuscation interference item pool includes several obfuscation interference items.

[0186] The fine-grained identification module 902 is configured to perform fine-grained identification based on the pool of obfuscated interference items to obtain the optimal obfuscated interference item.

[0187] The quality verification module 903 is configured to perform quality verification based on the optimal obfuscation item and the real answer, obtain the verified optimal obfuscation item, and obtain the corresponding preset exclusionary prompt words based on the verified optimal obfuscation item.

[0188] The alignment optimization module 904 is configured to perform alignment optimization processing on the small-scale multimodal large language model based on the preset exclusionary prompt words, the group relative strategy optimization algorithm and the preset reward function, so as to guide the small-scale multimodal large language model to compare the input image data with the optimal confusion interference item and exclude the optimal confusion interference item during inference.

[0189] It should be noted that the functions or operations of the confusion interference item pool construction module 901, fine-grained identification module 902, quality verification module 903, and alignment optimization module 904 described in the embodiments of this application correspond one-to-one with the steps in the exclusionary comparison reasoning method based on multimodal large models described above, so they will not be repeated here.

[0190] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0191] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of this application, depending on actual needs. For example, the functional modules / units in the various embodiments of this application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0192] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0193] This application also provides an electronic device. Figure 10 The diagram shown is a structural schematic of the electronic device 1000 described in an embodiment of this application. Figure 10 As shown, in this embodiment, the electronic device 1000 includes a memory 1010 and a processor 1020.

[0194] The memory 1010 is used to store computer programs; preferably, the memory 1010 includes various media that can store program code, such as ROM, RAM, magnetic disk, USB flash drive, memory card or optical disk.

[0195] Specifically, memory 1010 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) and / or cache memory. Electronic device 1000 may further include other removable / non-removable, volatile / non-volatile computer system storage media. Memory 1010 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this application.

[0196] The processor 1020 is connected to the memory 1010 and is used to execute the computer program stored in the memory 1010 so that the electronic device 1000 executes the exclusionary comparison reasoning method based on a multimodal large model as described in any embodiment of this application.

[0197] Optionally, the processor 1020 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0198] Optionally, in this embodiment, the electronic device 1000 may further include a display 1030. The display 1030 is communicatively connected to the memory 1010 and the processor 1020, and is used to display the graphical user interface (GUI) of the image processing method described in this application embodiment.

[0199] This application also provides a computer-readable storage medium. Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor. The program can be stored in a computer-readable storage medium, which is a non-transitory medium, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state drive (SSD)).

[0200] In summary, the exclusionary comparative reasoning method, system, and device based on a multimodal large model described in this application have the following beneficial effects:

[0201] This application utilizes a large-scale multimodal language model to construct an adversarial thought chain process, initially identifying candidate confusing distractors from difficult-to-distinguish categories. Subsequently, it employs Group Relative Policy Optimization (GRPO) technology, using a designed discriminative logic activation mechanism to align the small-scale multimodal language model. This mechanism integrates confidence-aware optimization and backoff strategies, injecting validated distractors as cognitive constraints into the reasoning path, thereby effectively stimulating complex elimination logic.

[0202] This application activates discriminative ability by introducing adversarial contrastive reasoning construction and discriminative logic activation. By reconstructing the reasoning process from a simple forward description to a dual "exclusion-contrast" paradigm, the model is forced to explicitly exclude easily confused negation terms and compare and verify verified confusing interference terms. This effectively solves the bottleneck problem and illusion phenomenon of "semantically reasonable but insufficient discriminative ability", and also realizes the bridging role between general perception and fine discriminative ability.

[0203] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0204] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.< / answer> < / think>

Claims

1. A method for exclusionary comparative reasoning based on a multimodal large model, characterized in that, include: Based on the preset interference pool and the candidate category set, an adversarial thought chain is constructed to obtain a pool of confusing interference items; The preset interference pool includes input image data, a candidate category set, and the true answer; the candidate category set includes several candidate interference items; the confusion interference item pool includes several confusion interference items. Fine-grained identification is performed based on the aforementioned pool of obfuscated interference items to obtain the optimal obfuscated interference item; Quality verification is performed based on the optimal obfuscation items and the actual answers to obtain the verified optimal obfuscation items, and corresponding preset exclusionary prompt words are obtained based on the verified optimal obfuscation items. Based on the preset exclusionary prompts, the group relative strategy optimization algorithm, and the preset reward function, alignment optimization processing is performed on the small-scale multimodal large language model, guiding the small-scale multimodal large language model to compare the input image data with the optimal confusion interference item and exclude the optimal confusion interference item during inference.

2. The exclusionary comparative reasoning method based on a multimodal large model according to claim 1, characterized in that, Based on the preset interference pool and the candidate category set, an adversarial thought chain is constructed to obtain the confusion interference item pool, including: Based on the input image data, a large-scale multimodal language model is used to extract semantic embedding vectors to obtain the semantic embedding vectors of the input image data. Based on the candidate category set, semantic embedding vectors are extracted using a large-scale multimodal language model to obtain the semantic embedding vectors of each candidate interference item. Semantic similarity is calculated based on the semantic embedding vector of the input image data and the semantic embedding vector of each candidate interference item to obtain the semantic similarity between the input image data and each candidate interference item. Based on the semantic similarity between the input image data and each candidate interference item, a pool of candidate interference items with high semantic similarity that are not the true answer is constructed.

3. The exclusionary comparative reasoning method based on a multimodal large model according to claim 1, characterized in that, Fine-grained identification based on the aforementioned pool of obfuscated interference items to obtain the optimal obfuscated interference item includes: Fine-grained multimodal feature extraction is performed based on each confusing interference item and the true answer to obtain the corresponding multimodal features; Cross-modal aligned encoding is performed based on the aforementioned multimodal features, and the encoding results are mapped to a unified joint embedding space; The visual similarity, semantic similarity, and discriminative difference difficulty corresponding to each obfuscation item are calculated based on the representation of each obfuscation item in the joint embedding space and the representation of the true answer in the joint embedding space. The confusion metric is calculated based on the visual similarity, semantic similarity, and discriminative difference difficulty of each of the aforementioned confusion interference items to obtain the confusion score of each of the aforementioned confusion interference items; The confusion scores of each of the aforementioned confusion interference items are sorted according to their confusion scores, and the confusion interference item with the highest score is selected as the optimal confusion interference item.

4. The exclusionary comparative reasoning method based on a multimodal large model according to claim 1, characterized in that, Quality verification is performed based on the optimal obfuscation terms and the actual answer. The verified optimal obfuscation terms include: Confidence is calculated based on the optimal obfuscation terms and the true answer to obtain the corresponding confidence score; An exclusion comparison judgment is made based on the confidence score and the preset confidence threshold. If the confidence score is greater than or equal to the preset confidence threshold, it means that the judgment is passed, and the current optimal obfuscation item is taken as the verified optimal obfuscation item. If the confidence score is less than the preset confidence threshold, it means that the judgment has failed. Then, the other obfuscation items in the obfuscation item pool are sequentially subjected to quality verification to obtain the verified optimal obfuscation item.

5. The exclusionary comparative reasoning method based on a multimodal large model according to claim 4, characterized in that, Based on the optimal obfuscation terms and the true answer, confidence scores are calculated to obtain the corresponding confidence ratings, including: Semantic similarity is calculated based on the optimal obfuscation item and the true answer to obtain the semantic similarity of the optimal obfuscation item; Visual semantic alignment is calculated based on the optimal obfuscation item and the true answer to obtain the visual semantic alignment of the optimal obfuscation item; Visual distinguishability is calculated based on the optimal distractor and the true answer to obtain the visual distinguishability of the optimal distractor. The validity of the discrimination reason is evaluated based on the optimal obfuscation item and the true answer to obtain the validity of the discrimination reason of the optimal obfuscation item; A weighted fusion calculation is performed based on the semantic similarity, visual semantic alignment, visual distinguishability, and the validity of the discrimination reason to obtain the confidence score of the optimal confusion interference item; wherein, the confidence score is expressed as: The weights are configured as follows: , The function `clamp(x,0,1)` is defined as: `clamp(x,0,1)=min(max(0,1),1)`. sem S represents semantic similarity; vsa Indicates visual semantic alignment; S vd Indicates visual distinguishability; q R This indicates the validity of the reasoning used to make the judgment.

6. The exclusionary comparative reasoning method based on a multimodal large model according to claim 1, characterized in that, Alignment optimization is performed on a small-scale multimodal large language model based on the preset exclusionary prompt words, group relative strategy optimization algorithm, and preset reward function. This guides the small-scale multimodal large language model to compare the input image data with the optimal obfuscation term and exclude the optimal obfuscation term during inference, including: The input question is obtained based on the input image data and preset exclusion prompts; Based on the current small-scale multimodal large language model, multiple candidate prediction results are generated for the input question; Calculate the scalar reward corresponding to each candidate prediction result based on a preset reward function; Calculate the advantage estimation function for each candidate prediction result based on the scalar reward corresponding to each candidate prediction result; Based on the aforementioned advantage estimation function and the differences between the current small-scale multimodal large language model and the reference small-scale multimodal large language model, an objective function is constructed; Alignment optimization is performed based on the objective function, and the parameters of the current small-scale multimodal large language model are updated through the aligned and optimized objective function to obtain the final small-scale multimodal large language model. The final small-scale multimodal large language model has an exclusion comparison logic, which compares the input image data with the optimal obfuscation interference item during inference and excludes the optimal obfuscation interference item.

7. The exclusionary comparative reasoning method based on a multimodal large model according to claim 6, characterized in that, The preset reward function includes an accuracy reward function and a format reward function. The accuracy reward function is: The reward function in the specified format is: The preset reward function is: R i =R acc +R format Among them, R i R represents the preset reward function. acc R represents the accuracy reward function. format Representation of the reward function format; The objective function is: in, Denotes the objective function of GRPO; E represents the expected data. This represents the current small-scale multimodal large language model. Compared to the old small-scale multimodal large language model Regarding the current candidate prediction result o i The probability variation is assigned, where q represents the input problem; A i This indicates the degree of change between the current small-scale multimodal large language model and the old small-scale multimodal large language model; This represents the cropped ratio; `clip` indicates the cropping function. Hyperparameters that control the magnitude of model updates; This represents the KL divergence between the new small-scale multimodal large language model and the reference small-scale multimodal large language model. This represents a pre-defined strategy model. This represents the hyperparameter controlling the KL divergence.

8. The exclusionary comparative reasoning method based on a multimodal large model according to claim 1, characterized in that, Guiding the small-scale multimodal large language model to compare the input image data with the optimal obfuscation term and eliminate the optimal obfuscation term during inference includes: Based on the input image data, image features are obtained through analysis. Based on the image features, candidate interference items in the preset interference pool that have significantly different features from the image features are excluded to obtain the excluded interference pool; Based on preset exclusion prompts, confusing interference items with obvious and easily confused features with the image features in the excluded interference pool are excluded to obtain classification results; the classification results include prediction output results and inference process.

9. A system for exclusionary comparative reasoning based on a multimodal large model, characterized in that, include: The obfuscation interference item pool construction module is configured to construct an obfuscation interference item pool based on a preset interference pool and the candidate category set; The preset interference pool includes input image data, a candidate category set, and the true answer; the candidate category set includes several candidate interference items; the confusion interference item pool includes several confusion interference items. The fine-grained identification module is configured to perform fine-grained identification based on the pool of obfuscated interference items to obtain the optimal obfuscated interference item; The quality verification module is configured to perform quality verification based on the optimal obfuscation item and the real answer, obtain the verified optimal obfuscation item, and obtain the corresponding preset exclusionary prompt words based on the verified optimal obfuscation item. The alignment optimization module is configured to perform alignment optimization processing on the small-scale multimodal large language model based on the preset exclusionary prompt words, the group relative strategy optimization algorithm, and the preset reward function, so as to guide the small-scale multimodal large language model to compare the input image data with the optimal confusion interference item and exclude the optimal confusion interference item during inference.

10. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for running the computer program to implement the exclusionary comparative reasoning method based on a multimodal large model as described in any one of claims 1 to 8.

Images