Multi-modal image forensics model training method, system, device and storage medium based on multi-stage data reconstruction and preference reinforcement learning

Abstract

The application discloses a multi-modal image forensics model training method and system based on multi-stage data reconstruction and preference reinforcement learning, an equipment and a storage medium, and particularly relates to the technical field of image forgery detection. The method acquires image forensics data, reconstructs the image forensics data into structured evidence containing fields such as abnormal bounding boxes, and generates a data set constructed by action, observation and reasoning logic thinking labels; a multi-dimensional training sample set covering label perception, local area and difficult negative samples is constructed based on the structured evidence; a pre-trained visual language model is inputted, multi-stage supervised fine-tuning with a label perception ratio decreasing and a free sample ratio increasing is performed, and a candidate strategy model is outputted; the model is called to generate answers to construct fine-grained preference pairs, combined rewards are calculated according to indexes such as format and explanation consistency, and the candidate strategy model is subjected to reinforcement optimization according to the rewards to output a final model. The system is used for implementing the above method.

Description

Technical Field

[0001] This invention relates to the field of image forgery detection technology, specifically to a training method, system, device, and storage medium for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning. Background Technology

[0002] With the development of diffusion models, generative adversarial networks, and multimodal content generation technologies, the number of AI-generated images and deepfake images continues to increase. The visual quality of these forged images is constantly improving, and some forged images are highly indistinguishable from real images under human observation. This poses more stringent identification requirements for scenarios such as internet content review, media credibility authentication, electronic evidence identification, and public safety.

[0003] Existing image forgery detection technologies have substantial limitations in practical applications. Traditional binary classification models only output the probability of authenticity, blocking the tracing path to specific forged areas and underlying judgment criteria, resulting in poor interpretability of the evidence collection process. While object detection-based methods can locate abnormal areas, they have technical blind spots when comprehensively judging multi-dimensional evidence collection logic such as structural integrity, local texture, and lighting and shadow. Existing visual language models typically use free text as annotation signals, leading to serious discrepancies in the model's description of the same anomaly. They are prone to directly outputting high-frequency label templates from the training set as shortcuts, thus completely ignoring the observation of the actual visual content of the image. At the same time, conventional supervised fine-tuning modes often only reward based on the final judgment conclusion, causing the model to easily produce false positives for normal areas and technical defects such as contradictions between evidence and interpretation when faced with local forgeries or easily confused real defects. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a method, system, device, and storage medium for training a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, comprising the following steps: S1, acquiring image forensics data with authenticity labels and original natural language interpretations, reconstructing the original natural language interpretations into structured evidence data containing anomalous modules, anomalous types, anomalous targets, anomalous bounding boxes, and anomalous descriptions, generating thought annotations containing actions, observations, and reasoning logic for the structured evidence data, and constructing a standardized forensics dataset; S2, based on the structured evidence data in the standardized forensics dataset, extracting features to construct features including label-aware samples, label-free samples, module-focused samples, and local... S3. Input the training sample set into the pre-trained visual language model and perform multi-stage supervised fine-tuning training, gradually decreasing the proportion of label-perceived samples and gradually increasing the proportion of label-free samples and difficult negative samples, and output the candidate policy model; S4. For the same image input, call the candidate policy model to generate multiple candidate answers, select candidate answers with similar overall form and fine-grained evidence defects to construct preference pairs, extract and calculate the combined reward by combining format, label consistency, evidence quality and evidence-interpretation consistency, perform reinforcement optimization on the candidate policy model according to the preference pairs and combined rewards, and output the multimodal image evidence collection model.

[0006] Preferably, image evidence data with authenticity labels and original natural language interpretations are acquired. The original natural language interpretations are reconstructed into structured evidence data containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions. Thinking annotations containing actions, observations, and reasoning logic are generated for the structured evidence data. A standardized evidence dataset is constructed, including: identifying anomaly analysis modules in the original natural language interpretations and extracting anomaly modules, anomaly types, anomaly targets, and anomaly descriptions; locating the regions of anomaly targets in the image, generating and normalizing normalized anomaly bounding boxes; combining anomaly modules, anomaly types, anomaly targets, anomaly descriptions, and anomaly bounding boxes to construct evidence items, and summarizing all evidence items to form an evidence array; generating a comprehensive interpretation based on the evidence array, filling in conclusions according to authenticity labels, combining to generate structured evidence data, and performing quality checks; extracting the object, region, and attribute features to be checked from the structured evidence data to generate action annotations; extracting the phenomenon features observed from the image evidence data to generate observation annotations; determining whether the observation features constitute the conditions for evidence collection and excluding defects in actual shooting; generating reasoning logic annotations; and merging action annotations, observation annotations, and reasoning logic annotations to generate thinking annotations.

[0007] Preferably, based on the structured evidence data in the standardized evidence dataset, features are extracted to construct a training sample set including label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and difficult negative samples. This includes: parsing the structured evidence data in the standardized evidence dataset to read image, authenticity labels, anomalous bounding boxes, and anomalous evidence fields; combining image evidence data, authenticity labels, and structured evidence data to construct label-aware samples; removing authenticity labels from the image evidence data to construct label-free samples; specifying a target analysis module from the structured evidence data, deleting evidence corresponding to other modules besides the target analysis module from the structured evidence data, and constructing module-focused samples; and expanding the sample set according to a preset expansion ratio. The abnormal bounding boxes are expanded and cropped from the image forensics data to generate local images. The abnormal bounding boxes are then mapped to the local image coordinate system to construct local region samples. Occlusion is applied to the image regions corresponding to the abnormal bounding boxes, and it is determined whether the abnormal regions are still visible. If they are visible, the evidence data is retained and the certainty of the abnormal description is reduced. If they are completely invisible, the corresponding evidence data is deleted, and occlusion consistency samples are constructed. Normal specified module regions within forged images are extracted from the image forensics data, or real image regions with motion blur, illumination distortion, and resolution degradation are extracted to construct difficult negative samples. Sample quality inspection and format standardization are performed on each of the above-constructed samples, including checking field completeness, bounding box legality, and consistency between evidence and interpretation. The final training sample set is then output.

[0008] Preferably, the training sample set is input into a pre-trained visual language model, and multi-stage supervised fine-tuning training is performed, gradually decreasing the proportion of label-aware samples and gradually increasing the proportion of label-free samples and hard negative samples, to output a candidate policy model. This includes: a first stage, configuring the label-aware samples in the training sample set to a first proportion, inputting the training sample set with the first proportion into the pre-trained visual language model, and supervising the output of formatted structured evidence data and thought annotations through an autoregressive cross-entropy loss function; a second stage, decreasing the proportion of label-aware samples in the first stage training sample set, increasing the proportion of label-free samples, local region samples, occlusion consistency samples, and hard negative samples in the training sample set and configuring them to a second proportion, inputting the training sample set with the second proportion into the pre-trained visual language model trained in the first stage for supervised fine-tuning, and outputting a candidate policy model.

[0009] Preferably, for the same image input, multiple candidate answers are generated by calling the candidate strategy model. Candidate answers with similar overall form and fine-grained evidentiary defects are selected to construct preference pairs. This includes: changing decoding parameters to sample the same image input multiple times, generating multiple sets of candidate answers by calling the candidate strategy model; constructing a rule-based scorer containing positive and negative scoring dimensions, inputting candidate answers into the rule-based scorer, and outputting the total score and defect type of the candidate answers; positive scoring dimensions include conclusions consistent with the truth label, valid bounding boxes, and specific anomaly descriptions; negative scoring dimensions include conclusions inconsistent with the truth label, bounding boxes exceeding boundaries, false positives in normal regions, and inconsistent comprehensive interpretations with anomaly descriptions; extracting the candidate answer with the highest total score as the preferred answer, and extracting candidate answers with defects such as incorrect bounding boxes, vague evidence, introduction of new anomalies, incorrect module types, and false positives in normal regions from the remaining candidate answers as rejection answers, and combining the preferred answer and the rejection answer to construct preference pairs.

[0010] Preferably, the combined reward is calculated by integrating format, label consistency, evidence quality, and evidence-interpretation consistency. This includes: extracting output labels from candidate answers and checking field integrity to generate a format reward; extracting output conclusions from candidate answers, comparing the output conclusions with the authenticity labels, and generating a label consistency reward; extracting the legality of abnormal modules, the legality of abnormal types, the specificity of abnormal targets, and the legality of abnormal bounding boxes from candidate answers, and summing them to generate an evidence quality reward; extracting overlapping features of abnormal targets in the comprehensive interpretation and cited evidence array from candidate answers, checking for new abnormalities not present in the cited evidence array, and generating an evidence-interpretation consistency reward; and combining the format reward, label consistency reward, evidence quality reward, and evidence-interpretation consistency reward with semantic similarity reward and length repetition control reward according to preset weights to generate a combined reward.

[0011] Preferably, the candidate policy model is enhanced and optimized based on the preference pair and combined reward to output a multimodal image forensics model, including: calling the current candidate policy model to sample multiple sets of responses for the image input in the preference pair, and calculating the combined reward corresponding to each set of responses; calculating the mean and standard deviation of the combined reward within the same set of responses to generate the intra-group relative advantage; setting the candidate policy model output from the multi-stage supervised fine-tuning training stage as the reference model, and extracting the divergence features between the current candidate policy model and the reference model to generate constraint terms; updating the parameters of the current candidate policy model based on the intra-group relative advantage and constraint terms, and outputting the multimodal image forensics model.

[0012] A training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning is used to execute the aforementioned training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning. The system includes: an evidence reconstruction module, used to acquire image forensics data with authenticity labels and original natural language interpretations; reconstructing the original natural language interpretations into structured evidence data containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions; generating thought annotations containing actions, observations, and reasoning logic for the structured evidence data; and constructing a standardized forensics dataset. A sample construction module is used to extract features from the structured evidence data in the standardized forensics dataset and construct samples including label-aware samples, label-free samples, and module samples. The system comprises a training sample set focusing on samples, local region samples, occlusion consistency samples, and hard negative samples; a supervised training module, which inputs the training sample set into a pre-trained visual language model and performs multi-stage supervised fine-tuning training, gradually decreasing the proportion of label-perceived samples and gradually increasing the proportion of label-free samples and hard negative samples, to output a candidate policy model; and a reinforcement learning module, which, for the same image input, calls the candidate policy model to generate multiple candidate responses, selects candidate responses with similar overall form and fine-grained evidence defects to construct preference pairs, extracts and calculates combined rewards based on format, label consistency, evidence quality, and evidence-interpretation consistency, and performs reinforcement optimization on the candidate policy model based on preference pairs and combined rewards, outputting a multimodal image forensics model.

[0013] The training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning also includes: an electronic device, which includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-mentioned training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning.

[0014] A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method.

[0015] The technical effects and advantages of this invention are as follows: (1) In the technical solution described in this invention, by acquiring image evidence data and reconstructing the original natural language into structured evidence data containing abnormal modules, types, targets and bounding boxes, thinking annotations of actions, observations and reasoning logic are generated, and free text annotations are forcibly converted into unified and computable supervision signals, effectively separating inspection actions from visual fact descriptions; furthermore, based on the features extracted from the above structured evidence data, a multi-dimensional training sample set such as local region samples, occlusion consistency samples and difficult negative samples is constructed, forcing the model to focus on specific abnormal modules and fine-grained local regions, effectively suppressing the phenomenon of the model randomly reporting in invisible areas or falsely reporting real defects such as normal hands and faces.

[0016] (2) In the technical solution described in this invention, a multi-dimensional training sample set is input into a pre-trained visual language model, and multi-stage supervised fine-tuning training is performed, in which the proportion of label-perceived samples gradually decreases and the proportion of label-free samples gradually increases. This allows the model to gradually transition from interpreting based on labels to autonomously collecting evidence based on images, effectively cutting off the shortcut path of the model directly outputting conclusions using high-frequency label templates. Furthermore, the candidate strategy model is called to generate multiple candidate answers to construct preference pairs, and the combined reward is extracted and optimized by combining format, label consistency and interpretation consistency. This enables the model to accurately distinguish fine-grained quality differences such as correct conclusions but incorrect positioning or correct evidence but contradictory interpretations, significantly improving the standardization and credibility of the structured evidence and natural language interpretation output by the multimodal model.

[0017] Of course, any product implementing this invention does not necessarily need to achieve all of the advantages described above at the same time. Attached Figure Description

[0018] Figure 1 This is a flowchart of the training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, as presented in this invention.

[0019] Figure 2 This is a flowchart illustrating the training process of the multimodal image forensics model in an embodiment of the present invention.

[0020] Figure 3 This is a flowchart of the structured annotation process in an embodiment of the present invention.

[0021] Figure 4 This is a flowchart illustrating the construction process of six types of training samples in an embodiment of the present invention.

[0022] Figure 5 This is a flowchart of the training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, as presented in this invention. Detailed Implementation

[0023] This application's embodiments address the problems of existing visual models' heavy reliance on label shortcuts, chaotic output evidence structure, and inconsistencies between fine-grained localization and interpretation when conducting image forensics training, through a multimodal image forensics model training method, system, device, and storage medium based on multi-stage data reconstruction and preference reinforcement learning.

[0024] Example 1; please refer to Figure 1 This invention provides a technical solution: a training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, comprising the following steps: S1, acquiring image forensics data with authenticity labels and original natural language interpretations, reconstructing the original natural language interpretations into structured evidence data containing anomalous modules, anomalous types, anomalous targets, anomalous bounding boxes, and anomalous descriptions, generating thought annotations containing actions, observations, and reasoning logic for the structured evidence data, and constructing a standardized forensics dataset; S2, based on the structured evidence data in the standardized forensics dataset, extracting features to construct features including label-aware samples, label-free samples, module-focused samples, and local regions. S3. Input the training sample set of sample, occlusion consistency sample, and hard negative sample; S4. Input the training sample set into the pre-trained visual language model, perform multi-stage supervised fine-tuning training with the proportion of label-aware sample gradually decreasing and the proportion of label-free sample and hard negative sample gradually increasing, and output the candidate policy model; S5. For the same image input, call the candidate policy model to generate multiple candidate answers, select candidate answers with similar overall form and fine-grained evidence defects to construct preference pairs, extract and calculate the combined reward by combining format, label consistency, evidence quality, and evidence-interpretation consistency, perform reinforcement optimization on the candidate policy model according to the preference pairs and combined reward, and output the multimodal image evidence collection model.

[0025] In this implementation plan, step S1 involves a deep format standardization and logical decomposition process for the original evidence data. After acquiring the original data containing the image to be analyzed, authenticity labels, and natural language explanations, the free text explanations are parsed and reconstructed into structured evidence data. Here, structured evidence data refers to extracting and mapping non-standardized text into unified computational fields containing anomaly analysis modules, specific anomaly types, anomaly target objects, anomaly normalized bounding boxes, and specific anomaly descriptions, thus solving the problem that free text is difficult to form standardized supervision signals. Subsequently, thought annotations are generated for the structured evidence data. Thought annotations refer to explicitly decomposing the model's analysis process into three independent logical layers: actions to check the preparation area, observations to describe actual visible phenomena, and reasoning to exclude defects in normal shooting. This prompts the model to form an evidence collection process that first checks the image facts, then describes visual features, and finally judges whether the facts constitute forged evidence, thus completing the construction of a standardized evidence dataset.

[0026] Step S2 performs multi-dimensional feature mutations based on the aforementioned standardized dataset to construct a training sample set that eliminates feature shortcuts. Specifically, based on the dimensional features of the structured evidence data, label-aware samples retaining the authenticity labels are constructed for initial format learning, label-free samples removing the authenticity labels for unannounced autonomous anti-counterfeiting learning, and module-focused samples that extract a single analysis module and remove evidence from other modules. Simultaneously, abnormal bounding boxes are extracted and proportionally expanded to construct local region samples. Masks or random noise are applied within the abnormal bounding boxes to construct occlusion consistency samples. Furthermore, normal module regions or image regions with real defects such as lighting distortion in forged images are extracted to construct difficult negative samples. Difficult negative samples refer to visually similar artifacts but actually caused by natural textures or physical factors, making them easily confused. The combination of these six types of samples forces the model to focus on specific abnormal modules and fine-grained local regions, suppressing the model's operation of fabricating invisible anomalies based on background or training memory.

[0027] Step S3 utilizes the constructed multi-class sample set to perform multi-stage supervised fine-tuning training on the pre-trained visual language model, i.e., updating network parameters in stages using the autoregressive cross-entropy loss function. In the first stage of training, a high proportion of label-aware samples are configured to supervise the pre-trained visual language model in learning the classification of each analysis module, bounding box coordinate representation, and output structure format of action observation inference. In the second stage, the proportion of label-aware samples in the input data is gradually reduced, while the proportion of label-free samples, local region samples, occlusion consistency samples, and difficult negative samples is significantly increased. This multi-stage supervised fine-tuning training mechanism means that the model first establishes the underlying mapping relationship between input and output using known labels, then removes label prompts and increases the detection difficulty, prompting the model to gradually break away from its dependence on high-frequency label templates and autonomously perform fine-grained anomaly localization and authenticity judgment based on the visual content of the image, ultimately outputting a candidate strategy model with basic evidence collection capabilities.

[0028] Step S4 is the process of fine-grained preference alignment and multi-dimensional joint constraint optimization of the candidate strategy model. For the same image input, the decoding parameters are changed, and the candidate strategy model is invoked to generate multiple sets of candidate answers. The highest-scoring preferred answer is extracted using a rule-based scorer. Answers with minor defects such as bounding box out-of-bounds errors, lack of evidence, or the introduction of non-image anomalies are rejected from the remaining answers. The preferred and rejected answers are combined to form preference pairs. Preference pairs are data pairs that are highly similar in overall form but have clear quality differences in localization or interpretation, used to teach the model to distinguish fine-grained erroneous outputs under extremely high similarity. Subsequently, the field integrity of the candidate answers is extracted to generate a format reward; the output conclusion is compared to generate a label consistency reward; the legality of the anomaly module and bounding box values ​​is accumulated to generate an evidence quality reward; the overlapping features of the target object cited in the interpretation content are calculated to generate an evidence and interpretation consistency reward. The results of the above calculations are weighted and summed to generate a combined reward. Finally, the relative advantage within the group is calculated by combining the preference pairs and the combined reward, and the candidate strategy model parameters are updated to output a multimodal image forensics model. (See also...) Figure 2 The process begins with data cleaning and preprocessing of the original input images, authenticity labels, and natural language interpretations. This is followed by structured annotation reconstruction and conceptual annotation construction, completing the construction of standardized evidence-gathering data. Next, six types of training samples are constructed based on the constructed data, with strict sample quality control and format standardization. In the model fine-tuning phase, a first-stage SFT training focusing on label-aware samples and a second-stage SFT training focusing on increasing the proportion of label-free and hard samples are performed sequentially. Finally, DPO optimization is applied to the constructed data using preferences, and GRPO reinforcement learning is executed based on an intra-group relative reward optimization strategy model. After multi-dimensional evaluation and achieving the required standards, the final evidence-gathering model is output.

[0029] Specifically, image evidence data with authenticity labels and original natural language interpretations are acquired. The original natural language interpretations are reconstructed into structured evidence data containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions. Thinking annotations containing actions, observations, and reasoning logic are generated for the structured evidence data. A standardized evidence dataset is constructed, including: identifying anomaly analysis modules in the original natural language interpretations and extracting anomaly modules, anomaly types, anomaly targets, and anomaly descriptions; locating the regions of anomaly targets in the images and generating and normalizing normalized anomaly bounding boxes; combining anomaly modules, anomaly types, anomaly targets, anomaly descriptions, and anomaly bounding boxes to construct evidence items, and summarizing all evidence items to form an evidence array; generating a comprehensive interpretation based on the evidence array, filling in conclusions according to authenticity labels, combining to generate structured evidence data, and performing quality checks; extracting the object, region, and attribute features to be checked from the structured evidence data to generate action annotations; extracting the phenomenon features observed from the image evidence data to generate observation annotations; determining whether the observation features constitute the conditions for evidence collection and excluding defects in actual shooting; generating reasoning logic annotations; and merging action annotations, observation annotations, and reasoning logic annotations to generate thinking annotations.

[0030] In this implementation plan, for the reconstruction and annotation of structured evidence data, it is first necessary to establish a multi-dimensional artifact and anomaly analysis system that includes structural integrity analysis, local artifact analysis, imaging consistency analysis, physical consistency analysis, semantic consistency analysis, and special evidence analysis, and to pre-set a set of allowed anomaly types for each module. When generating normalized bounding boxes to locate anomalous targets, if the original image width is W and the height is H, the normalized coordinates are calculated according to... Perform calculations; for evidence that is difficult to pinpoint precisely, such as global noise or overall tone, the bounding box can be set to null. During the generation of thought labels, the action part describes the object, region, and attributes to be examined; the observation part only describes the phenomena actually observed in the image; and the reasoning part explains why the phenomenon constitutes evidence. It also mandates that the model exclude normal shooting defects caused by real imaging factors such as motion blur, compression, occlusion, depth of field, or lighting. Simultaneously, the thought content maintains semantic relevance to the anomaly description but does not repeat it verbatim, reducing the probability that the model directly generates anomaly descriptions based on labels. Please refer to [link to relevant documentation]. Figure 3This process employs a bottom-up construction logic. First, the preprocessed data undergoes an anomaly analysis module to identify any extractable anomalous evidence. If such evidence exists, candidate evidence items containing module, type, target, and description are extracted, and anomalous regions are located, generating bounding boxes normalized to [0, 1000]. Subsequently, the normalized bounding boxes and candidate items are combined to construct independent evidence items, which are then aggregated into an evidence array. Based on this evidence array, a comprehensive interpretation is generated, and conclusions are filled in according to authenticity labels. Finally, a structured annotation result is output, which must undergo rigorous quality checks before output, including format, labels, bounding boxes, and consistency between evidence and interpretation.

[0031] Specifically, based on structured evidence data in the standardized evidence dataset, features are extracted to construct a training sample set including label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and difficult negative samples. This includes: parsing the structured evidence data in the standardized evidence dataset to read image, authenticity labels, anomalous bounding boxes, and anomalous evidence fields; combining image evidence data, authenticity labels, and structured evidence data to construct label-aware samples; removing authenticity labels from the image evidence data to construct label-free samples; specifying a target analysis module from the structured evidence data, deleting evidence corresponding to other modules besides the target analysis module from the structured evidence data, and constructing module-focused samples; and expanding the sample set according to a preset expansion ratio. The abnormal bounding boxes are expanded and cropped from the image forensics data to generate local images. The abnormal bounding boxes are then mapped to the local image coordinate system to construct local region samples. Occlusion is applied to the image regions corresponding to the abnormal bounding boxes, and it is determined whether the abnormal regions are still visible. If they are visible, the evidence data is retained and the certainty of the abnormal description is reduced. If they are completely invisible, the corresponding evidence data is deleted, and occlusion consistency samples are constructed. Normal specified module regions within forged images are extracted from the image forensics data, or real image regions with motion blur, illumination distortion, and resolution degradation are extracted to construct difficult negative samples. Sample quality inspection and format standardization are performed on each of the above-constructed samples, including checking field completeness, bounding box legality, and consistency between evidence and interpretation. The final training sample set is then output.

[0032] In this implementation scheme, when constructing six training sample sets by extracting features, multiple training variants are generated from the same original image to enhance specific anti-spoofing capabilities. Label-aware samples are used in the early stages of training to learn label meanings and structured formats; label-free samples are preferentially extracted from data with accurate and conflict-free bounding boxes to cultivate autonomous observation capabilities under unprovoked conditions. Module-focused samples reduce the risk of misclassifying structural anomalies as lighting or texture anomalies by limiting the model to check only specific modules and deleting evidence from other modules. Local region samples enhance the recognition of subtle structural errors such as fingers and local edges by expanding the anomalous bounding box outwards at a preset ratio of 5%–100% (preferably 10%–50%) of its width and height and cropping the local image. Occlusion consistency samples train the model to adhere to the principle of not reporting what is not seen by applying solid color occlusion, random noise, blurring, or neighborhood filling to anomalous regions and deleting corresponding evidence when the region is completely invisible. Hard negative samples specifically extract normal designated modules from forged images, or real images containing motion blur, strong shadows, and reflective distortion, to suppress false reporting of common normal parts by the model. (See also...) Figure 4 The process uses structured annotation parsing as a unified input, reading images and various annotation fields, and then distributing them to six parallel sample construction branches. These branches include: a label-aware branch that retains the standard answer; a label-free branch that removes true labels; a module-focused branch that retains only evidence for the corresponding module; a local region branch that crops and enlarges bounding boxes; and a hard negative sample branch that specifically selects normal modules or easily confused regions to construct empty evidence or suppress false positives. Specifically, in the occlusion consistency sample branch, after applying occlusion, it is necessary to determine whether the anomaly is still visible. If it is visible, the evidence is retained but its certainty is reduced; if it is completely invisible, the corresponding evidence is directly deleted. All samples constructed from these branches must be aggregated and subjected to sample quality control and format standardization before they can be used as compliant input data for subsequent training.

[0033] Specifically, the training sample set is input into a pre-trained visual language model, and multi-stage supervised fine-tuning training is performed, gradually decreasing the proportion of label-aware samples and gradually increasing the proportion of label-free samples and hard negative samples, to output a candidate policy model. This includes: First stage: configuring the label-aware samples in the training sample set to a first proportion; inputting the training sample set with the first proportion into the pre-trained visual language model; and supervising the output of formatted structured evidence data and thought annotations through an autoregressive cross-entropy loss function. Second stage: decreasing the proportion of label-aware samples in the first stage training sample set; increasing the proportion of label-free samples, local region samples, occlusion consistency samples, and hard negative samples in the training sample set and configuring them to a second proportion; inputting the training sample set with the second proportion into the pre-trained visual language model trained in the first stage for supervised fine-tuning; and outputting a candidate policy model.

[0034] In this implementation plan, during multi-stage supervised fine-tuning training, the first stage primarily uses labeled, perceptual samples, with a primary proportion of 50%–90%. The focus is on learning the structured output formats of conclusions, evidence, and explanations; the classification of each analysis module; bounding box coordinate representation; and the action-observation-reasoning format. In the first stage, the model output is supervised using an autoregressive cross-entropy loss function, the formula of which is: Where I represents the input image and X represents the text command. This represents the t-th target output word. This indicates whether the corresponding word element participates in the loss calculation. During this process, the loss can be masked for the word element corresponding to the system prompt word, and only the model output is supervised. The second stage uses the model from the first stage as the initial model, increases the proportion of free labeled samples to 30%–80%, and increases the proportion of high-quality local samples, occlusion consistency samples, and difficult negative samples while reducing high-frequency labeled perception samples. This improves the model's ability to make autonomous judgments under unlabeled conditions and reject normal regions.

[0035] Specifically, for the same image input, a candidate strategy model is invoked to generate multiple candidate responses. Candidate responses with similar overall form and fine-grained evidentiary defects are selected to construct preference pairs. This includes: changing decoding parameters to sample the same image input multiple times, invoking the candidate strategy model to generate multiple sets of candidate responses; constructing a rule-based scorer containing positive and negative scoring dimensions, inputting candidate responses into the rule-based scorer, and outputting the total score and defect type of the candidate responses; positive scoring dimensions include conclusions consistent with the truth label, valid bounding boxes, and specific anomaly descriptions; negative scoring dimensions include conclusions inconsistent with the truth label, bounding boxes exceeding boundaries, false positives in normal regions, and inconsistencies between the overall interpretation and the anomaly description; extracting the candidate response with the highest total score as the preferred answer, and extracting candidate responses with defects such as incorrect bounding boxes, vague evidence, introduction of new anomalies, incorrect module types, and false positives in normal regions from the remaining candidate responses as rejection answers, and combining the preferred and rejection answers to construct preference pairs.

[0036] In this implementation scheme, when constructing fine-grained preference pairs, 4 to 8 candidate responses are generated by sampling the same input image multiple times by changing the temperature, top-p, top-k, random seed, maximum generation length, or system prompt format. Candidate responses can also be generated by a supervised fine-tuning model or other visual language models. A scorer is then used to score the candidate responses. Positive scoring dimensions include complete labels, valid JSON, non-empty evidence from forged samples, specific targets, and explanations citing evidence. Negative scoring dimensions include bounding box out-of-bounds errors, parsing failures, obvious forgery traces in real sample reports, or severe template duplication. When selecting rejection answers, garbled text or completely irrelevant text is strictly avoided. Instead, answers with an overall format close to the preferred answer are selected from the remaining responses, but those with clear defects in a few key dimensions, such as correct conclusions but incorrect bounding boxes, correct conclusions but vague evidence, explanations introducing new anomalies, incorrect module types, or misreporting of normal / occluded regions. This allows the model to learn fine-grained evidence quality preferences.

[0037] Specifically, the combined reward is calculated by integrating format, label consistency, evidence quality, and evidence-interpretation consistency. This includes: extracting output labels from candidate answers and checking field completeness to generate a format reward; extracting output conclusions from candidate answers, comparing the output conclusions with the authenticity labels, and generating a label consistency reward; extracting the legality of abnormal modules, the legality of abnormal types, the specificity of abnormal targets, and the legality of abnormal bounding boxes from candidate answers, and summing them to generate an evidence quality reward; extracting overlapping features of abnormal targets in the comprehensive interpretation and cited evidence array from candidate answers, checking for new abnormalities not present in the cited evidence array, and generating an evidence-interpretation consistency reward; and combining the format reward, label consistency reward, evidence quality reward, and evidence-interpretation consistency reward with semantic similarity reward and length repetition control reward according to preset weights to generate a combined reward.

[0038] In this implementation plan, the multi-dimensional combined reward is calculated by weighting multiple sub-rewards, specifically including format reward, label consistency reward, evidence quality reward, evidence and interpretation consistency reward, semantic similarity reward, and length duplication control reward. The evidence quality reward is accumulated according to the following rules: 1.0 point is added for non-empty evidence of forged samples, 0.5 points are added for modules and types belonging to the allowed set, 0.5 points are added for specific target descriptions, 0.5 points are added for including specific visual phenomena, and 0.5 points are added for valid bounding boxes; for genuine samples, an empty evidence set receives a positive evaluation when there are no reliable anomalies. The evidence and interpretation consistency reward is obtained by extracting module names, anomaly keywords, and comparing them with the comprehensive explanation, prohibiting the introduction of new anomalies not present in the evidence array. The total reward calculation formula for the multi-dimensional combined reward is as follows: Where each R corresponds to a sub-reward mentioned above, and each w is a corresponding preset weight, which can be configured during implementation. .

[0039] Specifically, the candidate policy model is enhanced and optimized based on preference pairs and combined rewards to output a multimodal image forensics model. This includes: calling the current candidate policy model to sample multiple sets of responses for the image input in the preference pair, and calculating the combined rewards corresponding to the multiple sets of responses respectively; calculating the mean and standard deviation of the combined rewards within the same set of responses to generate intra-group relative advantage; setting the candidate policy model output from the multi-stage supervised fine-tuning training stage as the reference model, and extracting the divergence features between the current candidate policy model and the reference model to generate constraint terms; updating the parameters of the current candidate policy model based on the intra-group relative advantage and constraint terms, and outputting the multimodal image forensics model.

[0040] In this implementation plan, when performing reinforcement optimization and outputting the final model, the current policy model samples multiple responses for the same input and calculates the total reward for each response. Then, it calculates the relative advantage within the group based on the mean and standard deviation of the rewards for the same group of responses. During the process of updating the network parameters of the policy model based on the relative advantage within the group, a KL divergence constraint term relative to the reference model must be added to prevent the model from deviating from its original language generation and visual understanding capabilities during reinforcement learning. After reinforcement learning optimization, the final multimodal image forensics model will sequentially perform global observation, structural inspection, local artifact inspection, imaging consistency inspection, physical consistency inspection, semantic consistency inspection, reflection on defects in real-world shooting, and evidence summarization during actual inference. The final output is a structured forensics explanation containing modules, types, targets, bounding boxes, and descriptions. When deployed to business users, the annotation process can be hidden; when debugging the model, the annotations are retained for analysis and inspection.

[0041] Example 2; please refer to Figure 5A training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning is provided to execute the multimodal image forensics model training method based on multi-stage data reconstruction and preference reinforcement learning described in the embodiments. The system includes: an evidence reconstruction module for acquiring image forensics data with authenticity labels and original natural language interpretations; reconstructing the original natural language interpretations into structured evidence data containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions; generating thought annotations containing actions, observations, and reasoning logic for the structured evidence data; and constructing a standardized forensics dataset. A sample construction module is also provided for extracting features from the structured evidence data in the standardized forensics dataset to construct samples including label-aware samples and label-free samples. The system comprises three training sample sets: a module focusing on samples, local region samples, occlusion consistency samples, and hard negative samples; a supervised training module, which inputs the training sample set into a pre-trained visual language model and performs multi-stage supervised fine-tuning training, gradually decreasing the proportion of labeled perception samples and gradually increasing the proportion of labeled free samples and hard negative samples, to output a candidate policy model; and a reinforcement learning module, which, for the same image input, calls the candidate policy model to generate multiple candidate responses, selects candidate responses with similar overall forms and fine-grained evidence defects to construct preference pairs, extracts and calculates combined rewards based on format, label consistency, evidence quality, and evidence-interpretation consistency, and performs reinforcement optimization on the candidate policy model based on preference pairs and combined rewards, outputting a multimodal image forensics model.

[0042] In this implementation scheme, the system's evidence reconstruction module is specifically used to perform the parsing operation of the original evidence data. It extracts the natural language interpretation mapping from free text into specific fields containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions. It then concatenates three layers of logic—action, observation, and reasoning—to generate thought labels and output a standardized evidence dataset. The system's sample construction module takes the aforementioned standardized evidence dataset and calls image cropping and masking processing programs. By proportionally expanding and cropping the original image and bounding boxes, applying occlusion masks, removing authenticity labels, or specifying anomaly analysis modules, it constructs corresponding label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and difficult negative samples. These are then concatenated to form a multi-dimensional training sample set.

[0043] The system's supervised training module is used to fine-tune the parameters of the pre-trained visual language model. This module controls the input distribution of the training sample set. In the first stage, labeled perception samples are input according to a first proportion, and the autoregressive cross-entropy loss is calculated. In the second stage, the proportion of labeled perception samples is reduced, while free labeled samples and hard negative samples are increased to a second proportion for further supervised fine-tuning. The model weights are calculated and updated, and a candidate policy model is output. The system's reinforcement learning module is used to optimize the parameters of the candidate policy model. This module changes the decoding parameters, calls the candidate policy model to generate multiple sets of candidate answers, runs scoring rules to extract preferred and rejected answers to construct preference pairs, and simultaneously extracts output labels, conclusions, bounding box legality, and overlapping features of interpretation to generate corresponding format, label consistency, evidence quality, and evidence-interpretation consistency combination rewards. The relative advantage within the group is calculated, and divergence constraints are fused to iteratively update the parameters of the candidate policy model, outputting a multimodal image forensics model.

[0044] The training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning also includes: an electronic device, which includes a processor and a memory, the memory storing a computer program, and the processor executing the computer program to implement the steps of the above-mentioned training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning.

[0045] In this implementation scheme, the electronic device constitutes the underlying hardware foundation for executing the aforementioned image forensics model training method. The internal memory of the electronic device stores the original image forensics data, the training sample sets constructed at each stage, the network weight tensors generated at different training stages, and the computer program code containing the operational logic of each system module. During runtime, the processor inside the electronic device reads the program code from the memory, allocates computing resources at the physical circuit level, and sequentially executes mathematical operations such as text field parsing and mapping, bounding box coordinate normalization, neural network forward inference and gradient backpropagation, and weighted summation of combined reward values. This completes the actual physical operation of each functional module of the model training system and the persistent deployment of network parameters.

[0046] A computer-readable storage medium storing computer instructions that, when executed by one or more processors, cause the one or more processors to perform the steps of the method.

[0047] In this implementation scheme, a computer-readable storage medium serves as a non-volatile or volatile data carrier for the model training program, transforming its embedded computer instructions into a sequence of operations that can be invoked by the machine. When the computer instructions are read and executed by one or more processors of an external electronic device, the underlying computing hardware is driven to automatically initiate the execution flow according to the code logic. This controls the data processing pipeline to complete the decomposition and feature mapping of structured evidence fields, controls the training framework to dynamically distribute and adjust the multi-stage sample mixing ratio, and controls the reinforcement learning engine to complete the joint calculation process of preference pair construction and divergence constraint penalty. Consequently, the physical device automatically executes all the steps in the multi-modal image forensics model training method based on multi-stage data reconstruction and preference reinforcement learning.

[0048] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, characterized in that, Includes the following steps: S1. Obtain image evidence data with authenticity labels and original natural language interpretations, reconstruct the original natural language interpretations into structured evidence data containing abnormal modules, abnormal types, abnormal targets, abnormal bounding boxes, and abnormal descriptions, generate thought annotations containing actions, observations, and reasoning logic for the structured evidence data, and construct a standardized evidence dataset. S2. Based on the structured evidence data in the standardized evidence dataset, extract features to construct a training sample set including label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and difficult negative samples. S3. Input the training sample set into the pre-trained visual language model, perform multi-stage supervised fine-tuning training with the proportion of label-perceived samples gradually decreasing and the proportion of label-free samples and difficult negative samples gradually increasing, and output candidate policy models. S4. For the same image input, call the candidate strategy model to generate multiple candidate answers, select candidate answers with similar overall form and fine-grained evidence defects to construct preference pairs, extract and calculate combined rewards by combining format, label consistency, evidence quality and evidence-interpretation consistency, perform reinforcement optimization on the candidate strategy model according to the preference pairs and combined rewards, and output a multimodal image evidence collection model.

2. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: Acquire image forensic data with authenticity labels and original natural language interpretations. Reconstruct the original natural language interpretations into structured evidence data containing anomalous modules, anomalous types, anomalous targets, anomalous bounding boxes, and anomalous descriptions. Generate thought annotations containing actions, observations, and reasoning logic for the structured evidence data. Construct a standardized forensic dataset, including: The anomaly analysis module identifies the original natural language interpretation and extracts the anomaly module, anomaly type, anomaly target, and anomaly description. Locate the region of the abnormal target in the image, generate and normalize it to obtain a normalized abnormal bounding box; Anomaly modules, anomaly types, anomaly targets, anomaly descriptions, and anomaly bounding boxes are combined to construct evidence items, and all evidence items are summarized to form an evidence array; a comprehensive interpretation is generated based on the evidence array, and conclusions are filled in according to the authenticity labels, and structured evidence data is generated and a quality check is performed; Extract the object, region, and attribute features to be examined from the structured evidence data to generate action annotations; extract the phenomenon features observed from the image evidence data to generate observation annotations; determine the conditions for observation features to constitute evidence and exclude defects in actual shooting; generate reasoning logic annotations; and merge action annotations, observation annotations, and reasoning logic annotations to generate thinking annotations.

3. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: Based on structured evidence data from standardized forensic datasets, features are extracted to construct a training sample set including label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and hard negative samples, including: The structured evidence data in the standardized evidence dataset is parsed to read images, authenticity labels, abnormal bounding boxes, and abnormal evidence fields. The label-aware sample is constructed by combining image evidence data, authenticity labels, and structured evidence data. Remove the authenticity labels from image forensics data to construct free-label samples; Specify the target analysis module from the structured evidence data, delete the evidence corresponding to other modules in the structured evidence data except for the target analysis module, and construct the module-focused sample; The abnormal bounding box is expanded according to a preset expansion ratio, and the image evidence data is cropped to generate a local image. The abnormal bounding box is then mapped to the local image coordinate system to construct a local region sample. An occlusion method is applied to the image region corresponding to the abnormal bounding box. It is determined whether the abnormal region is still visible. If it is visible, the evidence data is retained and the certainty of the abnormal description is reduced. If it is completely invisible, the corresponding evidence data is deleted, and an occlusion consistency sample is constructed. Extract normal designated module regions from forged images in image forensics data, or extract real image regions with motion blur, illumination distortion and resolution degradation to construct difficult negative samples; Perform sample quality checks and format standardization on each of the constructed samples, including checking field completeness, bounding box validity, and consistency between evidence and interpretation, and output a training sample set.

4. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: The training sample set is input into a pre-trained visual language model, and multi-stage supervised fine-tuning training is performed, gradually decreasing the proportion of label-aware samples and gradually increasing the proportion of label-free samples and hard negative samples. The output candidate policy model includes: In the first stage, the labeled perception samples in the training sample set are configured as the first proportion. The training sample set configured as the first proportion is then input into the pre-trained visual language model. The pre-trained visual language model is supervised to output formatted structured evidence data and thought annotations through the autoregressive cross-entropy loss function. In the second stage, the proportion of label-aware samples in the training sample set of the first stage is reduced, and the proportion of label-free samples, local region samples, occlusion consistency samples and difficult negative samples in the training sample set is increased and configured as the second ratio. The training sample set configured with the second ratio is input into the pre-trained visual language model trained in the first stage for supervised fine-tuning, and the candidate policy model is output.

5. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: For the same image input, a candidate policy model is invoked to generate multiple candidate responses. Preference pairs are constructed by selecting candidate responses that are similar in overall form and exhibit fine-grained evidentiary flaws, including: By changing the decoding parameters and sampling the same image input multiple times, the candidate strategy model is called to generate multiple sets of candidate responses. Construct a rule-based scorer that includes positive and negative scoring dimensions. Input candidate answers into the rule-based scorer and output the total score and defect type of the candidate answers. Positive scoring dimensions include the consistency between the conclusion and the truth label, the legality of the bounding box, and the specificity of the anomaly description. Negative scoring dimensions include the inconsistency between the conclusion and the truth label, the bounding box going out of bounds, false positives in the normal area, and the inconsistency between the comprehensive interpretation and the anomaly description. The candidate answer with the highest total score is selected as the preferred answer. From the remaining candidate answers, candidate answers with bounding box errors, vague evidence, introduction of new anomalies, module type errors, and false alarm defects for normal regions are selected as rejection answers. The preferred answer and the rejection answer are combined to construct a preference pair.

6. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: The combined reward is calculated by considering format, label consistency, evidence quality, and consistency between evidence and interpretation, including: Extract output labels from candidate answers and check field integrity; calculate the generated format reward. Extract the output conclusions from candidate answers, compare the output conclusions with the authenticity labels, and calculate the consistency reward for generating labels. Extract the legality of abnormal modules, the legality of abnormal types, the specificity of abnormal targets, and the legality of abnormal bounding boxes from candidate answers, and sum them up to generate an evidence quality reward; Extract overlapping features of anomalous targets in the comprehensive explanation and cited evidence array of candidate answers, check for new anomalies that do not exist in the cited evidence array, and calculate the consistency reward between generated evidence and explanation; The format reward, label consistency reward, evidence quality reward, and evidence-interpretation consistency reward are combined with semantic similarity reward and length duplication control reward according to preset weights to generate a combined reward.

7. The training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning according to claim 1, characterized in that: The candidate policy model is enhanced and optimized based on preference pairs and combined rewards, outputting a multimodal image forensics model, including: The current candidate policy model is invoked to sample multiple sets of responses for the image input in the preference pair, and the combined reward corresponding to each set of responses is calculated. Calculate the mean and standard deviation of the combined rewards within the same group of answers to generate relative advantages within the group; The candidate policy model output from the multi-stage supervised fine-tuning training stage is set as the reference model, and the divergence features between the current candidate policy model and the reference model are extracted to generate constraint terms. The parameters of the current candidate strategy model are updated based on the relative advantages within the group and the constraints, and a multimodal image forensics model is output.

8. A training system for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning, used to execute the training method for a multimodal image forensics model based on multi-stage data reconstruction and preference reinforcement learning as described in any one of claims 1-7, characterized in that, include: The evidence reconstruction module is used to acquire image evidence data with authenticity labels and original natural language interpretations, reconstruct the original natural language interpretations into structured evidence data containing anomaly modules, anomaly types, anomaly targets, anomaly bounding boxes, and anomaly descriptions, generate thought annotations containing actions, observations, and reasoning logic for the structured evidence data, and construct a standardized evidence dataset. The sample construction module is used to extract features from structured evidence data in standardized forensic datasets to construct a training sample set including label-aware samples, label-free samples, module-focused samples, local region samples, occlusion consistency samples, and hard negative samples. The supervised training module is used to input the training sample set into the pre-trained visual language model, perform multi-stage supervised fine-tuning training with the proportion of label-aware samples gradually decreasing and the proportion of label-free samples and hard negative samples gradually increasing, and output candidate policy models. The reinforcement learning module is used to generate multiple candidate responses by calling the candidate policy model for the same image input. It selects candidate responses with similar overall form and fine-grained evidence defects to construct preference pairs. It extracts and calculates combined rewards by combining format, label consistency, evidence quality, and evidence-interpretation consistency. Based on the preference pairs and combined rewards, it performs reinforcement optimization on the candidate policy model and outputs a multimodal image evidence collection model.

9. The multimodal image forensics model training system based on multi-stage data reconstruction and preference reinforcement learning according to claim 8, characterized in that, Also includes: An electronic device, comprising a processor and a memory, wherein a computer program is stored in the memory, and when the processor executes the computer program, it implements the steps of the multimodal image forensics model training method based on multi-stage data reconstruction and preference reinforcement learning as claimed in any one of claims 1-7.

10. A computer-readable storage medium storing computer instructions, characterized in that, When the computer instructions are executed by one or more processors, the one or more processors cause the processors to perform the steps of the method according to any one of claims 1-7.

Images