Method for training target model based on reinforcement learning

Abstract

According to the method for training the target model based on reinforcement learning, the target model is used for executing tampering detection on an input image, and the method comprises the steps that a training sample is obtained, the training sample comprises a target image and boundary information of a plurality of truth value boxes, and the truth value boxes are borders of a part of images marked in the target image and comprising tampering content; and inputting the target image into the target model, and outputting boundary information of the plurality of prediction frames. And calculating the overall overlap ratio of the plurality of prediction frames and the plurality of true value frames. Determining a first reward score based on a preset piecewise function according to a numerical range of the overall overlap ratio; the function corresponding to each numerical range of the piecewise function is a constant function, and the value of the corresponding constant function is increased along with the improvement of the overall overlap ratio. And based on a reinforcement learning algorithm, according to the first reward score, updating the target model.

Description

A method for training objective models based on reinforcement learning Technical Field

[0001] The embodiments in this specification relate to the field of machine learning technology, and in particular to a method for training a target model based on reinforcement learning. Background Technology

[0002] With the rapid development of generative artificial intelligence (AIGC) technology, the barriers to creating and altering image content have been significantly lowered, leading to a proliferation of forged and altered images (e.g., altered invoices, legal documents, announcements, etc.). These altered images, with their highly realistic visual effects, are difficult to distinguish with the naked eye, and their spread introduces potential risks in key areas such as evidence collection and risk control. Therefore, developing efficient and accurate image tampering detection technology to accurately identify, precisely locate, and reasonably attribute altered image content has become a pressing issue in the field of artificial intelligence.

[0003] As artificial intelligence evolves towards generalization and multimodality, the paradigm for intelligent processing of visual information has undergone a fundamental shift. It has evolved from single-unit models designed to solve specific tasks to comprehensive models integrating understanding, reasoning, and generation. Multimodal Large Language Models (MLLMs), by combining visual input with a large-scale trained language model, not only possess powerful visual feature extraction and text understanding capabilities but also leverage the logical reasoning and natural language generation advantages of large language models to output the analysis results of images in structured text form. The cross-modal reasoning capabilities of MLLMs enable them to be applied to complex tasks such as image tampering detection.

[0004] In training a large model (hereinafter referred to as the target model) to perform image tampering detection tasks, one of the core challenges is improving the accuracy of the target model in detecting bounding boxes containing tampered content. A bounding box refers to a closed geometric shape used to define a portion of the image containing tampered content (hereinafter referred to as the tampered region), typically defined by its boundary information. For example, for a rectangular bounding box, its boundary information may include the coordinates of the top-left and bottom-right corners, or it may include the coordinates of the top-left corner along with its length and width.

[0005] The accuracy of the target model in predicting bounding boxes directly determines the reliability of its output in image tampering detection tasks. To improve the bounding box localization accuracy of the target model, various training schemes have been proposed, but all have shortcomings.

[0006] One related technique employs the Cross-Entropy Loss function and supervised fine-tuning (SFT) training, treating bounding box boundary information as ordinary text words to predict the pattern of the next word, thus training the target model to generate bounding box boundary information. However, this training method only allows the target model to learn the probability distribution of coordinate numbers, but it cannot enable the target model to understand the geometric spatial meaning implied in the boundary information. This results in the trained target model having extremely low accuracy in detecting bounding boxes, failing to meet the needs of practical applications.

[0007] In another related technique, an attempt was made to use the difference between the directly computed predicted bounding box and the ground-truth bounding box as a reward within a reinforcement learning framework to train the target model. However, the difference between the bounding boxes is a continuously varying value within a finite range. In the early stages of training, the prediction results of the target model fluctuate greatly, resulting in a high variance in the reward. That is, even a small change in coordinates can cause the reward value to fluctuate drastically. This unstable reward signal can easily make the reinforcement learning training process difficult to converge or get stuck in a local optimum.

[0008] Another related technique attempts to use manual annotation to judge the difference between the predicted bounding box and the real bounding box, and use this as a reward signal to train the target model. However, although this training method can introduce human prior instructions, its annotation cost is extremely high. Moreover, for boundary information with pixel-level accuracy, manual identification is not only extremely costly, but also cannot guarantee the annotation quality, making it difficult to apply to large-scale training.

[0009] Therefore, it is hoped that a technical solution can be provided that closely integrates the bounding box detection characteristics in the image tampering detection task and designs a targeted reward function in order to effectively accelerate the convergence speed of the target model and improve the training effect. Summary of the Invention

[0010] This specification provides a method for training a target model based on reinforcement learning, wherein the target model is used to perform tamper detection on an input image, the method comprising:

[0011] Obtain training samples, which include the target image and boundary information of several ground truth boxes, wherein the ground truth boxes are the borders of the portion of the target image that includes the tampered content.

[0012] The target image is input into the target model, and the boundary information of several prediction boxes is output.

[0013] Calculate the overall overlap between the predicted boxes and the true boxes.

[0014] Based on the numerical range of the overall overlap, a first reward score is determined using a preset piecewise function; the function corresponding to each numerical range of the piecewise function is a constant function, and the value of the corresponding constant function increases as the overall overlap increases.

[0015] Based on the reinforcement learning algorithm, the target model is updated according to the first reward score.

[0016] A second aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method described in the first aspect.

[0017] According to the method provided in the embodiments of this specification, a preset piecewise function can be used to map the overall overlap between the predicted bounding box and the ground truth bounding box into a discrete reward score. Using this reward score as the optimization signal for training the target model, the non-differentiable bounding box geometric accuracy index can be converted into a reward signal suitable for reinforcement learning. This significantly reduces the high reward variance caused by fluctuations in the target model's predictions in the early stages of training, improves the convergence stability of reinforcement learning, and avoids training getting trapped in local optima. Simultaneously, setting a constantly increasing constant function in the preset piecewise function provides a clear and explicit parameter update target for the target model, enabling it to quickly focus on achieving a higher level of bounding box detection accuracy, rather than excessively focusing on minor numerical fluctuations, thereby significantly improving the training convergence speed of the target model. Applying the method provided in the embodiments of this specification, the target model can be trained efficiently and stably within the reinforcement learning framework, enabling the target model to accurately locate the tampered region and generate high-precision bounding boxes when performing image tampering detection tasks. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments in this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 is a flowchart of a method for training a target model based on reinforcement learning according to an embodiment of this specification;

[0020] Figure 2A is an exemplary training sample disclosed in the embodiments of this specification;

[0021] Figure 2B shows an exemplary output of a prediction box disclosed in an embodiment of this specification;

[0022] Figure 2C illustrates an exemplary scenario for calculating overall overlap disclosed in an embodiment of this specification.

[0023] Figure 3 shows an illustrative truth output text disclosed in an embodiment of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0025] In this specification, Large Language Model (LLM), also referred to simply as Large Model, is a natural language processing model based on deep learning technology. Its parameter count typically ranges from billions to hundreds of billions or even higher, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.). This architecture utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, giving them good generalization capabilities. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Prompts can be task descriptions or instructions in text form, used to stimulate the Large Language Model's semantic understanding and generation capabilities. In fine-tuning mode, large language models are further trained on small-scale datasets within a specific domain to optimize their performance on specific tasks. The powerful generalization capabilities and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0026] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, the large language model can also be called a multimodal large language model (MLLM). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, an MLLM can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0027] The above is a brief explanation of large language models and multimodal large language models. Next, we will first review the shortcomings of the techniques described above in training image tampering detection models, and then, with reference to the accompanying drawings, introduce the improved methods disclosed in the embodiments of this specification.

[0028] As mentioned earlier, in related technologies, using the cross-entropy loss function for supervised fine-tuning of the target model can only enable the target model to mimic the text probability distribution of coordinates in the bounding box information, but it cannot learn the mapping relationship between coordinates and geometric space, resulting in the localization accuracy of the predicted bounding box failing to meet actual needs. In the reinforcement learning framework, directly using continuous bounding box differences as reward signals will result in high-frequency jitter and high variance in the reward signal due to the instability of the target model's prediction in the early stages of training, making it difficult for the target model to converge effectively or get stuck in local optima. Technical solutions that rely on manually annotating the quality of bounding boxes suffer from high annotation costs and difficulty in ensuring objectivity and accuracy for pixel-level precision annotations.

[0029] In view of this, the inventors propose a method for training a target model based on reinforcement learning in the embodiments of this specification. The target model is used to perform tamper detection on the input image. Figure 1 shows a flowchart of the method. It is understood that this method can be executed by any device, equipment, platform, or device cluster with computing and processing capabilities. Referring to Figure 1, in one embodiment, the method includes at least the following steps: Step S101: Obtain training samples, which include the target image and boundary information of several ground truth boxes, wherein the ground truth boxes are the borders of the portion of the target image that includes the tampered content. Step S103: Input the target image into the target model and output the boundary information of several prediction boxes. Step S105: Calculate the overall overlap between the several prediction boxes and the several ground truth boxes. Step S107: Determine a first reward score based on a preset piecewise function according to the numerical range of the overall overlap. Step S109: Update the target model based on the first reward score using a reinforcement learning algorithm.

[0030] As mentioned earlier, multimodal large models possess the ability to understand, fuse, and generate cross-modal information, and can simultaneously process the visual features of images and the semantics of text. They are well-suited for image tampering detection tasks and are therefore frequently used to perform such tasks. In the various embodiments of this specification, different architectures of multimodal large models can be selected as the target model depending on the specific image tampering detection task. The embodiments of this specification do not limit the specific selection of the target model.

[0031] Step S101: Obtain training samples, which include the target image and boundary information of several ground truth boxes. The ground truth boxes are the borders of the portion of the target image that includes the tampered content.

[0032] Ground-truth bounding box: In various embodiments of this specification, the ground-truth bounding box refers to the accurate bounding box that marks the tampered region in the image on the training sample image, prepared in advance during the target model training phase. Its boundary information (e.g., the coordinates of the upper left and lower right corners of the rectangular ground-truth bounding box) provides a reference for the target model to learn the location of the tampered region and is the core basis for measuring the accuracy of the target model in locating the tampered region.

[0033] In the training of the target model, the concept corresponding to the ground truth bounding box is the predicted bounding box output by the target model. In the various embodiments of this specification, the predicted bounding box refers to the border generated by the target model for regions suspected of containing tampered content after performing tamper detection analysis on the input image. Generally speaking, the definition of the boundary information of the predicted bounding box is consistent with that of the ground truth bounding box.

[0034] Figure 2A shows an exemplary training sample containing a tampered target image and ground truth boxes (shown as dotted dashed rectangles in the figure) labeled for each portion of the image containing the tampered content. It can be understood that in some training samples, if the image is untampered, then the number of ground truth boxes in that training sample is zero.

[0035] In some practices, content tampering is not limited to altering the visual content of a target image (e.g., stitching images together, modifying image colors or outlines), but can also include altering the textual content contained within an image (e.g., forging or erasing text). For example, in invoice photographs, altering textual information such as subtotal amount, tax rate, and invoice date; in passport scans, altering textual information such as holder information, passport ID, and issuance date; and in financial documents, altering textual information such as transaction amount, payer information, and document number. This specification does not list all such examples.

[0036] Next, in step S103: the target image is input into the target model, and the boundary information of several prediction boxes is output.

[0037] Specifically, the target model receives the target image, performs tamper detection, and outputs a text sequence containing the boundary information (i.e., predicted boxes) predicted by the target model for regions that may contain tampered content. This boundary information is usually embedded in the text sequence generated by the target model in a specific format, for example, in the form of coordinate pairs.

[0038] Figure 2B shows an exemplary output of the predicted bounding boxes. For simplicity, the boundary information of the predicted bounding boxes output by the target model is presented in the form of a layered target image in the accompanying figure. Referring to Figure 2B, the target model, based on the parameters learned during its training phase, outputs several predicted bounding boxes (shown as dashed rectangles in the figure) after performing tampering detection analysis on the input target image. The portion of the image selected by any predicted bounding box represents the suspected tampering content determined by the target model.

[0039] After obtaining several predicted boxes output by the target model, in step S105, the overall overlap between the several predicted boxes and the several ground truth boxes is calculated.

[0040] In this step, calculating the overall overlap aims to quantify the degree of matching between the predicted bounding box and the ground truth bounding box, in order to reflect the overall accuracy of the target model in locating the tampered region.

[0041] Since the target image may contain multiple tampered regions (i.e. multiple ground truth boxes), and the target model may output multiple predicted boxes, the calculation of the overall overlap is a matching calculation between sets. It is necessary to use reasonable matching logic to match the predicted boxes with the corresponding ground truth boxes in order to avoid mismatches affecting the calculation results.

[0042] Figure 2C illustrates an exemplary scenario for calculating overall overlap. The dotted rectangles represent truth boxes (some truth boxes are obscured and not shown in the figure), and the dotted rectangles represent prediction boxes (some prediction boxes are obscured and not shown in the figure). The black fill indicates the overlapping area between the prediction box and the truth box after matching, which can be used to calculate the overall overlap. It can be seen that a prediction box can overlap with multiple truth boxes. In this case, a reasonable matching logic can help identify the matching relationship between the prediction box and the truth box. It is understood that in practice, the overlap relationship between prediction boxes and truth boxes can be one-to-one, one-to-many, many-to-one, many-to-many, or non-overlapping, etc. This specification does not provide examples of all of these relationships in its embodiments.

[0043] In a practical application, for any predicted bounding box, the overlap degree can be calculated between it and several ground truth bounding boxes corresponding to the target image provided in the training samples. The highest overlap degree is then taken as the overall overlap degree of the predicted bounding box. Then, based on the overlap degrees of the predicted bounding boxes output by the target model, the average value is calculated to obtain the overall overlap degree.

[0044] Specifically, the matching logic is designed based on the following principle: each predicted bounding box can correspond to at most one ground truth bounding box. Taking the maximum overlap ensures that each predicted bounding box is associated with its best-matching ground truth bounding box for bounding box overlap calculation, thus avoiding distortion in overlap calculation caused by one-to-many or many-to-many matching relationships. The average overlap of all predicted bounding boxes reflects the overall accuracy of the target model in locating several tampered regions, taking into account both the accuracy of individual predicted bounding box location and avoiding the impact of individual extreme values ​​on the objectivity of the overall overlap. Typically, the overlap can be determined by the Intersection over Union (IoU) ratio.

[0045] Based on the above calculation method, the several truth boxes in the training samples are set as a set. This means that the predicted bounding boxes output by the target model are set together. If we express this as an expression, then the overall overlap ratio mIoU (Mean Intersection over Union) can be represented as:

[0046]

[0047] in, This indicates the number of elements contained in the set.

[0048] It should be noted that when the target image is a manipulated image, and the target model does not detect any predicted bounding boxes (i.e., In this case, the mIoU value can be set to a constant value of 0.0 (i.e., a low-score constant). When the target image is an unaltered image and the target model does not detect any prediction boxes, the mIoU value can be set to a constant value of 1.0 (i.e., a high-score constant).

[0049] Next, in step S107, the first reward score is determined based on the numerical range of the overall overlap degree and a preset piecewise function.

[0050] Looking back at the techniques described above, if the continuous bounding box difference (e.g., overall overlap) is directly used as the reward signal for training the target model, the reward signal will exhibit high-frequency jitter and variance characteristics because the prediction results of the target model fluctuate drastically in the early stage of training. Small changes in difference will cause large fluctuations in the reward signal, which will seriously interfere with the update direction of the target model, making it difficult for the training to converge or get stuck in a local optimum.

[0051] Therefore, in this step, a piecewise function can be used to calculate the reward score based on the overall overlap. Specifically, a pre-defined standard for the numerical range of the piecewise function can be set. This standard can be flexibly set according to the accuracy requirements of the image tampering detection task in actual applications. For example, it can be divided into three intervals: low, medium, and high, based on the overall overlap value. Preferably, the function corresponding to each numerical range of the piecewise function is a constant function, and the value of the corresponding constant function increases as the overall overlap increases. That is, the higher the overall overlap, the higher the reward score, guiding the model to continuously improve the accuracy of locating the tampered area.

[0052] by This represents the reward score (i.e., the first reward score). In one example... It can be defined as:

[0053]

[0054] It should be understood that the various thresholds (e.g., 0.5, 0.8) and reward values / reward coefficients (e.g., 0.6, 0.4) shown in the various embodiments of this specification are for illustrative purposes only and are not intended to be limiting. In practical applications, the thresholds and reward values / reward coefficients can be dynamically adjusted according to the task accuracy requirements.

[0055] By matching the overall overlap calculated in step S105 with the preset range of piecewise function values, the corresponding reward score can be obtained. In this way, the continuous and volatile overlap values ​​can be converted into discrete reward scores. This stepped reward score can significantly reduce the variance of the overall overlap, providing a stable and clear optimization direction for updating the target model parameters, avoiding reward score fluctuations caused by small differences, and also improving the convergence speed of model training.

[0056] After determining the first reward score through the above steps, in step S109, the target model can be updated based on the first reward score using a reinforcement learning algorithm.

[0057] The Crossover Union (CUC) ratio, commonly used to evaluate bounding box dissimilarity, is mathematically non-differentiable. Specifically, the CUC calculation relies on the boundary information between the predicted and ground truth boxes. This boundary information contains discrete coordinate values. When the predicted box boundary aligns with the ground truth box boundary, its derivative is undefined, leading to a break in the gradient of the CUC function. Furthermore, when the relative positions of the predicted and ground truth boxes change slightly, such as from no overlap to overlap, the CUC value undergoes a step jump from zero to a positive number, rather than a continuous and smooth change. This discontinuity means that the derivative of the CUC function does not exist at critical points, failing to satisfy the continuity and smoothness conditions required for differentiability. In gradient descent-based model training, the optimization of the target model's parameters relies on backpropagation to calculate gradients. The non-differentiability of the CUC function means that its effective gradient relative to the target model parameters cannot be directly calculated, thus preventing the updating of the target model parameters through standard backpropagation.

[0058] Therefore, reinforcement learning algorithms can be used to train the target model based on the first reward score, thereby effectively avoiding the optimization obstacle caused by the non-differentiability of the intersection-union-ratio (IUU) function. Within the reinforcement learning framework, training the target model does not require the reward function itself to be differentiable; its core lies in adjusting the policy (i.e., the parameter distribution of the target model) by evaluating the reward value obtained from actions (i.e., the prediction boxes generated by the target model).

[0059] Specifically, in this step, the first reward score can be used as the reward signal for the action performed under the current target model policy, and its gradient with respect to the model parameters can be calculated using a policy gradient estimation method. The calculation of this gradient depends on the magnitude of the first reward score to adjust the probability distribution of the target model's actions, without needing to calculate the derivative of the reward function with respect to the target model parameters. Therefore, even if the first reward score originates from a non-differentiable intersection-union function, the reinforcement learning algorithm can still effectively update the target model parameters based on this score, gradually guiding the target model to generate prediction boxes with higher geometric accuracy.

[0060] In practical applications, when a target model performs image tampering detection, in addition to detecting predicted bounding boxes, it typically needs to provide a detection conclusion on whether the image contains tampered content and explain the clues upon which the tampering detection was based. Specifically, this task requires the target model not only to perform binary classification of the image (i.e., outputting the conclusion whether the image has been tampered with), but also to output boundary information indicating the tampered region and a textual explanation of the reasoning behind the judgment. In other words, this task essentially encompasses three sub-tasks: detection, localization, and interpretation. These sub-tasks are interconnected and together constitute a comprehensive evaluation of the image to be detected.

[0061] As mentioned earlier, in some practices, the images to be detected may also contain text information. Typically, these are images where text is the primary content and key information carrier, such as invoice photos, passport scans, and bank receipts. The text in these images usually carries important semantic information, possessing legal validity or commercial value; its authenticity and integrity are paramount. When detecting tampering in such images containing text, relying solely on visual features is often insufficient for accurate judgment. This is because many tampering techniques are extremely realistic at the pixel level, making them difficult to identify using the visual detection methods of the target model. Therefore, the target model needs to possess the ability to reason collaboratively using both visual and logical methods. On one hand, the target model needs to capture pixel-level anomalies through visual analysis, such as font differences, inconsistent boundaries, and color anomalies. On the other hand, the target model also needs to perform logical verification in conjunction with the text content, such as checking for arithmetic errors, time mismatches, weak semantic relevance, and so on.

[0062] In the training process of the target model, in order to achieve comprehensive learning and optimization of the image tampering detection task, the training process can introduce ground-truth output text corresponding to the target image. This ground-truth output text serves as the standard answer output by the target model, fully containing the truth information required to perform the image tampering detection task. It can include ground-truth explanations to illustrate the basis of tampering, ground-truth detection conclusions to indicate the state of image tampering, truth reasoning chains describing the reasoning process, and boundary information of the truth box defining the real tampered area, providing a reference standard for multi-dimensional optimization of the target model.

[0063] Figure 3 illustrates a schematic truth value output text. For visual clarity, the output text is presented in a more readable document format. However, it should be noted that in practical applications, for ease of data storage, exchange, and programmatic parsing, truth value output text is typically organized in a structured data format, such as JSON or XML. Furthermore, Figure 3 is only intended to illustrate the structure of the truth value output text; some content is not fully shown and is represented by ellipses. Referring to Figure 3, the truth value output text may contain the following:

[0064] The ground truth detection result refers to the determination of whether the target image has been tampered with, usually a binary classification label, such as Authentic or Forged. This is displayed as the "Detection Result" in the ground truth output text shown in Figure 3.

[0065] The ground truth explanation can be a structured text containing complete tampering detection analysis, used to explain the ground truth detection conclusion and / or the reason for the detection of the ground truth box. For example, it could be "the subtotal amount on the invoice has inconsistent fonts," or "there are obvious splicing marks at the edges of the passport photo area," etc. In optional embodiments, the ground truth explanation can be divided into abnormal visual information and abnormal logical information. The abnormal visual information is determined based on pixel analysis of the target image, and the abnormal logical information is determined based on semantic analysis of the target image. The abnormal visual information and abnormal logical information can be cross-validated to verify the tampered content existing in the tampered area. This corresponds to the "explanation reason" displayed in the ground truth output text shown in Figure 3.

[0066] Truth-based reasoning chain: This structure fully illustrates the step-by-step reasoning process from the input image to be detected to the generation of the final report, and consists of several thought texts. Referring to the "Thought Text" section in the truth output text shown in Figure 3, each thought text represents the analysis content of the large model at a specific reasoning stage in the image tampering detection task. Typically, the truth-based reasoning chain can be logically divided into multiple reasoning stages based on the focus of the reasoning. Each reasoning stage corresponds to the output of a thought text, and multiple thought texts corresponding to multiple reasoning stages can be sequentially combined to form the truth-based reasoning chain.

[0067] Abnormal visual information: determined based on the visual consistency among several pixels contained in the target image. For example, visual consistency can be: font consistency, detecting whether the text in different or the same text area in the target image is consistent in font type, font size, and stroke thickness; boundary consistency, detecting whether there are blurred edge transitions or splicing marks in the target image; color consistency, detecting whether adjacent areas or different areas with the same lighting conditions in the target image are consistent in color distribution, brightness, and saturation.

[0068] Exceptional logical information: determined based on the logical consistency among several textual pieces of information contained in the target image. For example, logical consistency can be: arithmetic consistency, verifying whether the numerical calculations involved in the textual information are correct (e.g., whether the unit price multiplied by the quantity in an invoice equals the subtotal amount, and whether the sum of the subtotals equals the total amount); temporal consistency, verifying whether the date and time information involved in the textual information conforms to logical order or common sense (e.g., whether the signature date of a document is later than the document's creation date); and semantic consistency, verifying whether the textual content in the textual information matches the scene of the target image (e.g., whether the nationality information on a passport scan matches the country of the issuing authority).

[0069] Cross-validation: Used to determine whether multiple anomalous visual and / or anomalous logical information points to the same tampered content in the target image. For example, if one anomalous visual information is "the font in the invoice amount area is inconsistent with other fonts," and another anomalous logical information is "the invoice amount does not match the calculated results of the subtotal and tax rate," cross-validation can determine that both clues point to the same amount area. If one anomalous visual information points to area A in the target image, but no other anomalous visual / logical information points to area A, cross-validation can clearly indicate that the anomalous visual information may be a false positive, and its reliability is low.

[0070] In a specific practice, while training the target model to output the accuracy of the predicted bounding box localization, the accuracy of the target model in judging the image tampering status can also be optimized simultaneously, so that the target model can correctly output the detection conclusion of whether the image has been tampered with.

[0071] In this practice, the training samples may further include ground truth detection conclusions corresponding to the target image, which are used to indicate whether the target image has been tampered with. During training, the target image is input into the target model, which can output boundary information of several prediction boxes and prediction detection conclusions (usually consistent with the form of ground truth detection conclusions, such as binary labels or text labels).

[0072] Next, a second reward score can be determined based on the predicted detection conclusion and the ground truth detection conclusion. If the predicted detection conclusion matches the ground truth detection conclusion, the second reward score can be set to full; if they do not match, the second reward score can be set to zero, thus forming clear positive and negative feedback to guide the target model to improve the accuracy of its judgment on the tampering state of the target image. To indicate the truth test conclusion, in The second reward score represents the predicted detection result output by the target model. It can be represented as:

[0073]

[0074] in, This is an indicator function, which takes the value 0 or 1.

[0075] After determining the second reward score, the target model can be updated based on the first and second reward scores using a reinforcement learning algorithm. Specifically, the first and second reward scores calculated in the preceding steps can be fused together as a total reward score, which is then used to update the target model. A typical fusion method is to perform a weighted summation of multiple reward scores to obtain the total reward score. Therefore, it can be represented as:

[0076]

[0077] in, and These are preset weighting coefficients used to balance the relative importance of their respective reward scores within the overall optimization objective. Training the target model in this way allows it to be guided within a reinforcement learning framework to simultaneously optimize multiple aspects of its predictive capabilities.

[0078] In a specific practice, while training the target model to output the accuracy of predicted bounding box localization, the model's ability to judge the number of tampered regions in an image can be optimized simultaneously. This ensures that the number of predicted bounding boxes output by the target model matches the actual number of tampered regions. This aims to avoid the target model generating too many fragmented predicted bounding boxes, while also preventing the generation of single predicted bounding boxes with excessively large coverage areas.

[0079] In this practice, a second reward score can be determined based on the number of predicted boxes and the number of ground truth boxes, used to evaluate the accuracy of the target model in generating the number of predicted boxes. (Using a set...) Representing the aforementioned truth boxes, in a set The several predicted boxes representing the output of the target model are, in this practice, the second reward score. It can be represented as:

[0080]

[0081] in, This is an indicator function, which takes the value 0 or 1.

[0082] After determining the second reward score, the target model can be updated based on the first and second reward scores using a reinforcement learning algorithm. The implementation method for this step has been described above and will not be repeated here. Therefore, in this practice, the total reward score... It can be represented as:

[0083]

[0084] In a specific practice, while training the target model to output the accuracy of the predicted bounding box, the semantic rationality and / or format standardization of the output text generated by the target model can also be optimized simultaneously, so that the target model can output output text that is not only accurate in positioning, but also has a rigorous thought chain, clear explanatory logic, and complete structure.

[0085] In this practice, the training samples may also include ground truth output text corresponding to the target image. This ground truth output text contains explanations and reasoning behind the annotations of the target image. The specific content contained in the ground truth output text has been described above and will not be repeated here. During training, inputting the target image into the target model can output boundary information for several prediction boxes and prediction output text (usually consistent with the text format of the ground truth output text, for example, JSON text generated based on the same JSON schema).

[0086] According to one implementation, a second reward score can be determined based on the semantic similarity between the ground truth output text and the predicted output text. This allows for the evaluation of the similarity between the predicted and ground truth output texts at both semantic and logical levels, rather than simply word matching. Preferably, a pre-trained text encoder (Embedding Model) can be used to map the predicted and ground truth output texts into high-dimensional vectors in a high-dimensional semantic space. Then, the cosine similarity between these two high-dimensional vectors is calculated as the second reward score. Cosine similarity focuses on the consistency of vector directions, which can better capture the deep semantic connections in text, thereby guiding the model to generate content that is semantically similar to the ground truth output text, rather than merely imitating words and phrases. The text to indicate the truth value is output. The predicted output text represents the output of the target model; the pre-trained text encoder uses... This indicates that, in this implementation, the second reward score... It can be represented as:

[0087]

[0088] After determining the second reward score, the target model can be updated based on the first and second reward scores using a reinforcement learning algorithm. The implementation method for this step has been described above and will not be repeated here.

[0089] According to one implementation, a second reward score can be determined based on the hit rate of several structural tags contained in the predicted output text on the target tag set. This aims to ensure that the predicted output text output by the target model contains necessary content sections, guaranteeing the readability and standardization of the predicted output text. The target tag set consists of several structural tags contained in the true value output text; the structural tags are used to indicate the text content type; the text content type includes one or more of the following: thought text, report text.

[0090] Thinking text: refers to the textual expression of the reasoning thought chain within the target model, usually embedded in the generated structural tags. <think>and< / think> Between. The thought-provoking text demonstrates the process by which the target model extracts clues from the input target image, performs verification and reasoning, and can improve the interpretability of the model's output text.

[0091] Report text: refers to the final analysis conclusions after integration and verification, usually embedded in the generated structure tags. <report> and< / report> between.

[0092] Based on the thought chain design introduced earlier, it can be understood that when the large model performs the image tampering detection task, it needs to sequentially execute several reasoning stages, including the reasoning stage for outputting thought text and the reasoning stage for outputting report text. These reasoning stages are orderly divided according to the analysis flow of the image tampering detection task, covering the entire process from image information extraction, clue verification, logical reasoning to report integration. Each reasoning stage corresponds to a specific detection (e.g., text / image recognition, anomaly analysis, tampering content location, report generation, etc.), and is a necessary link to ensure that the output text is rigorous, reliable, and logically complete. In practice, according to the design requirements of the specific task, all or part of these reasoning stages can be defined as necessary reasoning stages. The output content corresponding to the necessary reasoning stages (i.e., the thought text or report text embedded in the corresponding structural tags) is the key paragraph, which can be used as the benchmark for format completeness detection in subsequent steps, and the reward score can be calculated based on this to ensure that the number of key paragraphs in the predicted output text meets the design requirements.

[0093] In the specific calculation, the target tag set can be predefined. ,For example, Then, the proportion of structural labels contained in the predicted output text on the target label set is calculated as the second reward score. .by Indicates structural tags, This represents the predicted output text of the target model. In this implementation, the second reward score... It can be represented as:

[0094]

[0095] in, For indicator functions, when Existing in the predicted output text When the condition is met, the value is 1; otherwise, it is 0.

[0096] According to one implementation method, the inference quality of the target model can also be ensured by statistically analyzing the completeness of the thought chain. For example, a complete truth-based thought chain can contain multiple thought texts corresponding to multiple inference stages, with each thought text corresponding to a... <think>< / think> Structural label pairs. Therefore, by predicting the frequency of these structural label pairs in the output text, we can evaluate whether the thought chain generated by the target model is complete compared to the truth thought chain, and whether it contains a sufficient number of key paragraphs, i.e., performs a sufficient number of necessary inference stages. For example, if the training samples define three necessary inference stages for outputting the truth thought chain, and one necessary inference stage for outputting the report, then the target label set can be... , set as (The set shown here allows for duplicate elements; in practice, this can be avoided by adding unique identifiers to avoid the problem of duplicate element definitions in the array.) In specific calculations, structural tags in the predicted output text can be counted. <think> 、< / think> , <report> as well as< / report> The number of [tags], and the number of structural tags contained in the target tag set (i.e., [tags]). Calculate the ratio to obtain the second reward score. For example, N=4, At that time, the second reward score .

[0097] After determining the second reward score, the target model can be updated based on the first and second reward scores using a reinforcement learning algorithm. The implementation method for this step has been described above and will not be repeated here.

[0098] The above examples, through multiple embodiments, detail how to optimize the multi-dimensional performance of the target model in image tampering detection tasks by designing different reward scores. In short, reward scores... Related to the correctness of the format of the target model's output, this can be called a format bonus score; bonus score Related to the semantic correctness of the target model's output, this can be called the content reward score; reward score , and All of these are related to the accuracy of the predicted bounding boxes output by the target model, and can together constitute the reward score. This is called the border bonus score. Typically, the border bonus score can be a bonus score. , and The weighted sum is shown as:

[0099]

[0100] in, , and These are preset weighting coefficients.

[0101] In a preferred embodiment, the border reward score, format reward score, and content reward score can be fused to construct a total reward function that can comprehensively evaluate the model's performance. That is, the total reward score can be determined based on a weighted sum of the border reward score, the format reward score, and the content reward score. , shown as:

[0102]

[0103] in, , and These are preset weighting coefficients.

[0104] It is easy to understand that in this embodiment, it is necessary to parse the predicted output text output by the target model to extract the structural labels, prediction detection conclusions, bounding box information, and explanations contained therein, thereby calculating the total reward score. In a specific implementation, the predicted output text and prompt words can be input into a lightweight instruction model, instructing the model to parse the predicted output text according to a given syntax format (e.g., JSON Schema), outputting the structural labels, prediction detection conclusions, bounding box information, and explanations contained therein. This converts the predicted output text output by the target model into a structured text object that conforms to the syntax format definition. Simultaneously, using prompt words to instruct the instruction model to parse the predicted output text based on the syntax format, instead of using regular expression matching to extract text elements, can effectively handle occasional phantom symbols output by the target model (e.g., Markdown code block markers), improving the generalization of the predicted output text parsing. According to one implementation, if the instruction model encounters an error during the parsing of the predicted output text, such as the model API returning null or the output structured text object lacking necessary fields, the format reward score can be... Set to 0, or set the total reward score to 0. Set it to the lowest value (e.g., -1.0) to send a negative reinforcement feedback signal to the target model, prompting the target model to correct the format of the predicted output text during subsequent training.

[0105] After determining the total reward score, the parameters of the target model can be updated based on a reinforcement learning algorithm with the goal of maximizing the total reward score.

[0106] In a specific practice, Group Relative Policy Optimization (GRPO) can be used as a training framework for reinforcement learning. Based on the total reward score disclosed in the above embodiments, the target model is trained to ensure that the target model improves positioning accuracy, interpretation quality and format standardization while taking into account training stability and convergence efficiency.

[0107] In the above practice, to avoid bias in reward score evaluation caused by sampling a single output, for each training sample, the target model can generate G predicted output texts under the current policy, forming an output group. The group size G can be flexibly set according to the specific needs of computing resources and training efficiency; for example, it can be set to 4.

[0108] Next, the total reward score corresponding to each predicted output text in this output group is calculated. The average of all total reward scores within the output group is used as the baseline for evaluating the quality of the individual predicted output text.

[0109] Then, for each predicted output text within the output group, its advantage over the baseline in terms of total reward score can be calculated. This advantage quantifies the quality of the predicted output text relative to other predicted output texts in the same group; a positive value indicates it is better than the group average, while a negative value indicates it is worse than the group average. This effectively reduces the variance of the reward signal and enhances the stability of subsequent training. In one example, the advantage... It can be represented as:

[0110]

[0111] in, Indicates the first The total reward score corresponding to each predicted output text. This represents the average total reward score for the output group. This represents the standard deviation of the total reward score for the output group.

[0112] The optimization objective of the target model is to maximize the expected cumulative reward. The policy gradient method utilizes the calculated advantage value to update the parameters of the target model. Specifically, the update direction of the target model parameters is to increase the probability of generating high-advantage outputs while decreasing the probability of generating low-advantage outputs. In this process, Kullback-Leibler divergence can be introduced as a regularization term to control the magnitude of policy updates. During each round of policy parameter updates, the Kullback-Leibler divergence between the new policy and historical policies is calculated. If the calculated Kullback-Leibler divergence exceeds the Kullback constraint threshold, the step size of the policy update can be reduced through weight decay, forcibly constraining the difference between the new and historical policies within a reasonable range. This prevents policy mutations in the target model, ensures smooth policy iteration, avoids performance collapse due to over-optimization, and guarantees the stability and convergence of the training process.

[0113] In addition, in order to systematically verify the effectiveness of the reward function disclosed in the embodiments of this specification and to quantitatively evaluate the contribution of each component in the total reward score to the model performance, the inventors performed an ablation experiment on the target model trained using the total reward score disclosed in the above embodiments.

[0114] Table 1 below shows the performance of the target model in image tampering detection on the test dataset in an ablation experiment. The detection results are used to evaluate the binary classification accuracy of the target model in determining whether an image has been tampered with, measured using standard classification metrics, including accuracy and F1 score. Boundary detection is used to evaluate the accuracy of the target model in locating tampered regions, quantified by pixel-level mean Intersection over Union (mIoU) and mean F1 score (mF1). Output text is used to evaluate the semantic quality of the reports generated by the target model, quantified by calculating the cosine similarity score (CSS) and BERT Score F1 between the predicted and ground truth output texts.

[0115]

[0116] When removing format rewards ( ) or content rewards ( After training, the quality of the target model's output text decreased, but the detection performance of the other two items was less affected.

[0117] When the border bonus is removed ( After training, the performance of the target model in detection conclusions and bounding box detection significantly decreased, indicating that the reward component plays a dominant role in improving the accuracy of the target model in recognizing image tampering status and locating tampered content.

[0118] The target model trained using the complete reward function achieved high scores across all evaluation items, demonstrating that by weighting and summing the bounding box reward, format reward, and content reward, the target model can effectively achieve a balanced and excellent comprehensive capability across the three dimensions of detection, localization, and interpretation during training.

[0119] The preceding section detailed a method for training a target model based on reinforcement learning. This method utilizes a pre-defined piecewise function to map the overall overlap between the predicted bounding box and the ground truth bounding box into a discrete reward score. Using this reward score as the optimization signal for training the target model transforms the non-differentiable bounding box geometric accuracy index into a reward signal suitable for reinforcement learning. This significantly reduces the high reward variance caused by prediction fluctuations in the target model during the initial training phase, improving the convergence stability of reinforcement learning and preventing training from getting trapped in local optima. Furthermore, setting sequentially increasing constant functions within the pre-defined piecewise function provides a clear and explicit parameter update objective for the target model, enabling it to quickly focus on achieving a higher level of bounding box detection accuracy rather than excessively focusing on minor numerical fluctuations. This significantly improves the training convergence speed of the target model. Applying the method provided in the embodiments of this specification, the target model can be trained efficiently and stably within the reinforcement learning framework. This allows the target model to accurately locate the tampered region and generate high-precision bounding boxes when performing image tampering detection tasks.

[0120] In this instruction manual, the terms "first" in phrases such as "first reward score" and "second" or "third" (if any) are used merely for ease of distinction and description and do not have any limiting meaning.

[0121] The foregoing description describes specific embodiments of this specification; other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than those shown in the embodiments, and the desired result may still be achieved. Furthermore, the processes depicted in the drawings do not necessarily need to follow the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0122] This specification also provides a computing device in its embodiments, including a memory and a processor. The memory stores computer programs / instructions, and when the processor executes the computer programs / instructions, it implements the methods in the foregoing embodiments.

[0123] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0124] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0125] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0126] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0127] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0128] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in one or more blocks of the flowchart illustrations and / or one or more blocks of the block diagrams.

[0129] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowcharts and / or one or more block diagrams.

[0130] These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more block diagrams.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0135] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0136] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0137] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for training a target model based on reinforcement learning, wherein the target model is used to perform tamper detection on an input image, the method comprising: Acquire training samples, which include a target image and boundary information of several ground truth boxes, where the ground truth boxes are the borders of the portion of the target image that includes the tampered content; input the target image into the target model, and output the boundary information of several predicted boxes; calculate the overall overlap between the predicted boxes and the ground truth boxes; determine a first reward score based on a preset piecewise function according to the numerical range of the overall overlap; the function corresponding to each numerical range of the piecewise function is a constant function, and the value of the corresponding constant function increases as the overall overlap increases; update the target model based on the first reward score using a reinforcement learning algorithm.

2. The method according to claim 1, wherein, Calculating the overall overlap between the predicted bounding boxes and the ground truth boxes includes: for any predicted bounding box, calculating the overlap with each of the ground truth boxes, and taking the largest overlap as the bounding box overlap corresponding to the predicted bounding box; and calculating the average value based on the bounding box overlap corresponding to each of the predicted bounding boxes to obtain the overall overlap.

3. The method according to claim 2, wherein, The degree of overlap is determined by crossover ratio calculation.

4. The method according to claim 1, wherein, The step of updating the target model based on a reinforcement learning algorithm and a first reward score includes: determining a format reward score, which is related to the format correctness of the target model's output content; determining a content reward score, which is related to the semantic correctness of the target model's output content; determining a total reward score based on a weighted sum of the first reward score, the format reward score, and the content reward score; and updating the parameters of the target model based on a reinforcement learning algorithm with the goal of maximizing the total reward score.

5. The method according to claim 4, wherein, The format reward score is determined based on the number of key paragraphs included in the output content; the key paragraphs are the outputs of any necessary reasoning phases performed during the tamper detection process.

6. The method according to claim 1, wherein, The training samples also include ground truth detection conclusions corresponding to the target image, which are used to indicate whether the target image has been tampered with; The step of inputting the target image into the target model and outputting boundary information of several prediction boxes includes: inputting the target image into the target model and outputting boundary information of several prediction boxes and prediction detection conclusions; and the step of updating the target model based on a reinforcement learning algorithm and a first reward score includes: determining a second reward score based on the prediction detection conclusions and the ground truth detection conclusions; and updating the target model based on a reinforcement learning algorithm and the first reward score and the second reward score.

7. The method according to claim 1, wherein, The step of updating the target model based on the reinforcement learning algorithm and the first reward score includes: determining a second reward score based on the number of the plurality of prediction boxes and the number of the plurality of ground truth boxes; and updating the target model based on the reinforcement learning algorithm and the first reward score and the second reward score.

8. The method according to claim 1, wherein, The training samples also include ground truth output text corresponding to the target image, which includes explanations and thought processes for the annotation of the target image; the step of inputting the target image into the target model and outputting boundary information of several prediction boxes includes: inputting the target image into the target model and outputting boundary information of several prediction boxes and prediction output text.

9. The method according to claim 8, wherein, The step of updating the target model based on a reinforcement learning algorithm and a first reward score includes: determining a second reward score based on the semantic similarity between the ground truth output text and the predicted output text; and updating the target model based on the reinforcement learning algorithm and the first and second reward scores.

10. The method according to claim 8, wherein, The step of updating the target model based on the reinforcement learning algorithm and the first reward score includes: determining a second reward score based on the hit rate of several structural labels contained in the predicted output text on a target label set; the target label set consists of several structural labels contained in the true output text; the structural labels are used to indicate the text content type; the text content type includes one or more of the following: thought text, report text; and updating the target model based on the reinforcement learning algorithm and the first reward score and the second reward score.

11. A computing device comprising a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program, implements the method of any one of claims 1-10.

Images