Picture replication detection method and device, electronic equipment and storage medium

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for detecting image copying, belonging to the field of artificial intelligence technology. The method includes: inputting the image to be identified into a feature extraction model to obtain multi-dimensional image visual features; if the multi-dimensional image visual features meet preset copying judgment conditions, then inputting the multi-dimensional image visual features into an image detection model to obtain a copying detection result; the image detection model includes a rule encoding network, a cross-modal fusion network, and an output network; the rule encoding network encodes the copying image judgment rules to generate a rule condition vector; the cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features; and the output network determines the copying detection result based on the fused features. This invention achieves targeted capture of copying features by encoding the copying judgment rules into vectors and fusing them with visual features across modalities, thereby improving the accuracy of copying recognition and robustness against complex copying methods.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting image reproduction. Background Technology

[0002] Currently, compliance checks for after-sales installation mainly rely on workers taking photos on-site and then using image recognition to determine if the installation is compliant. However, some workers, to save time, may try to evade compliance checks by copying previous photos. This behavior could lead to non-compliant installations and create safety hazards. Therefore, identifying copied images has become a crucial step in ensuring installation compliance.

[0003] Existing technologies for recognizing copied images primarily rely on traditional convolutional neural networks or machine learning algorithms. However, these methods suffer from incomplete feature extraction and poor adaptability to countermeasures against copying when faced with high-quality screen copies or specially processed images, resulting in low accuracy in copy recognition. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for detecting image copying, which solves the defects in the prior art that result in incomplete feature extraction and poor adaptability to countermeasures against copying when faced with high-quality screen copying or specially processed images, thus leading to low accuracy in copying recognition.

[0005] This invention provides a method for detecting image reproduction, comprising the following steps:

[0006] The image to be identified is input into the feature extraction model to obtain the multidimensional image visual features output by the feature extraction model;

[0007] If the multidimensional image visual features meet the preset re-photographing judgment conditions, then the multidimensional image visual features are input into the image detection model to obtain the re-photographing detection result output by the image detection model; the preset re-photographing judgment conditions are that there are no texture features and / or pixel grid features in the multidimensional image visual features;

[0008] The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for determining whether an image is a copy, generating a rule condition vector. The cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features. The output network determines the copy detection result based on the fused features.

[0009] According to the image copy detection method provided by the present invention, the rule encoding network includes multiple rule extraction sub-networks, and each rule extraction sub-network extracts different rule sub-features;

[0010] The image detection model further includes a condition aggregation module, which concatenates and maps multiple rule sub-features to generate the rule condition vector.

[0011] The cross-modal fusion network uses the rule-based conditional vector to perform residual injection processing on the multidimensional image visual features to generate the fused features.

[0012] According to the image reproduction detection method provided by the present invention, the cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate the fused features, specifically including:

[0013] Generate a query vector based on the rule condition vector;

[0014] Generate key vectors and value vectors based on the multidimensional image visual features;

[0015] Calculate the similarity score between the query vector and the key vector, and determine the attention weight based on the similarity score;

[0016] The fused feature is obtained by weighting and summing the value vector based on the attention weights.

[0017] According to the image copy detection method provided by the present invention, the output network determines the copy detection result of the image to be identified based on the fused features, specifically including:

[0018] If the fusion features contain physical features and / or optical anomalies of the display device, then the image to be identified is determined to be a reproduced image;

[0019] If the fusion feature indicates a logical conflict between the semantic content of the image to be identified and the imaging medium, then the image to be identified is determined to be a reproduced image.

[0020] According to the image reproduction detection method provided by the present invention, the image detection model is trained based on the following steps:

[0021] Obtain a general visual language model as the base model;

[0022] Construct a sample dataset of reproduced images, wherein the sample dataset contains reproduced image samples with reproduced image judgment labels;

[0023] Based on the sample dataset, gradient path analysis is performed on the basic model to determine the target network layer in the basic model whose gradient response value to the re-photographing feature is higher than a preset threshold.

[0024] Keep the pre-trained parameters of the base model frozen, and introduce trainable adapter parameters into the target network layer;

[0025] The adapter parameters are trained and updated using the sample dataset, and the base model containing the trained adapter parameters is determined as the image detection model.

[0026] According to the present invention, a method for detecting image copying, wherein gradient path analysis is performed on the base model based on the sample dataset to determine the target network layer in the base model whose gradient response value to copying features is higher than a preset threshold, includes:

[0027] The sample of the re-photographed image in the sample dataset is input into the base model for forward propagation to obtain the prediction result, and the loss value between the prediction result and the re-photographing judgment label is calculated.

[0028] Backpropagation is performed on the loss value to obtain the gradient matrix corresponding to the weight parameters of each network layer in the base model;

[0029] The norm of the gradient matrix of each network layer is calculated, and the calculation result is determined as the gradient response value of the network layer.

[0030] The network layer whose gradient response value is greater than a preset response threshold is determined as the target network layer.

[0031] According to the image reproduction detection method provided by the present invention, the reproduced image sample is obtained based on the following steps:

[0032] Collect the detection feedback state information of the image detection model in the historical training rounds, and input the detection feedback state information into the policy network;

[0033] Obtain the distortion generation parameters output by the policy network based on the detection feedback state information;

[0034] The original compliant image is reconstructed using a sample generation network based on the distortion generation parameters to obtain the re-photographed image sample.

[0035] According to the image reproduction detection method provided by the present invention, after obtaining the distortion generation parameters output by the policy network based on the detection feedback state information, the method further includes:

[0036] Physical environment parameters are obtained from a physical parameter library, including at least screen reflectivity, screen color gamut parameters, pixel arrangement structure parameters, and ambient light color temperature parameters.

[0037] The physical environment parameters are input into the physical constraint generation network to obtain the style modulation vector generated by the physical constraint generation network.

[0038] The original compliant image is reconstructed using the sample generation network based on the style modulation vector and the distortion generation parameters to obtain the re-photographed image sample.

[0039] The present invention also provides an image reproduction detection device, comprising the following modules:

[0040] The feature extraction unit is used to input the image to be identified into the feature extraction model and obtain the multi-dimensional image visual features output by the feature extraction model.

[0041] A re-enactment detection unit is used to input the multi-dimensional image visual features into an image detection model and obtain the re-enactment detection result output by the image detection model if the multi-dimensional image visual features meet a preset re-enactment judgment condition; the preset re-enactment judgment condition is that there are no texture features and / or pixel grid features in the multi-dimensional image visual features.

[0042] The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for determining whether an image is a copy, generating a rule condition vector. The cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features. The output network determines the copy detection result based on the fused features.

[0043] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image reproduction detection method described above.

[0044] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image reproduction detection method as described above.

[0045] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image reproduction detection method described above.

[0046] The image copy detection method, device, electronic device, and storage medium provided by this invention achieve targeted capture of copy features by encoding copy judgment rules into vectors and fusing them with visual features across modalities, thereby improving the accuracy of copy recognition and robustness against complex copy methods. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0048] Figure 1 This is one of the flowcharts of the image reproduction detection method provided by the present invention.

[0049] Figure 2 This is one of the flowcharts illustrating the training image detection model provided by the present invention.

[0050] Figure 3 This is the second schematic diagram of the training image detection model provided by the present invention.

[0051] Figure 4 This is one of the flowcharts for generating reproduced image samples provided by the present invention.

[0052] Figure 5 This is the second schematic diagram of the process for generating reproduced image samples provided by the present invention.

[0053] Figure 6 This is the second flowchart of the image reproduction detection method provided by the present invention.

[0054] Figure 7 This is the third flowchart of the image reproduction detection method provided by the present invention.

[0055] Figure 8 This is a schematic diagram of the image reproduction detection device provided by the present invention.

[0056] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0057] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0058] It should be noted that, in the description of this invention, the terms "comprising," "including," or any other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Those skilled in the art will understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0059] All actions involving the acquisition of signal information or data in this invention are carried out in compliance with the relevant data protection laws and policies of the country where the device is located, and with the authorization granted by the owner of the device.

[0060] In after-sales installation compliance testing, compliance checks primarily rely on workers taking photos on-site and then using recognition models to determine if the installation is compliant. However, in practice, some workers, to save time, may simply copy previous photos to evade compliance checks. This behavior can lead to non-compliant installations and create safety hazards. Therefore, recognizing copied images has become a crucial step in ensuring installation compliance.

[0061] Existing image recognition technologies suffer from the following shortcomings in practical applications: First, feature extraction is incomplete. Current technologies rely heavily on single texture, frequency domain, or reflection features, making them susceptible to interference from environmental factors (such as lighting and screen material), leading to unstable features and affecting recognition accuracy. Second, they are poorly adaptable to countermeasures against image copying. Existing technologies lack the ability to adapt to complex scenarios where workers attempt to evade detection, such as rotating phones, adjusting screen brightness, or adding obstructions, making them easily bypassed by simple interference methods. Third, they are insufficient in recognizing phone screen borders. When copied images contain phone screen borders, existing technologies lack sufficient adaptability to the differences in border shape, color, and size across different phone models, easily misclassifying non-copied images as copied images or missing genuine copied images. Finally, they are inadequate in recognizing extremely blurry images, especially when the shooting distance is far or the shooting equipment is of poor quality, resulting in very blurry copied images. Existing technologies struggle to extract effective features, easily leading to misjudgments.

[0062] To effectively identify falsification by workers taking photos during after-sales installation and to ensure the accuracy and reliability of installation compliance testing, this invention provides a method, device, electronic device, and storage medium for detecting photo falsification, thereby addressing the limitations of existing technologies in identifying photo falsification scenarios such as moiré patterns, mobile phone screen borders, high reflectivity, and super blurriness.

[0063] The following is combined with Figures 1-9This invention describes the image reproduction detection method, apparatus, electronic device, and storage medium provided by the present invention.

[0064] Figure 1 This is one of the flowcharts illustrating the image reproduction detection method provided by this invention. For example... Figure 1 As shown, the execution subject of the image reproduction detection method provided by the present invention can be a server, a cloud computing platform, or a computer capable of executing the method of the present invention, etc. Unless otherwise specified, the following embodiments will be described using a server as an example.

[0065] As an optional embodiment, this image reproduction detection method mainly includes, but is not limited to, the following steps:

[0066] Step 110: Input the image to be identified into the feature extraction model to obtain the multi-dimensional image visual features output by the feature extraction model.

[0067] The image to be identified refers to the on-site credential image collected and uploaded to the system by the operators (such as installation workers) during the after-sales installation compliance inspection process. For example, the image to be identified could be a color RGB image recording the installation location and pipe connection status of home appliances (such as air conditioners and water heaters). In a real-world scenario, the image to be identified could be an original photograph taken directly of the object, or it could be a secondary photograph (re-photographed) of an image displayed on an electronic display device (such as a mobile phone screen or tablet) or on a paper medium.

[0068] Feature extraction models refer to deep neural network models used to extract high-level semantic representations and low-level detailed features from raw image data. For example, a feature extraction model can be a vision transformer (ViT) based on the Transformer architecture or a visual encoding component in a large-scale pre-trained multimodal model, which can process high-resolution input images and capture fine-grained visual features.

[0069] Multidimensional image visual features refer to feature vectors or feature tensors that contain multi-level information about an image, output after calculation by a feature extraction model. For example, such multidimensional image visual features can be represented as a feature matrix with dimensions [N, D], where N represents the number of patches into which the image is divided, and D represents the dimension of the feature channels. These multidimensional image visual features not only contain the global semantic information of the image, but also retain low-level visual information that may characterize traces of copying, such as texture distribution (e.g., the presence of moiré patterns), pixel arrangement structure, illumination reflection patterns, and edge geometry.

[0070] Step 120: If the multidimensional image visual features meet the preset re-enactment judgment conditions, then input the multidimensional image visual features into the image detection model and obtain the re-enactment detection result output by the image detection model; the preset re-enactment judgment conditions are that there are no texture features and / or pixel grid features in the multidimensional image visual features.

[0071] Considering that in typical low-quality reproduction scenarios, images are usually accompanied by visible or easily extractable moiré patterns, pixel patterns, and other obvious features, when faced with high-definition screen reproduction (such as Retina display) or specially processed reproduction images, these obvious features are often extremely weak or even disappear, causing traditional methods to fail.

[0072] Therefore, this invention employs hierarchical detection, first determining whether the multidimensional image visual features meet the condition of "no texture features and / or pixel grid features," that is, filtering out those samples that are difficult to identify. Only for images lacking these features, subsequent complex image detection models are used for in-depth analysis. This allows for the concentrated use of computing resources to overcome the challenging task of re-enactment recognition, while simultaneously uncovering hidden re-enactment clues through a targeted rule fusion mechanism, effectively improving the recognition accuracy of high-quality re-enactment images.

[0073] Texture features and pixel grid features refer to the characteristics of whether there are periodic interference patterns (i.e., moiré patterns) in an image caused by digital sampling aliasing, and micro-matrix features that characterize the physical pixel arrangement structure of a display device screen. For example, texture features can be wavy stripes that appear as a specific frequency distribution in image frequency domain analysis, while pixel grid features can be the RGB sub-pixel arrangement structure of a liquid crystal display (LCD) or organic light-emitting diode (OLED) screen as shown after local magnification of the image.

[0074] The image detection model refers to the deep neural network model constructed in this embodiment, which integrates rule priors and visual perception. Specifically, it includes sub-modules such as a rule encoding network, a cross-modal fusion network, and an output network. For example, this image detection model can be a dedicated detection model obtained by structurally fine-tuning a pre-trained multimodal large model (such as QwenVL-2.5-7B).

[0075] The re-photographing detection result refers to the model's final judgment on the authenticity of the current input image. For example, the re-photographing detection result can be a binary label (such as "re-photographed image" or "original image"), or it can be a confidence probability value representing the possibility of re-photographing. When the confidence probability value is higher than a preset threshold, it is determined that the current input image is a re-photographed image taken by a worker to avoid inspection.

[0076] The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for judging re-photographed images and generates rule condition vectors. The cross-modal fusion network fuses the rule condition vectors and multi-dimensional image visual features to generate fused features. The output network determines the re-photographing detection result based on the fused features.

[0077] A rule encoding network refers to a neural network module used to transform business logic rules or expert prior knowledge into feature vectors that can be understood by computers. For example, in this embodiment of the invention, the rule encoding network can be a text encoder or an embedding layer, whose input is a predefined description of re-photographing judgment rules (such as natural language descriptions or corresponding category indices for the presence of moiré patterns, reflections, trapezoidal distortion, device borders, abnormal highlights, abnormal blur, and ghosting), which is then encoded into a high-dimensional feature representation.

[0078] The rule condition vector refers to the vector sequence output by the rule encoding network that contains semantic information about the re-photographing features. This vector sequence essentially constitutes the query signal in the subsequent feature fusion process, which guides the image detection model to focus on which specific visual patterns in complex image features (e.g., focusing on frequency features similar to moiré patterns, or geometric features similar to screen borders).

[0079] A cross-modal fusion network refers to a neural network structure capable of processing data from different modalities (such as text rule modality and image visual modality) and achieving feature interaction. For example, in this embodiment of the invention, the cross-modal fusion network employs a cross-attention mechanism, which is embedded in the decoder layer of a multimodal large model or a specific fusion layer to establish a mapping relationship between rule conditions and visual features.

[0080] The fusion of rule-based condition vectors and multi-dimensional image visual features refers to the process of using an attention mechanism to calculate the correlation between rules and images, and then weighting and aggregating image features accordingly. Specifically, the rule-based condition vector is used as the query vector in the attention mechanism, and the multi-dimensional image visual features are used as the key and value vectors. By calculating the similarity matrix between the query vector and the key vector, an attention weight map is generated. This weight map is then used to weight and sum the value vectors, allowing the image detection model to automatically focus on regions in the image that are highly correlated with the re-photographing rules.

[0081] Fusion features refer to image feature representations generated after fusion processing, incorporating prior knowledge of rules. These fusion features are no longer general visual descriptions, but rather enhance feature responses related to copying traces (such as enhancing feature values ​​in areas with weak moiré patterns) while suppressing interference from background noise (such as irrelevant furniture backgrounds).

[0082] The output network refers to the fully connected layer or classifier used to output the final prediction result based on the fused features. For example, the output network can be a classification head consisting of a multilayer perceptron (MLP) and activation functions (such as sigmoid or softmax). The output network receives the fused features as input and transforms the high-dimensional features into probability values ​​representing whether an image is a copy through linear mapping.

[0083] The image copy detection method provided by this invention achieves targeted capture of copy features by encoding copy judgment rules into vectors and fusing them with visual features across modalities, thereby improving the accuracy of copy recognition and robustness against complex copy methods.

[0084] In another embodiment of the present invention, the rule encoding network includes multiple rule extraction sub-networks, each of which extracts different rule sub-features; the image detection model also includes a condition aggregation module, which concatenates and maps multiple rule sub-features to generate a rule condition vector; the cross-modal fusion network uses the rule condition vector to perform residual injection processing on multi-dimensional image visual features to generate the fused features.

[0085] The rule extraction sub-network refers to an independent neural network branch or encoding module used to process different categories of re-photographing judgment rules. For example, in the embodiments of the present invention, multiple independent feature encoders can be set up in parallel for multiple re-photographing judgment rules. Each encoder focuses on mapping a specific type of text description or rule index to the corresponding vector representation, or performing feature pre-extraction for specific image attributes (such as a sub-network specifically for recognizing moiré patterns or a sub-network specifically for recognizing borders).

[0086] Each rule extraction sub-network extracts different rule sub-features, meaning that each sub-network works in parallel and outputs semantic vectors or feature maps representing different re-production attributes. For example, the first sub-network outputs a moiré feature vector representing moiré texture attributes, the second sub-network outputs a highlight feature vector representing screen reflection attributes, the third sub-network outputs a geometric feature vector representing trapezoidal distortion attributes, and so on, thereby achieving a multi-dimensional decoupled description of re-production features.

[0087] The concatenation and mapping of multiple rule sub-features refers to the process of chaining multiple independent rule sub-features together along the feature channel dimension and adjusting their dimensions through linear transformation to adapt to the input requirements of the fusion network. For example, the conditional aggregation module concatenates the feature vectors output by all sub-networks in sequence into a long vector, and then projects this concatenated long vector onto a dimensional space that matches the visual features of the multi-dimensional image through a fully connected layer or a linear projection layer, thereby obtaining a global rule conditional vector that can comprehensively represent all re-photographing rules.

[0088] Residual injection refers to the operation of preserving information from the original input features during feature fusion and superimposing incremental features with fused rule information onto the original features. For example, the specific computation process could involve first using a cross-attention mechanism to calculate the interaction between the multidimensional image visual features and the rule-guided vector, obtaining intermediate features containing rule directionality. Then, these intermediate features are directly added element-wise to the original multidimensional image visual features, forming a fused feature that contains both original visual information and superimposed rule-guided information. This structure helps solve the gradient vanishing problem in deep network training and ensures that the detailed features of the original image are not forgotten.

[0089] The image reproduction detection method provided by this invention employs multiple rule extraction sub-networks to independently extract rule features and perform aggregation mapping, combined with residual injection for feature fusion. This enables decoupled representation of various reproduction features, avoiding mutual interference between different rule features. At the same time, it effectively preserves the low-level visual details of the original image during the fusion process, preventing feature loss during deep network transmission. This further enhances the image detection model's ability to analyze complex mixed reproduction scenarios and improves its training stability.

[0090] In another embodiment of the present invention, the cross-modal fusion network performs fusion processing on the rule condition vector and multi-dimensional image visual features to generate fused features, specifically including: generating a query vector based on the rule condition vector; generating a key vector and a value vector based on the multi-dimensional image visual features; calculating the similarity score between the query vector and the key vector, and determining the attention weight based on the similarity score; and performing a weighted summation of the value vector based on the attention weight to obtain the fused features.

[0091] Specifically, this invention establishes a semantic association between the re-production judgment rule and the local region of the image through an attention mechanism. That is, it uses the rule vector as a guiding signal to search for and focus on the visual region in the image feature map that highly matches the re-production judgment rule.

[0092] For example, firstly, a linear transformation layer maps the rule condition vector to a query vector, and the multidimensional image visual features to a key vector and a value vector. Then, a dot product operation is used to calculate the similarity score between the query vector and the key vector. This similarity score reflects whether the features at each location in the image conform to the description of the re-enactment rules (e.g., whether a certain area has moiré patterns). Next, the Softmax normalization function is used to convert these similarity scores into a probabilistic form of attention weight distribution. Finally, the value vector is weighted and aggregated according to the attention weight, so that the final fused feature can enhance the feature expression of image areas that conform to the re-enactment rules, while suppressing irrelevant background information.

[0093] The image re-photographing detection method provided by this invention generates query vectors based on rule vectors and uses an attention mechanism to calculate the similarity between the query vector and the visual features of the image to determine the weight. This can establish a precise mapping relationship between rule semantics and local regions of the image, enabling the image detection model to automatically focus on key visual regions that are highly related to the re-photographing judgment rules during the feature fusion stage. This effectively suppresses the interference of irrelevant background noise and improves the directionality and signal-to-noise ratio of feature extraction.

[0094] In another embodiment of the present invention, the output network determines the re-photograph detection result of the image to be identified based on the fusion features, specifically including: if the fusion features contain physical features and / or optical anomaly features of the display device, then the image to be identified is determined to be a re-photographed image; if the fusion features indicate that there is a logical conflict between the semantic content of the image to be identified and the imaging medium, then the image to be identified is determined to be a re-photographed image.

[0095] Considering that single-dimensional feature judgment is prone to false detection, such as a real glass reflective object being misjudged as a screen reflection, or a real grid-like object being misjudged as a pixel array, this invention adopts a multi-level rule verification chain analysis mechanism to perform comprehensive verification from microscopic traces at the physical level to macroscopic logic at the semantic level. This enables the construction of a rigorous verification mechanism that effectively distinguishes similar interference features in natural scenes from real copying traces, and significantly reduces the false alarm rate.

[0096] The physical and optical anomalies of a display device refer to the hard physical traces and optical correction deviations introduced during the secondary imaging process in the reproduction process, respectively. For example, physical features include, but are not limited to, screen bezels (such as the black bezels of mobile phones and monitors), sub-pixel arrangement structures, and fingerprints or dust particles on the screen surface; optical anomalies include, but are not limited to, trapezoidal distortion caused by tilted shooting angles, specular reflection highlights caused by ambient light illuminating the screen, or local blurring and ghosting phenomena caused by inaccurate focusing.

[0097] A logical conflict exists between the semantic content of the image to be identified and the imaging medium. This refers to a situation where the scene content shown in the image does not logically match the form of the medium in which it is presented. For example, the content of the image to be identified may show an outdoor natural landscape or a large appliance installation site, but the edges of the content appear as curled photographic paper, or the background of the content may contain indoor desktop textures, keyboards, mice, and other office environment objects. This inconsistency between the content (scene) and the medium environment (photograph of an office desktop) constitutes a logical conflict and is the semantic basis for determining whether it is a photocopy.

[0098] The image copying detection method provided by this invention constructs a multi-level verification mechanism from microscopic traces to macroscopic semantics by comprehensively detecting the physical and optical characteristics of the display device and the logical conflicts between content and medium. It can effectively distinguish similar interference in natural scenes (such as real reflective objects) from real copying behavior, solve the problem of easy misjudgment of single features, and thus improve the reliability of detection results.

[0099] Figure 2 This is one of the flowcharts illustrating the training image detection model provided by the present invention, such as... Figure 2 As shown, as another optional embodiment provided by the present invention, the image detection model is trained based on the following steps:

[0100] Step 210: Obtain a general visual language model as the base model.

[0101] A general visual language model refers to a multimodal deep learning model that has been pre-trained on a large-scale public dataset and possesses general image understanding and text generation capabilities. For example, in this embodiment of the invention, the base model can be the QwenVL-2.5-7B model, which has a massive number of parameters and has mastered general object recognition, scene description, and image-text alignment capabilities.

[0102] Step 220: Construct a sample dataset of reproduced images. The sample dataset contains reproduced image samples with reproduced image judgment labels.

[0103] A re-enactment label refers to the annotation information used to indicate the true category of an image. For example, the re-enactment label can be a binary classification label of 0 (representing the original image) and 1 (representing a re-enactment image), or it can be a fine-grained attribute label that includes the specific re-enactment type (such as screen re-enactment or photo re-enactment).

[0104] The sample images refer to image data specifically collected or generated for model training. For example, this sample set not only includes real business scenario images, but also synthetic images generated through data augmentation techniques, featuring different degrees of distortion, screen textures, and reflection conditions, to ensure the diversity of data distribution.

[0105] Step 230: Perform gradient path analysis on the basic model based on the sample dataset to determine the target network layer in the basic model whose gradient response value to the re-photographing feature is higher than a preset threshold.

[0106] Gradient path analysis refers to monitoring the numerical distribution of the gradients of the loss function with respect to the parameters of each layer of the model during backpropagation, in order to assess the sensitivity of each layer's parameters to a specific task. For example, when inputting a batch of reproduced samples, the loss is calculated and the gradients are backpropagated to calculate the gradient norm or gradient variance of the weights of each layer.

[0107] Identifying target network layers in the base model whose gradient response values ​​to re-enactment features exceed a preset threshold refers to selecting those network layer locations that are most sensitive to changes in re-enactment features. For example, if the gradient response values ​​of shallow convolutional layers in the visual encoder or projection layers in the attention mechanism are found to be significantly higher than other layers, it indicates that these layers are mainly responsible for extracting key re-enactment features such as texture and edges, and therefore they are marked as target network layers that require focused adjustment.

[0108] Step 240: Keep the pre-trained parameters of the base model frozen and introduce trainable adapter parameters into the target network layer.

[0109] Specifically, to save training resources and prevent the forgetting of general knowledge, this invention employs a parameter-efficient fine-tuning strategy. For example, this invention can lock the update permissions of all original weights of the base model, and only insert low-rank adapter (LoRA) modules in parallel next to the selected target network layers, or insert bottleneck structure adapter layers in sequence. The number of these newly added parameters is much smaller than the number of parameters in the original model.

[0110] Step 250: Use the sample dataset to train and update the adapter parameters, and determine the base model containing the trained adapter parameters as the image detection model.

[0111] Specifically, this invention minimizes the loss function between the predicted result and the true label through supervised learning. For example, during training, the stochastic gradient descent algorithm is used to update only the newly introduced adapter parameters, enabling the model to gradually master the ability to identify re-photographing features such as weak moiré patterns, distortion, and borders. The final model retains the generalization ability of the basic model while possessing high-precision discrimination capability for re-photographing detection tasks.

[0112] The image copy detection method provided by this invention locates sensitive layers by performing gradient path analysis on the basic model and introduces an adapter for targeted fine-tuning. Under the premise of freezing most of the pre-trained parameters, it can update the parameters only for the network layers that are sensitive to copy features. This significantly reduces the training computing power cost and storage overhead while effectively avoiding catastrophic forgetting, and achieves efficient adaptation and performance improvement of the image detection model for the specific task of copy detection.

[0113] Figure 3 This is the second schematic diagram of the training image detection model provided by the present invention, as shown below. Figure 3 As shown, as another optional embodiment provided by the present invention, gradient path analysis is performed on the base model based on the sample dataset to determine the target network layer in the base model whose gradient response value to the re-photographing feature is higher than a preset threshold, including but not limited to the following steps:

[0114] Step 310: Input the reproduced image samples in the sample dataset into the basic model for forward propagation to obtain the prediction results, and calculate the loss value between the prediction results and the reproduced judgment label.

[0115] For example, a batch of images containing different types of remakes can be input into the base model to obtain the class probability distribution output by the base model. Then, the cross-entropy loss function can be used to calculate the difference between the predicted probability distribution and the true label. This difference value is the loss value that guides the subsequent parameter adjustment.

[0116] Step 320: Backpropagate the loss value to obtain the gradient matrix corresponding to the weight parameters of each network layer in the base model.

[0117] Specifically, this invention utilizes the chain rule to calculate the partial derivatives of the loss function with respect to the parameters of each network layer. For example, using an automatic differentiation engine (such as PyTorch's Autograd), the calculation proceeds layer by layer from the output layer, obtaining the gradient value corresponding to each element in the weight matrix of each convolutional layer, attention layer, or fully connected layer. These gradient values ​​constitute a gradient matrix with the same shape as the original weight matrix, reflecting the direction and extent of the influence of parameter changes in that layer on the final loss value.

[0118] Step 330: Calculate the norm of the gradient matrix of each network layer, and determine the calculated result as the gradient response value of the network layer.

[0119] Specifically, this invention compresses the complex gradient matrix into a scalar index that can represent the overall sensitivity of the layer using mathematical norms. For example, for the gradient matrix of the i-th layer, its L2 norm value is calculated. The larger the norm value, the stronger the activation response generated by the network layer when processing re-production features, that is, the more important the layer plays in the process of re-production feature extraction. Therefore, it is defined as the gradient response value of the layer.

[0120] Step 340: The network layer whose gradient response value is greater than the preset response threshold is determined as the target network layer.

[0121] Specifically, a threshold is set to filter out key parameter layers. For example, by statistically analyzing the distribution of gradient response values ​​across all layers, a preset response threshold can be set to 1.5 times the average gradient response value. Alternatively, the top 20% of network layers in terms of response value can be selected, and these layers that meet the criteria (usually shallow texture extraction layers or deep semantic alignment layers of the visual encoder) can be marked as target network layers, serving as the designated locations for subsequent insertion of adapter parameters.

[0122] As an optional embodiment, this invention employs an intra-block sparse low-rank update strategy for parameter optimization during model training. By calculating and evaluating the contribution of different parameter blocks to the loss function, it selectively updates only those parameter blocks that contribute the most to the task, thereby further improving parameter efficiency while ensuring model performance. Simultaneously, this invention inserts attention enhancement adapters at key layers of the network, significantly enhancing the model's ability to perceive visual details such as minute moiré patterns or edge distortions by introducing an additional attention mechanism. Furthermore, a parameter sharing regularization mechanism is used to establish constraints on inter-layer parameters, preventing overfitting of the model to the training data during fine-tuning. This structured low-rank fine-tuning method ultimately only requires adjusting approximately 0.5% of the original model's parameters, effectively ensuring computational efficiency while significantly improving the model's high-precision recognition capability and detection recall for re-photographed features, achieving efficient adaptation of large models under computationally limited conditions.

[0123] The image copy detection method provided by this invention calculates the backpropagation gradient of the loss value and uses norm to calculate and quantify the gradient response value of each layer. It can accurately identify the key network layer in the basic model that is most sensitive to the copy task, thereby providing a scientific basis for subsequent adapter insertion and ensuring that the fine-tuning process can focus on the region that contributes the most to feature extraction, thereby maximizing the optimization of model performance.

[0124] Figure 4 This is one of the flowcharts illustrating the process of generating reproduced image samples provided by the present invention, such as... Figure 4 As shown, as another optional embodiment provided by the present invention, the reproduced image sample is obtained based on the following steps:

[0125] Step 410: Collect the detection feedback state information of the image detection model in the historical training rounds, and input the detection feedback state information into the policy network.

[0126] The detection feedback state information refers to the indicator data reflecting the current image detection model's ability to recognize various types of samples during the training process. For example, this detection feedback state information may include the image detection model's recognition accuracy, false positive rate, and loss value trend for a specific type of re-photographed sample (such as a trapezoidal distortion sample at a certain angle) in the previous training round, as well as the confidence distribution of the image detection model's feature extraction for that specific type of re-photographed sample. These data together constitute the state space of the current training environment.

[0127] A policy network refers to a decision neural network model built based on reinforcement learning algorithms (such as the Actor-Critic algorithm). For example, an Actor Network receives the aforementioned detection feedback state information as input, and its goal is to learn how to generate distortion parameter policies that can maximally challenge the recognition capabilities of the current image detection model, thereby guiding the generation of samples towards the weak points of the current image detection model.

[0128] As an optional implementation, the Critic Network evaluates sample quality and provides optimization signals. Its main responsibility is to build a feedback evaluation mechanism to measure whether the distortion parameters generated by the policy network have truly and effectively improved the training value of the samples.

[0129] Specifically, the value network receives the same detection feedback state information as the policy network, along with the actions generated by the policy network (i.e., distortion parameters), and outputs a predicted value score. This value score aims to estimate the probability or loss gain that the generated copy sample will cause the image detection model to misclassify it. If, during actual training, the sample successfully fools the detection model (i.e., causes a high loss), the Critic network returns a positive optimization signal (Advantage function) to the policy network, encouraging it to continue adopting similar generation strategies. Conversely, if the generated sample is easily recognized, a negative signal is returned to suppress the generation of such parameters, thereby guiding the entire generation system to continuously evolve towards creating more challenging boundary samples.

[0130] Step 420: Obtain the distortion generation parameters output by the policy network based on the detection feedback state information.

[0131] Specifically, the policy network outputs a set of action vectors to guide image reconstruction based on the input state information. For example, the distortion generation parameters may include specific geometric transformation parameters (such as rotation angle, perspective transformation matrix coefficients), optical interference parameters (such as moiré frequency, reflection intensity, blur kernel size), and physical medium parameters (such as simulated screen pixel density). These parameters define the specific shape of the re-photographed sample to be generated.

[0132] Step 430: The original compliant image is reconstructed using a sample generation network based on the distortion generation parameters to obtain a re-photographed image sample.

[0133] Specifically, the sample generation network acts as an execution engine, applying the aforementioned parameters to the original image using computer graphics algorithms or Generative Adversarial Networks (GANs). For example, based on the received parameters, it performs a 3D rotation at a specific angle on the original clear image to simulate trapezoidal distortion, superimposes sinusoidal waves of a specific frequency to simulate moiré patterns, and fuses Gaussian noise to simulate shooting noise, ultimately synthesizing a realistic and highly difficult-to-recognize copy image sample for the next round of model training.

[0134] As an optional embodiment, the present invention employs a layered and progressive simulation strategy in the sample generation process: first, basic screen features are generated to simulate the pixel matrix and color gamut performance of screens of different materials; then, the influence of ambient light is superimposed to introduce the light source reflection effects of different angles and color temperatures; finally, the imaging process of the shooting device is simulated, adding perspective distortion, focus blur and photosensitive noise, thereby achieving accurate simulation of the complete re-shooting scene.

[0135] It should be noted that, to improve training efficiency, this invention employs a feature difficulty gradient sampling mechanism. This mechanism dynamically adjusts the distribution range of generated parameters based on the model's misjudgment patterns during training. Combined with a failure-case-driven generation mechanism, it specifically analyzes model detection blind spots and strengthens the generation of corresponding difficult samples, forming an adaptive optimization generation strategy. Furthermore, the training process follows the principle of tiered difficulty learning, controlling the generated samples to gradually transition from easily identifiable features with obvious signs of duplication to challenging scenarios with weaker features. This gradual approach not only significantly improves the model's generalization ability to complex duplication scenarios but also greatly reduces the reliance on manually labeled data.

[0136] The image copying detection method provided by this invention utilizes a policy network to dynamically generate targeted distortion parameters based on the model's historical training feedback, and reconstructs copying samples accordingly. This enables the construction of an adaptive adversarial training closed loop, automatically generating difficult samples that the current model cannot recognize, thereby forcing the model to continuously learn and overcome its own weaknesses, and improving the model's generalization ability and robustness when facing unknown and complex copying methods.

[0137] Figure 5 This is the second schematic diagram of the process for generating reproduced image samples provided by the present invention, as shown below. Figure 5 As shown, in another optional embodiment provided by the present invention, after obtaining the distortion generation parameters output by the policy network based on the detection feedback state information, the method further includes:

[0138] Step 510: Obtain physical environment parameters from the physical parameter library. The physical environment parameters include at least screen reflectivity, screen color gamut parameters, pixel arrangement structure parameters, and ambient light color temperature parameters.

[0139] Specifically, this invention pre-constructs a database containing attributes of common display devices and shooting environments. For example, this physical parameter database stores the different spectral reflectance curves of In-Plane Switching (IPS), Vertical Alignment (VA), and OLED screens, the coverage data of sRGB and P3 color gamuts, and the diamond-shaped or striped sub-pixel arrangement unique to different brands of mobile phone screens. It also includes the color temperature values ​​of common ambient light sources such as fluorescent lamps (cool light) and incandescent lamps (warm light).

[0140] Step 520: Input the physical environment parameters into the physical constraint generation network and obtain the style modulation vector generated by the physical constraint generation network.

[0141] A constraint-generating network is a neural network used to transform discrete physical parameters into potential feature vectors that stylize the image generation process. For example, this constraint-generating network can be the encoding part of a multilayer perceptron or a conditional variational autoencoder (CVAE), responsible for learning the potential influence of physical laws on the visual effects of images.

[0142] A style modulation vector is a control vector used to adjust the statistical properties of features (such as mean and variance) in the intermediate layers of a generative network. For example, this vector is similar to the style code in style transfer techniques, which carries physical property information such as the texture of highly reflective glass or the lighting in a warm-colored greenhouse, and is used to constrain the final generated image to conform to the laws of optical physics.

[0143] Step 530: The original compliant image is reconstructed using a sample generation network based on the style modulation vector and distortion generation parameters to obtain a re-photographed image sample.

[0144] Specifically, during the sample generation process, not only is geometric deformation performed based on distortion generation parameters, but the image features are also rendered using style modulation vectors through Adaptive Instance Normalization (AdaIN) technology. For example, while rotating the image, the sample generation network superimposes light spots conforming to the Fresnel reflection principle onto the image surface according to the style modulation vector, and injects corresponding mesh textures at the microscopic level according to the pixel arrangement parameters, thereby generating a reproduced image sample that both complies with distortion resistance and conforms to the physical laws of imaging.

[0145] The image copy detection method provided by this invention introduces a constraint mechanism that includes real physical parameters such as screen reflectivity and pixel structure, and uses style modulation vectors to correct the physical laws of the sample generation process. This ensures that the generated adversarial examples are not only difficult to identify, but also highly consistent with the physical imaging laws of the real world in terms of light, shadow, texture and color. This effectively avoids generating artifact data that is out of touch with reality, thereby significantly improving the authenticity of the training data and the generalization performance of the model in real physical scenarios.

[0146] It should be noted that this invention has demonstrated significant technical advantages and beneficial effects in practical applications. First, the ability to recognize multiple types of re-photographing features has been improved. This invention can accurately identify seven types of re-photographing features, including moiré patterns, screen reflections or shadows, visible shooting devices, electronic screen interface marks, pixelation distortion, perspective distortion, and abnormal highlights, with a recall rate of over 85%. Compared to traditional methods that can only cover one or two types of features and have a recall rate of less than 60%, its coverage is greatly improved, significantly expanding the adaptability to detection scenarios.

[0147] Secondly, the false positive rate is significantly reduced. This technology has a false positive rate of less than 1% for normal images, which is 29 percentage points lower than the approximately 30% false positive rate of traditional methods, effectively reducing the pressure of subsequent manual review. Thirdly, it exhibits outstanding zero-shot generalization ability. Through adaptive sample generation technology, this technology can effectively identify untrained copying types (such as trapezoidal distortion caused by non-direct shooting angles), and the detection process is unaffected by changes in image content, overcoming the limitation of traditional methods being easily influenced by content.

[0148] Finally, the labor cost of re-inspection has been significantly reduced. Based on the actual test data of the after-sales inspection system, in a scenario where 40 million files are processed annually and each file contains an average of 3 images, this solution reduces the false alarm rate from 30% in the traditional solution to 1%, thereby reducing the number of images to be reviewed annually from 36 million to 1.2 million. Assuming a review time of 30 seconds per image and 2,000 man-hours per person per year, the manpower requirement has been reduced from 150 person-years in the traditional solution to 5 person-years in this solution, with an overall reduction of 97%, which greatly saves operating costs.

[0149] Figure 6 This is the second flowchart of the image reproduction detection method provided by the present invention, as shown below. Figure 2 As shown, the process begins with the input image, which is then fed into two parallel processing branches. In the left-hand visual perception branch, the input image enters a visual feature extraction model, processed by deep convolutional or Transformer modules, outputting multi-dimensional visual features containing both global and local image information. In the right-hand rule-guided branch, the input image is fed in parallel into seven rule sub-networks. These sub-networks extract specific features for seven different types of copying marks, such as moiré patterns, reflections, and distortions, and output corresponding feature extraction and probability results. Subsequently, these dispersed sub-features undergo dimensional concatenation and unified mapping by a conditional aggregation module, forming an aggregated rule vector. Further, residual injection is used to incrementally stack the rule information to prevent feature degradation. Finally, the visual features from the left branch and the rule features from the right branch (processed with residual injection) converge into a cross-modal attention layer. Here, an attention mechanism is used to achieve deep interaction and fusion of features, and the fused features are transmitted to the downstream task output module. This downstream task output module ultimately determines and outputs the detection result of whether the image is a copy.

[0150] Figure 7 This is the third flowchart of the image reproduction detection method provided by the present invention, as shown below. Figure 7 As shown, Figure 7This paper demonstrates the detailed execution flow of the image copy detection method provided by an embodiment of the present invention. After the process begins, an input image step is performed to obtain the image data to be detected. Then, seven types of features are extracted from the input image to obtain basic visual features, including moiré patterns, reflections, and distortions. Next, a judgment step is initiated to determine whether the features exceed the set judgment conditions, i.e., to assess whether the intensity of the extracted features is sufficient for direct qualitative analysis. If the judgment result is yes, it indicates obvious copying, and the copying result step is directly output, entering the termination state. If the judgment result is no, it indicates that the input image belongs to a complex sample with weak features, and the feature rules are input into a fine-tuned image detection model for deep inference. Within this image detection model, the processing flow is further subdivided into two paths: one path enhances the image features through visual encoding injected via a cross-modal adapter, thereby calculating the regional semantic contradiction analysis result to identify logical conflicts between content and medium; the other path generates descriptive text output (copying result, copying features) from the image detection model, and performs mapping text result processing on this natural language description to convert it into quantitative indicators. Finally, the two sets of information are combined, the weighted results of each result are calculated, and the final detection conclusion is obtained. The process is then completed and ends.

[0151] Figure 8 This is a schematic diagram of the image reproduction detection device provided by the present invention, as shown below. Figure 8 As shown, it mainly includes, but is not limited to:

[0152] The feature extraction unit 810 is used to input the image to be recognized into the feature extraction model and obtain the multi-dimensional image visual features output by the feature extraction model.

[0153] The re-enactment detection unit 820 is used to input the multi-dimensional image visual features into the image detection model and obtain the re-enactment detection result output by the image detection model if the multi-dimensional image visual features meet the preset re-enactment judgment conditions. The preset re-enactment judgment conditions are that there are no texture features and / or pixel grid features in the multi-dimensional image visual features.

[0154] The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for judging re-photographed images and generates rule condition vectors. The cross-modal fusion network fuses the rule condition vectors and multi-dimensional image visual features to generate fused features. The output network determines the re-photographing detection result based on the fused features.

[0155] It should be noted that the image copy detection device provided by the present invention can execute the image copy detection method described in any of the above embodiments during specific operation, which will not be elaborated in this embodiment.

[0156] The image copy detection device provided by this invention achieves targeted capture of copy features by encoding copy judgment rules into vectors and fusing them with visual features across modalities, thereby improving the accuracy of copy recognition and robustness against complex copy methods.

[0157] Figure 9 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 9 As shown, the electronic device may include: a processor 910, a communications interface 920, a memory 930, and a communications bus 940, wherein the processor 910, the communications interface 920, and the memory 930 communicate with each other through the communications bus 940. The processor 910 can call logic instructions in the memory 930 to execute an image re-photographing detection method. This method includes: inputting the image to be identified into a feature extraction model to obtain multi-dimensional image visual features output by the feature extraction model; if the multi-dimensional image visual features meet a preset re-photographing judgment condition, then inputting the multi-dimensional image visual features into an image detection model to obtain a re-photographing detection result output by the image detection model; the preset re-photographing judgment condition is that there are no texture features and / or pixel grid features in the multi-dimensional image visual features; wherein, the image detection model includes a rule encoding network, a cross-modal fusion network, and an output network; the rule encoding network encodes the re-photographing image judgment rules to generate a rule condition vector; the cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features; and the output network determines the re-photographing detection result based on the fused features.

[0158] Furthermore, the logical instructions in the aforementioned memory 930 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0159] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer can execute the image re-photographing detection method provided in the above embodiments, the method including: inputting the image to be identified into a feature extraction model to obtain multi-dimensional image visual features output by the feature extraction model; if the multi-dimensional image visual features meet a preset re-photographing judgment condition, then inputting the multi-dimensional image visual features into an image detection model to obtain a re-photographing detection result output by the image detection model; the preset re-photographing judgment condition is that there are no texture features and / or pixel grid features in the multi-dimensional image visual features; wherein, the image detection model includes a rule encoding network, a cross-modal fusion network and an output network; the rule encoding network encodes the re-photographing image judgment rules to generate a rule condition vector; the cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features; the output network determines the re-photographing detection result based on the fused features.

[0160] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the image re-photographing detection method provided in the above embodiments. The method includes: inputting an image to be identified into a feature extraction model to obtain multi-dimensional image visual features output by the feature extraction model; if the multi-dimensional image visual features meet a preset re-photographing judgment condition, then inputting the multi-dimensional image visual features into an image detection model to obtain a re-photographing detection result output by the image detection model; the preset re-photographing judgment condition is that there are no texture features and / or pixel grid features in the multi-dimensional image visual features; wherein, the image detection model includes a rule encoding network, a cross-modal fusion network, and an output network; the rule encoding network encodes the re-photographing image judgment rules to generate a rule condition vector; the cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features; and the output network determines the re-photographing detection result based on the fused features.

[0161] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0162] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting image reproduction, characterized in that, include: The image to be identified is input into the feature extraction model to obtain the multidimensional image visual features output by the feature extraction model; If the multidimensional image visual features meet the preset re-photographing judgment conditions, then the multidimensional image visual features are input into the image detection model to obtain the re-photographing detection result output by the image detection model; the preset re-photographing judgment conditions are that there are no texture features and / or pixel grid features in the multidimensional image visual features; The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for determining whether an image is a copy, generating a rule condition vector. The cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features. The output network determines the copy detection result based on the fused features. The output network determines the re-photograph detection result based on the fused features, specifically including: If the fusion features contain physical features and / or optical anomalies of the display device, then the image to be identified is determined to be a reproduced image; the physical features are hard physical traces introduced during the reproduction process due to secondary imaging. If the fusion feature indicates a logical conflict between the semantic content of the image to be identified and the imaging medium, then the image to be identified is determined to be a reproduced image.

2. The image reproduction detection method according to claim 1, characterized in that, The rule encoding network includes multiple rule extraction sub-networks, each of which extracts different rule sub-features; The image detection model further includes a condition aggregation module, which concatenates and maps multiple rule sub-features to generate the rule condition vector. The cross-modal fusion network uses the rule-based conditional vector to perform residual injection processing on the multidimensional image visual features to generate the fused features.

3. The image reproduction detection method according to claim 1, characterized in that, The cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate the fused features, specifically including: Generate a query vector based on the rule condition vector; Generate key vectors and value vectors based on the multidimensional image visual features; Calculate the similarity score between the query vector and the key vector, and determine the attention weight based on the similarity score; The fused feature is obtained by weighting and summing the value vector based on the attention weights.

4. The image reproduction detection method according to claim 1, characterized in that, The image detection model is trained based on the following steps: Obtain a general visual language model as the base model; Construct a sample dataset of reproduced images, wherein the sample dataset contains reproduced image samples with reproduced image judgment labels; Based on the sample dataset, gradient path analysis is performed on the basic model to determine the target network layer in the basic model whose gradient response value to the re-photographing feature is higher than a preset threshold. Keep the pre-trained parameters of the base model frozen, and introduce trainable adapter parameters into the target network layer; The adapter parameters are trained and updated using the sample dataset, and the base model containing the trained adapter parameters is determined as the image detection model.

5. The image reproduction detection method according to claim 4, characterized in that, The step of performing gradient path analysis on the base model based on the sample dataset to determine the target network layer in the base model whose gradient response value to the re-production feature is higher than a preset threshold includes: The sample of the re-photographed image in the sample dataset is input into the base model for forward propagation to obtain the prediction result, and the loss value between the prediction result and the re-photographing judgment label is calculated. Backpropagation is performed on the loss value to obtain the gradient matrix corresponding to the weight parameters of each network layer in the base model; The norm of the gradient matrix of each network layer is calculated, and the calculation result is determined as the gradient response value of the network layer. The network layer whose gradient response value is greater than a preset response threshold is determined as the target network layer.

6. The image reproduction detection method according to claim 4, characterized in that, The reproduced image samples were obtained based on the following steps: Collect the detection feedback state information of the image detection model in the historical training rounds, and input the detection feedback state information into the policy network; Obtain the distortion generation parameters output by the policy network based on the detection feedback state information; The original compliant image is reconstructed using a sample generation network based on the distortion generation parameters to obtain the re-photographed image sample.

7. The image reproduction detection method according to claim 6, characterized in that, After obtaining the distortion generation parameters output by the policy network based on the detection feedback state information, the method further includes: Physical environment parameters are obtained from a physical parameter library, including at least screen reflectivity, screen color gamut parameters, pixel arrangement structure parameters, and ambient light color temperature parameters. The physical environment parameters are input into the physical constraint generation network to obtain the style modulation vector generated by the physical constraint generation network. The original compliant image is reconstructed using the sample generation network based on the style modulation vector and the distortion generation parameters to obtain the re-photographed image sample.

8. A photo reproduction detection device, characterized in that, include: The feature extraction unit is used to input the image to be identified into the feature extraction model and obtain the multi-dimensional image visual features output by the feature extraction model. A re-enactment detection unit is used to input the multi-dimensional image visual features into an image detection model and obtain the re-enactment detection result output by the image detection model if the multi-dimensional image visual features meet a preset re-enactment judgment condition; the preset re-enactment judgment condition is that there are no texture features and / or pixel grid features in the multi-dimensional image visual features. The image detection model includes a rule encoding network, a cross-modal fusion network, and an output network. The rule encoding network encodes the rules for determining whether an image is a copy, generating a rule condition vector. The cross-modal fusion network fuses the rule condition vector and the multi-dimensional image visual features to generate fused features. The output network determines the copy detection result based on the fused features. The output network determines the re-photograph detection result based on the fused features, specifically including: If the fusion features contain physical features and / or optical anomalies of the display device, then the image to be identified is determined to be a reproduced image; the physical features are hard physical traces introduced during the reproduction process due to secondary imaging. If the fusion feature indicates a logical conflict between the semantic content of the image to be identified and the imaging medium, then the image to be identified is determined to be a reproduced image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image reproduction detection method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image reproduction detection method as described in any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the image reproduction detection method as described in any one of claims 1 to 7.

Images