News picture tampering detection method based on deep learning

Abstract

The invention discloses a news picture tampering detection method based on deep learning, and relates to the technical field of news picture tampering detection methods, the news picture tampering detection method comprises the following steps: using a data collection method to collect news pictures, and preprocessing the news pictures to obtain a preprocessed data set; constructing a feature extraction model by adopting a multi-branch convolutional neural network (CNN) in combination with an attention mechanism, inputting the preprocessed data set into the feature extraction model, and performing feature extraction to obtain feature data; training the feature data by adopting a customized loss function to obtain an optimization target adaptive to the tampering detection task; optimizing the feature extraction model based on an optimization target adapted to the tampering detection task to obtain an optimized feature extraction model; refining the result of the optimized feature extraction model by adopting a post-processing and result analysis method to obtain a tampering detection report; a feature extraction model is constructed by combining an attention mechanism based on a multi-branch convolutional neural network CNN.

Description

Technical Field

[0001] The present invention relates to the technical field of news picture forgery detection methods, and particularly to a news picture forgery detection method based on deep learning. Background Art

[0002] The news picture forgery detection method is a technology aiming to identify and verify whether a news picture has been modified or forged in any form. The method usually uses computer vision and deep learning algorithms to judge the authenticity of the picture by analyzing the pixel-level features, metadata, and image processing traces of the image.

[0003] In the field of news picture forgery detection methods, in news picture forgery detection, the number of normal pictures far exceeds that of forged pictures, resulting in the problem of class imbalance when training the model, making the model tend to predict samples of most categories, that is, tend to identify all pictures as unforged, and the traditional convolutional neural network cannot fully capture the subtle forgery traces in the image, especially those changes in local details or global semantic information. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a news picture forgery detection method based on deep learning to solve the problem of class imbalance that occurs when training the model because the number of normal pictures far exceeds that of forged pictures in news picture forgery detection.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: In a first aspect, the present invention provides a news picture forgery detection method based on deep learning, which includes: Collect news pictures by using a data collection method, and preprocess the news pictures to obtain a preprocessed data set; Construct a feature extraction model by using a multi-branch convolutional neural network (CNN) combined with an attention mechanism, input the preprocessed data set into the feature extraction model for feature extraction to obtain feature data; Train the feature data by using a customized loss function to obtain an optimization target adapted to the forgery detection task; Optimize the feature extraction model based on the optimization target adapted to the forgery detection task to obtain an optimized feature extraction model; Refine the results of the optimized feature extraction model by using a post-processing and result analysis method to obtain a forgery detection report.

[0007] As a preferred solution of the deep learning-based news picture forgery detection method of the present invention, the method includes: collecting news pictures by using a data collection method, and preprocessing the news pictures to obtain a data set containing normal and forged samples. The specific steps are as follows: Use a data scraping technology based on RSS feeds and API interfaces to collect news pictures from multiple public news sources and social media platforms to obtain original news pictures; Annotate the original news pictures to clearly mark whether there is forgery behavior and its specific forgery area; Use image processing technology to enhance the original news pictures; The enhancement operations include scaling, rotation, and flipping, and a preprocessed data set is obtained.

[0008] As a preferred solution of the deep learning-based news picture forgery detection method of the present invention, the method includes: constructing a feature extraction model by using a multi-branch convolutional neural network CNN combined with an attention mechanism, inputting the preprocessed data set into the feature extraction model for feature extraction to obtain feature data. The specific steps are as follows: Based on the multi-branch convolutional neural network CNN; Design a multi-branch CNN architecture, where one branch focuses on extracting local detail features and the other branch is used to capture global semantic information; Introduce the Self-Attention mechanism into the multi-branch CNN architecture, and the expression is: ; Among them, , , are the query, key, and value matrices respectively, is the dimension size of the key.

[0009] As a preferred solution of the deep learning-based news picture forgery detection method of the present invention, the method includes: inputting the preprocessed data set into the feature extraction model to output a feature set. The specific steps are as follows: Input the preprocessed data set into the feature extraction model. Through the forward propagation process, calculate the local features in the local detail branch and the global features in the global semantic branch. The expression is: ; ; Among them, represents the input image, and represent the convolution operations of the local detail branch and the global semantic branch respectively; Combining local detail features and global semantic features, a weighted fusion strategy is used to integrate the two types of features, and the expression is: ; where is the weight coefficient, which is used to adjust the importance ratio of local detail features and global semantic features, is the fused feature set.

[0010] As a preferred solution of the news picture forgery detection method based on deep learning described in the present invention, wherein: the customized loss function is used to train the feature data to obtain an optimization objective suitable for the forgery detection task, and the specific steps are as follows: Define a weighted cross-entropy loss function to balance the influence of different category samples, and the expression is: ; where is the total number of samples, and are the weight coefficients of positive samples and negative samples respectively, satisfying > to increase the attention to minority-class forgery samples, is the predicted probability that the model assigns the th sample to the positive class, is the actual label; Based on the weighted cross-entropy loss, the Focal Loss mechanism is introduced to obtain a modified loss function, and the expression is: ; where is the adjustment parameter, which is used to adjust the ratio of positive and negative samples, is the focusing parameter.

[0011] As a preferred solution of the news picture forgery detection method based on deep learning described in the present invention, wherein: the feature extraction model is optimized based on the optimization objective suitable for the forgery detection task to obtain an optimized feature extraction model, and the specific steps are as follows: Initialize the multi-branch convolutional neural network with the model parameters pre-trained on a large-scale dataset; Adopt the Adam optimizer to minimize the Focal Loss; Input the entire training set into the model in batches, and perform forward propagation to calculate the loss for each batch; Perform backpropagation to calculate the gradient, and use the Adam optimizer to update the model parameters until the maximum number of iterations is reached, and obtain an optimized feature extraction model.

[0012] As a preferred solution of the deep learning-based news picture forgery detection method of the present invention, wherein: the post-processing and result analysis method is used to refine the results of the optimized feature extraction model to obtain a forgery detection report. The specific steps are as follows: Apply the connected component analysis technique to the forgery probability map predicted by each optimized feature extraction model; Let represent the forgery probability value of the pixel at coordinate . Then the pixels satisfying > are marked as suspected forgery areas; Set function to identify the connected regions formed by pixels. The expression is: ; wherein, , respectively represent the horizontal and vertical coordinates of the image pixels, is a preset probability threshold; Remove small connected regions with an area smaller than the threshold to improve the accuracy of the detection results.

[0013] As a preferred solution of the deep learning-based news picture forgery detection method of the present invention, wherein: removing small connected regions with an area smaller than the threshold to improve the accuracy of the detection results, the expression is: ; wherein, is the area of the connected region, is the set minimum area threshold; Generate a forgery detection report according to the processed results; The forgery detection report includes the location, size, and confidence score of each detected forgery area The form of the forgery detection report is: ; wherein, , are the coordinates of the upper left corner of the forgery area respectively, , are the width and height of the forgery area respectively, is the confidence score of this area, is the total number of detected forgery areas.

[0014] In a second aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the method for detecting forged news pictures based on deep learning as described in the first aspect of the present invention is implemented.

[0015] In a third aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the method for detecting forged news pictures based on deep learning as described in the first aspect of the present invention is implemented.

[0016] The beneficial effects of the present invention are as follows: By constructing a feature extraction model based on a multi-branch convolutional neural network (CNN) combined with an attention mechanism, and inputting the preprocessed data set into the model for feature extraction to obtain feature data, the effective capture of local detail features and global semantic information of news pictures is realized. By introducing the Self-Attention mechanism, the model can pay more attention to the key areas that may be forged. By using a customized loss function to train the feature data, an optimization target adapted to the forgery detection task is obtained, realizing the focused learning of forged samples in the case of an imbalanced data set, enabling the model to still effectively learn when the ratio of positive and negative samples is seriously imbalanced, solving the challenge brought by class imbalance, making the model more focused on those forged samples that are difficult to classify, thereby improving the detection efficiency and accuracy. Using the Adam optimizer to minimize the Focal Loss, combined with the connected component analysis technique and the method of removing isolated small noise regions, effectively improves the accuracy and interpretability of the detection results. Description of the Drawings

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a flowchart of the method for detecting forged news pictures based on deep learning in Embodiment 1.

[0019] Figure 2 It is a flowchart of the preprocessed data set in Embodiment 1. Detailed Embodiments

[0020] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following will give a detailed description of the specific embodiments of the present invention in conjunction with the drawings in the specification.

[0021] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. However, the present invention may be practiced in other ways different from those described herein. Those skilled in the art can make similar generalizations without departing from the spirit of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0022] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude each other with other embodiments.

[0023] Example 1, referring to Figure 1 and Figure 2 , is the first embodiment of the present invention. This embodiment provides a method for detecting forged news pictures based on deep learning, including the following steps: S1. Use a data collection method to collect news pictures and preprocess the news pictures to obtain a preprocessed data set; Furthermore, use a data scraping technology based on RSS feeds and API interfaces to collect news pictures from multiple public news sources and social media platforms to obtain original news pictures; Label the original news pictures to clearly mark whether there is a forgery behavior and its specific forgery area; Use image processing technology to enhance the original news pictures; The enhancement operations include scaling, rotation, and flipping, and a preprocessed data set is obtained; It should be noted that by using the data scraping technology based on RSS feeds and API interfaces, the extensiveness and real-time nature of the data source are ensured, and the latest news pictures can be obtained in a timely manner. The data collection method not only covers a variety of news sources but also can capture hot information on social media, providing rich materials for subsequent model training. In addition, by labeling and enhancing the original pictures, the quality of the data set is improved, enabling the model to learn under a wider range of conditions, thereby enhancing its robustness and generalization ability.

[0024] S2. Use a multi-branch convolutional neural network CNN combined with an attention mechanism to construct a feature extraction model, input the preprocessed data set into the feature extraction model for feature extraction to obtain feature data; Furthermore, based on the multi-branch convolutional neural network CNN; Design a multi-branch CNN architecture, where one branch focuses on extracting local detail features and the other branch is used to capture global semantic information; Introduce the Self-Attention mechanism into the multi-branch CNN architecture, and the expression is: ; Among them, , , are the query, key, and value matrices respectively, is the dimension size of the key; Input the preprocessed dataset into the feature extraction model. Through the forward propagation process, calculate the local features in the local detail branch and the global features in the global semantic branch. The expression is: ; ; Among them, represents the input image, and represent the convolution operations of the local detail branch and the global semantic branch respectively; Combine the local detail features and the global semantic features, and use a weighted fusion strategy to integrate the two types of features. The expression is: ; Among them, is the weight coefficient, used to adjust the importance ratio of the local detail features and the global semantic features, is the fused feature set; It should be noted that the introduction of the Self-Attention mechanism and the design of the multi-branch CNN architecture significantly improve the model's ability to capture local detail and global semantic information. The Self-Attention mechanism helps the model pay more attention to the key areas that may be tampered with by calculating the similarity scores between the query, key, and value matrices, while the multi-branch design allows the model to capture information at different levels simultaneously. This combination not only improves the accuracy of feature extraction but also enhances the model's ability to recognize complex tampering patterns, thereby improving the overall detection effect.

[0025] S3. Use a customized loss function to train the feature data to obtain an optimization target suitable for the tampering detection task; Furthermore, define a weighted cross-entropy loss function to balance the influence of different category samples. The expression is: ; Among them, is the total number of samples, and are the weight coefficients of the positive and negative samples respectively, satisfying > to increase the attention to the minority-class tampered samples, The predicted probability that the model classifies the th sample as a positive class, is the actual label; Based on the weighted cross-entropy loss, the Focal Loss mechanism is introduced to obtain the modified loss function, and the expression is: ; where, is the adjustment parameter used to adjust the proportion of positive and negative samples, is the focusing parameter; It should be noted that by defining the weighted cross-entropy loss function and introducing the Focal Loss mechanism, the challenges brought by class imbalance are solved. The traditional cross-entropy loss tends to favor majority-class samples when dealing with imbalanced data, while the weighted cross-entropy and Focal Loss make the model pay more attention to the minority class (i.e., tampered samples) by adjusting the weights and focusing parameters, thereby improving the learning efficiency of difficult-to-classify samples. This method not only improves the detection performance of the model but also shows stronger adaptability and stability in practical applications.

[0026] S4. Optimize the feature extraction model based on the optimization objective adapted to the tampering detection task to obtain the optimized feature extraction model; Furthermore, initialize the multi-branch convolutional neural network with the model parameters pre-trained on a large-scale dataset; Use the Adam optimizer to minimize the Focal Loss; Input the entire training set into the model in batches, and perform forward propagation to calculate the loss for each batch; Perform backpropagation to calculate the gradients and use the Adam optimizer to update the model parameters until the maximum number of iterations is reached, and obtain the optimized feature extraction model; It should be noted that using the Adam optimizer to minimize the Focal Loss and optimizing the model parameters through backpropagation and gradient update ensure the efficiency and stability of the model during training. The Adam optimizer combines the advantages of momentum and adaptive learning rate and can converge quickly in complex loss landscapes. In addition, initializing with the model parameters pre-trained on a large-scale dataset further accelerates the training process and improves the initial performance, making the finally obtained optimized model more accurate and reliable.

[0027] S5. Refine the results of the optimized feature extraction model using post-processing and result analysis methods to obtain the tampering detection report; Furthermore, apply the connected component analysis technique to the tampering probability map predicted by each optimized feature extraction model; Let represent the coordinate The tampering probability value of the pixel at the position satisfies > The pixel is marked as a suspected tampering area; Set The function identifies the connected region formed by the pixels, and the expression is: ; Among them, , respectively represent the horizontal and vertical coordinates of the image pixels, is the preset probability threshold; Remove small connected regions with an area smaller than the threshold to improve the accuracy of the detection result, and the expression is: ; Among them, is the area of the connected region, is the set minimum area threshold; Generate a tampering detection report based on the processed result; The tampering detection report includes the location, size, and confidence score of each detected tampering area The form of the tampering detection report is: ; Among them, , are the coordinates of the upper left corner of the tampering area respectively, , are the width and height of the tampering area respectively, is the confidence score of this area, is the total number of detected tampering areas; It should be noted that through the connected component analysis technology and the method of removing isolated small noise areas, the accuracy and interpretability of the detection result are significantly improved. The connected component analysis helps to accurately define the boundary of the tampering area, while removing small area noise areas reduces the possibility of false alarms. The finally generated tampering detection report not only provides the specific location and size of each tampering area, but also includes the confidence score, which provides clear and detailed tampering evidence for users and greatly improves the practical value of the system and the trust of users.

[0028] This embodiment also provides a computer device applicable to the case of the news picture tampering detection method based on deep learning, including: a memory and a processor; the memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions to implement the news picture tampering detection method based on deep learning as proposed in the above embodiment.

[0029] The computer device may be a terminal, and the computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through Wi-Fi, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0030] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for detecting news picture tampering based on deep learning proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0031] In summary, the present invention constructs a feature extraction model by combining a multi-branch convolutional neural network CNN with an attention mechanism, and inputs the preprocessed data set into the model for feature extraction to obtain feature data, realizing the effective capture of local detail features and global semantic information of news pictures. By introducing the Self-Attention mechanism, the model can pay more attention to the key areas where tampering may occur. By using a customized loss function to train the feature data, an optimization target suitable for the tampering detection task is obtained, realizing the focused learning of tampered samples in the case of an imbalanced data set, enabling the model to still effectively learn in the case of a serious imbalance in the ratio of positive and negative samples, solving the challenge brought by class imbalance, making the model more focused on those tampered samples that are difficult to classify, thereby improving the detection efficiency and accuracy. Using the Adam optimizer to minimize the Focal Loss, combined with the connected component analysis technique and the method of removing isolated small noise regions, effectively improves the accuracy and interpretability of the detection results.

[0032] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting forged news pictures based on deep learning, characterized in that: Including: Adopt a data collection method to collect news pictures and preprocess the news pictures to obtain a preprocessed data set; Construct a feature extraction model by using a multi-branch convolutional neural network (CNN) combined with an attention mechanism. Input the preprocessed data set into the feature extraction model for feature extraction to obtain feature data; Adopt a customized loss function to train the feature data to obtain an optimization objective suitable for the tampering detection task; Optimize the feature extraction model based on the optimization objective suitable for the tampering detection task to obtain an optimized feature extraction model; Adopt a post-processing and result analysis method to refine the results of the optimized feature extraction model to obtain a tampering detection report.

2. The method for detecting forged news pictures based on deep learning according to claim 1, wherein: The step of adopting a data collection method to collect news pictures and preprocess the news pictures to obtain a data set containing normal and tampered samples is as follows: Adopt a data scraping technology based on RSS feeds and API interfaces to collect news pictures from multiple public news sources and social media platforms to obtain original news pictures; Annotate the original news pictures to clearly mark whether there is a tampering behavior and its specific tampering area; Use image processing technology to enhance the original news pictures; The enhancement operations include scaling, rotation, and flipping, and a preprocessed data set is obtained.

3. The method for detecting news picture tampering based on deep learning according to claim 2, wherein: The step of constructing a feature extraction model by using a multi-branch convolutional neural network (CNN) combined with an attention mechanism, inputting the preprocessed data set into the feature extraction model for feature extraction to obtain feature data is as follows: Based on a multi-branch convolutional neural network (CNN); Design a multi-branch CNN architecture, where one branch focuses on extracting local detail features and the other branch is used to capture global semantic information; Introduce the Self-Attention mechanism into the multi-branch CNN architecture, and the expression is: ； Among them, , , are the query, key, and value matrices respectively, is the dimensionality size of the key.

4. The method for detecting forged news pictures based on deep learning according to claim 3, characterized in that: The step of inputting the preprocessed data set into the feature extraction model and outputting a feature set is as follows: Input the preprocessed data set into the feature extraction model. Through the forward propagation process, calculate the local features in the local detail branch and the global features in the global semantic branch, and the expression is: ； ； Among them, represents the input image, and respectively represent the convolution operations of the local detail branch and the global semantic branch; Combine the local detail features and the global semantic features, and use a weighted fusion strategy to integrate the two types of features, and the expression is: ; Among them, is a weight coefficient used to adjust the importance ratio of local detail features and global semantic features, is the fused feature set.

5. The method for detecting forged news pictures based on deep learning according to claim 4, wherein: The step of adopting a customized loss function to train the feature data to obtain an optimization objective suitable for the tampering detection task is as follows: Define a weighted cross-entropy loss function to balance the influence of different category samples, and the expression is: ; Among them, is the total number of samples, and are the weight coefficients of the positive samples and negative samples respectively, satisfying > to increase the attention to the minority-class tampered samples, is the predicted probability that the model assigns the -th sample to the positive class, is the actual label; Introduce the Focal Loss mechanism based on the weighted cross-entropy loss to obtain a modified loss function, and the expression is: ; Among them, is an adjustment parameter used to adjust the ratio of positive and negative samples, is a focusing parameter.

6. The method for detecting forged news pictures based on deep learning according to claim 5, characterized in that: The step of optimizing the feature extraction model based on the optimization objective suitable for the tampering detection task to obtain an optimized feature extraction model is as follows: Use the model parameters pre-trained on a large-scale data set to initialize the multi-branch convolutional neural network; Adopt the Adam optimizer to minimize the Focal Loss; Input the entire training set into the model in batches, and perform forward propagation to calculate the loss for each batch; Perform backpropagation to calculate gradients, and use the Adam optimizer to update the model parameters until the maximum number of iterations is reached, and obtain the optimized feature extraction model.

7. The method for detecting news picture forgery based on deep learning according to claim 6, characterized in that: The method of post-processing and result analysis is used to refine the results of the optimized feature extraction model to obtain a tampering detection report. The specific steps are as follows: Apply the connected component analysis technique to each tampering probability map predicted by the optimized feature extraction model; Let represent the tampering probability value of the pixel at the coordinate , then the pixels that satisfy > are marked as suspected tampering areas; Settings The function identifies the connected regions formed by pixel points, and the expression is: ； Among them, , respectively represent the horizontal and vertical coordinates of the image pixel, is a preset probability threshold; Remove small connected regions with an area smaller than the threshold to improve the accuracy of the detection results.

8. The method for detecting forged news pictures based on deep learning according to claim 7, characterized in that: The removed area is smaller than the threshold to improve the accuracy of the detection result. The expression is as follows: ； Among them, is the area of the connected region, is the set minimum area threshold; Generate a tampering detection report based on the processed results; The tampering detection report includes the location, size, and confidence score of each detected tampering area The form of the tampering detection report is: ; Among them, , are the coordinates of the upper left corner of the tampered area respectively, , are the width and height of the tampered area respectively, is the confidence score of this area, is the total number of detected tampered areas.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the deep learning-based news picture tampering detection method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the deep learning-based news picture tampering detection method according to any one of claims 1 to 8.

Images