A method, apparatus, device and medium for recognizing a fake image

Abstract

This invention discloses a method, apparatus, device, and medium for identifying forged images, relating to the field of artificial intelligence technology. The method includes: extracting features from an image to be identified to obtain image features; performing dimensionality reduction on the image features to obtain forensic features of the image to be identified; determining the Laplacian matrix of the forensic features, and based on the Laplacian matrix, determining the evaluation matrix of the forensic features and at least one feature block corresponding to the forensic features; determining the feature block weight map of the forensic features based on at least one feature block and the evaluation matrix; and determining whether the image to be identified is a forged image based on the feature block weight map. The technical solution of this invention, by determining the evaluation matrix and feature blocks of the forensic features of the image to be identified, and determining the feature block weight map, identifies the image to be identified. The image recognition process has low complexity, improving the accuracy and robustness of forged image identification.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and more particularly to the field of deep learning technology, specifically to a method, fault, device, and medium for identifying forged images. Background Technology

[0002] As the volume of image inspection services continues to grow, manual methods for identifying forged images are insufficient to meet the demands of this massive business. Intelligent forged image recognition methods, which require no human intervention, have been widely adopted.

[0003] Current intelligent methods for identifying forged images extract image features by establishing an evidence-gathering model, employing a long short-term memory network to capture the relationships between different image features, determining the probability of image tampering, and outputting a mask of the modified area. This method is highly complex, has low accuracy in identifying forged images, and exhibits poor robustness. Summary of the Invention

[0004] This invention provides a method, apparatus, device, and medium for identifying forged images, thereby improving the accuracy and robustness of forged image recognition.

[0005] According to one aspect of the present invention, a method for identifying forged images is provided, the method comprising:

[0006] Feature extraction is performed on the image to be identified to obtain the image features of the image to be identified;

[0007] The image features are reduced in dimensionality to obtain the forensic features of the image to be identified;

[0008] Determine the Laplace matrix of the evidence-gathering features, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering features and at least one feature block corresponding to the evidence-gathering features;

[0009] Based on at least one feature block and an evaluation matrix, determine the feature block weight map of the evidence-gathering features;

[0010] Based on the feature block weight map, determine whether the image to be identified is a forged image.

[0011] According to another aspect of the present invention, a forged image recognition device is provided, the device comprising:

[0012] The image feature determination module is used to extract features from the image to be identified, thereby obtaining the image features of the image to be identified.

[0013] The evidence feature determination module is used to perform dimensionality reduction processing on image features to obtain the evidence features of the image to be identified;

[0014] The evaluation matrix determination module is used to determine the Laplace matrix of the evidence-gathering feature, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering feature and at least one feature block corresponding to the evidence-gathering feature;

[0015] The weight map determination module is used to determine the feature block weight map of the evidence-gathering features based on at least one feature block and an evaluation matrix.

[0016] The forged image recognition module is used to determine whether the image to be recognized is a forged image based on the feature block weight map.

[0017] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory that is communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor, such that the at least one processor is able to perform the forgery image recognition method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for causing a processor to execute and implement the forged image recognition method of any embodiment of the present invention.

[0022] This invention provides a method for identifying images by extracting features from the image to be identified; performing dimensionality reduction on the image features to obtain forensic features; determining the Laplacian matrix of the forensic features; and based on the Laplacian matrix, determining the evaluation matrix of the forensic features and at least one feature block corresponding to the forensic features; determining the feature block weight map of the forensic features based on at least one feature block and the evaluation matrix; and determining whether the image to be identified is a forged image based on the feature block weight map. This invention's technical solution, by determining the evaluation matrix and feature blocks of the forensic features of the image to be identified, and determining the feature block weight map, enables the identification of the image. The image recognition process has low complexity, improving the accuracy and robustness of identifying forged images.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is a flowchart of a forged image recognition method provided in Embodiment 1 of the present invention;

[0026] Figure 2A This is a flowchart of a forged image recognition method provided in Embodiment 2 of the present invention;

[0027] Figure 2B This is a schematic diagram of the structure of a VGG network according to Embodiment 2 of the present invention;

[0028] Figure 3A This is a flowchart of a forged image recognition method provided in Embodiment 3 of the present invention;

[0029] Figure 3B This is a schematic diagram of a process for obtaining forensic features according to Embodiment 3 of the present invention;

[0030] Figure 4 This is a flowchart of a forged image recognition method provided in Embodiment 4 of the present invention;

[0031] Figure 5 This is a structural diagram of a forged image recognition device according to Embodiment 5 of the present invention;

[0032] Figure 6 This is a schematic diagram of the structure of an electronic device that implements the forged image recognition method provided in Embodiment Six of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Furthermore, it should be noted that the use, processing, transmission, provision, and disclosure of the feature data of the image to be identified in the technical solution of the present invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0036] The forged image recognition method provided by this invention can be applied to intelligent vehicle inspection scenarios and, optionally, integrated into a vehicle insurance policy issuance system. Specifically, after a user takes a vehicle inspection image and uploads it to the vehicle insurance policy issuance system, the front-end of the system preprocesses the image. This preprocessing may include adding time, adding an address watermark, or compressing the image size. The front-end then sends the preprocessed image to the back-end. The back-end calls a forged image recognition service to sequentially perform vehicle identification, VIN identification, license plate identification, re-photographing identification, and tampering identification on the image. After forged image recognition is completed, the results are stored in a database. After the user calculates the premium, the system retrieves the recognition results from the database and, based on a rule engine, determines whether the vehicle inspection image is a forged image.

[0037] Example 1

[0038] Figure 1 The flowchart of a forged image recognition method provided in Embodiment 1 of the present invention is applicable to the situation of forged image recognition of the image to be recognized, especially applicable to the situation of forged recognition of the captured vehicle inspection image in the intelligent vehicle inspection scenario. The method can be executed by a forged image device, which can be implemented in hardware and / or software and can be configured in an electronic device, such as a server.

[0039] like Figure 1 As shown, the method includes:

[0040] S101. Extract features from the image to be recognized to obtain the image features of the image to be recognized.

[0041] In this embodiment, the image to be identified can be an image currently waiting to be identified as a forged image, such as a vehicle inspection image uploaded by a user; image features refer to the features of the image to be identified, which may include, but are not limited to, color features, texture features, shape features, and spatial relationship features, and can be represented in matrix or vector form.

[0042] Specifically, a convolutional neural network (CNN) model is used to extract features from the image to be identified, thereby obtaining the image features of the image. It should be noted that the CNN model in this embodiment of the invention is not limited. For example, the CNN can be SENet (Squeeze-and-Excitation Networks), ResNet (Residual Neural Network), and DenseNet (Dense Convolutional Network), etc.

[0043] S102. Perform dimensionality reduction processing on the image features to obtain the evidence-gathering features of the image to be identified.

[0044] In this embodiment, the evidence-gathering features can refer to image features used for forged image identification, and can be represented in matrix or vector form.

[0045] Specifically, at least one dimensionality reduction method from existing technologies can be used to reduce the dimensionality of image features. For example, the dimensionality reduction method can be PCA (Principal Component Analysis), LEE (Locallylinear Embedding), or LDA (Linear Discriminant Analysis), etc.; and the dimensionality-reduced image features can be used as evidence features of the image to be identified.

[0046] S103. Determine the Laplace matrix of the evidence-gathering features, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering features and at least one feature block corresponding to the evidence-gathering features.

[0047] In this embodiment, the Laplacian matrix can be a standardized Laplacian matrix. The evaluation matrix can be used to characterize the similarity between pixels in the corresponding forensic feature. For example, if the difference between two element values ​​in the evaluation matrix is ​​small, then the pixels corresponding to those two element values ​​in the forensic feature are relatively similar; if the difference between two element values ​​in the evaluation matrix is ​​large, then the pixels corresponding to those two element values ​​in the forensic feature are significantly different. A feature block can be a block in the forensic feature composed of pixels with the same or similar features.

[0048] Specifically, based on the evidence feature matrix corresponding to the evidence features, an empty matrix with the same number of rows and columns as the evidence features is constructed. The pixel values ​​of each column of pixels in the evidence features are summed and sequentially assigned to the diagonal of the empty matrix to form a degree matrix. The evidence feature matrix is ​​subtracted from this degree matrix to obtain a non-normalized Laplacian matrix. The evidence feature matrix is ​​added to its transpose, and the result is multiplied by 0.5 to obtain a reference matrix. The reference matrix is ​​multiplied by the non-normalized Laplacian matrix, and then multiplied by the reference matrix again to obtain a normalized Laplacian matrix. Using a preset algorithm, the evaluation matrix of the evidence features and at least one feature block corresponding to the evidence features are determined based on the Laplacian matrix. For example, the Laplacian matrix of the evidence features can be determined using the following formula:

[0049] ;

[0050] ;

[0051] ;

[0052] Where L is the non-normalized Laplacian matrix; D is the degree matrix; W is the evidence feature matrix; C is the reference matrix; W T Let be the transpose of the evidence feature matrix; P is the standardized Laplace matrix.

[0053] S104. Based on at least one feature block and the evaluation matrix, determine the feature block weight map of the evidence collection features.

[0054] In this embodiment, the feature block weight map is a weight map composed of each feature block and its corresponding block weight; wherein, the block weight can be the weight corresponding to the feature block, used to characterize the correlation of pixels in the feature block. The block weight can be the nearest value of each corresponding value of the feature block in the evaluation matrix.

[0055] For example, if a feature block contains 4 pixels, and the corresponding values ​​of the 4 pixels in the evaluation matrix are 0.76, 0.77, 0.76, and 0.75, then the weight of the feature block can be determined as 0.8. In the feature block weight map, the weight of the corresponding feature block is 0.8.

[0056] S105. Based on the feature block weight map, determine whether the image to be identified is a forged image.

[0057] In this embodiment, the forged image can be a processed or altered image, such as a spliced ​​image, a copied and pasted image, or an image that has been modified by image editing software (Adobe Photoshop).

[0058] Optionally, based on the feature block weight map, determine whether the image to be identified is a forged image. Determining whether the image to be identified is a forged image based on the feature block weight map includes: if there is a block weight in the feature block weight map that is less than a weight threshold, then determine that the image to be identified is a forged image.

[0059] It should be noted that the weight threshold can be set independently by technical personnel based on actual needs and practical experience, and this invention does not impose any restrictions on it.

[0060] Understandably, by adopting the above technical solution, forgery recognition is performed on the image to be recognized based on the weight threshold, which improves the accuracy of forgery recognition, eliminates the need for manual forgery recognition, and improves the efficiency of forgery recognition.

[0061] This invention provides an embodiment that extracts features from an image to be identified, obtaining image features; performs dimensionality reduction on these image features to obtain forensic features; determines the Laplacian matrix of the forensic features; and based on the Laplacian matrix, determines the evaluation matrix of the forensic features and at least one feature block corresponding to the forensic features; determines the feature block weight map of the forensic features based on at least one feature block and the evaluation matrix; and determines whether the image to be identified is a forged image based on the feature block weight map. By employing the above technical solution, and by determining the evaluation matrix and feature blocks of the forensic features of the image to be identified, and determining the feature block weight map, the image to be identified is identified. This reduces the complexity of the image recognition process and improves the accuracy and robustness of forged image identification.

[0062] Example 2

[0063] Figure 2A This is a flowchart of a forged image recognition method provided in Embodiment 2 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the image feature extraction operation of the image to be recognized.

[0064] Furthermore, the process of "extracting features from the image to be identified to obtain image features of the image to be identified" is refined to "using a VGG network to extract features from the image to be identified to obtain at least two convolutional features; using at least two convolutional features as image features of the image to be identified", in order to improve the operation of obtaining image features of the image to be identified.

[0065] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.

[0066] like Figure 2A As shown, the method includes:

[0067] S201. Use the VGG (Visual Geometry Group) network to extract features from the image to be recognized, and obtain at least two convolutional features.

[0068] In this embodiment, the convolutional features can be feature maps of the image to be identified output by the VGG network, which can be represented in matrix or vector form.

[0069] Figure 2B This is a schematic diagram of a VGG network structure. (Example) Figure 2B As shown, the VGG network consists of five blocks. The first and second blocks each contain two convolutional layers and one pooling layer; the third, fourth, and fifth blocks each contain three convolutional layers and one pooling layer. The convolutional layers extract features from the image, and the pooling layers resize the image. Specifically, the height and width of the feature map output by the convolutional layer are obtained by subtracting the corresponding kernel size from the height and width of the input image, adding twice the zero-padding size, dividing by the kernel stride, and adding one. The dimension of the feature map output by the convolutional layer is the number of convolutional kernels. For example, the height, width, and dimension of the feature map output by the convolutional layer can be determined using the following formula:

[0070] ;

[0071] ;

[0072] ;

[0073] Where H1 is the height of the feature map output by the convolutional layer; H is the height of the image input to the convolutional layer; F1 is the kernel size of the convolutional layer; P is the zero-padding size; S1 is the kernel stride of the convolutional layer; W1 is the width of the feature map output by the convolutional layer; W is the width of the image input to the convolutional layer; D1 is the dimension of the feature map output by the convolutional layer; and K is the number of kernels in the convolutional layer.

[0074] Subtract the corresponding convolution kernel size from the height and width of the input pooling layer image, divide by the convolution kernel stride, and add one to obtain the height and width of the feature map output by the pooling layer; the dimension of the feature map output by the pooling layer remains unchanged. For example, the height, width, and dimension of the feature map output by the pooling layer can be determined using the following formula:

[0075] ;

[0076] ;

[0077] ;

[0078] Where H3 is the height of the feature map output by the pooling layer; H2 is the height of the image input to the pooling layer; F2 is the kernel size of the pooling layer; S2 is the stride of the kernel of the pooling layer; W3 is the width of the feature map output by the pooling layer; W2 is the width of the image input to the pooling layer; D3 is the dimension of the feature map output by the pooling layer; and D2 is the dimension of the image input to the pooling layer.

[0079] Figure 2B In the VGG network, each convolutional layer uses a 3x3 kernel with a stride of 1, and the pooling layer uses a 2x2 kernel with a stride of 2, employing max pooling. The processing operations of each block are similar. Taking the first, second, and third blocks of the VGG network as examples, the processing process is illustrated as follows: The input image size is 224x224x3, meaning a height of 224 pixels, a width of 224 pixels, and a dimension of 3. In the first block, there are 64 convolutional kernels. After processing by convolutional layers 1-1 and 1-2, the output first convolutional feature map has a size of 224x224x64; after processing by pooling layer 1, the output first pooling feature map has a size of 112x112x64; the second block… In the first block, there are 128 convolutional kernels. The first pooling feature map is processed by convolutional layers 2-1 and 2-2, resulting in a second convolutional feature map with a size of 112*112*128. After processing by pooling layer 2, the output second pooling feature map has a size of 56*56*128. In the third block, there are 256 convolutional kernels. The second pooling feature map is processed by convolutional layers 3-1, 3-2, and 3-3, resulting in a third convolutional feature map with a size of 56*56*256. After processing by pooling layer 3, the output third pooling feature map has a size of 28*28*256. The fourth block has 512 convolutional kernels, and the fifth layer has 4096 convolutional kernels. The processing of the fourth and fifth blocks is similar to that of the third block, and will not be repeated here.

[0080] Specifically, feature extraction of the image to be recognized is performed using the VGG network. At least two feature maps output by the VGG network are used as the corresponding convolutional features. It should be noted that any feature map output from the VGG network can be used as the convolutional feature; this invention is not limited in this regard. Preferably, two feature maps output from the second block of the VGG network are used as the convolutional features of the image to be recognized to reduce the complexity of convolutional feature extraction, retain more image details, and improve the feature refinement of the convolutional features.

[0081] In one specific embodiment, before performing feature extraction on the image to be recognized, the size of the image to be recognized is proportionally adjusted to 224*224*3, that is, the height is adjusted to 224 pixels, the width is adjusted to 224 pixels, and the dimension is adjusted to 3. The parts of the adjusted image to be recognized that are less than 224*224*3 are filled with black borders to facilitate feature extraction using the VGG network.

[0082] S202. Use at least two convolutional features as image features of the image to be identified.

[0083] S203. Perform dimensionality reduction processing on the image features to obtain the evidence-gathering features of the image to be identified.

[0084] Optionally, a certain dimensionality reduction method can be used to process the image features to obtain the evidence-gathering features of the image to be identified.

[0085] S204. Determine the Laplace matrix of the evidence-gathering features, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering features and at least one feature block corresponding to the evidence-gathering features.

[0086] S205. Based on at least one feature block and the evaluation matrix, determine the feature block weight map of the evidence collection features.

[0087] S206. Based on the feature block weight map, determine whether the image to be identified is a forged image.

[0088] The technical solution of this invention employs a VGG network to extract features from the image to be identified, obtaining at least two convolutional features. These at least two convolutional features are then used as image features of the image to be identified. The image features are then subjected to dimensionality reduction processing to obtain forensic features of the image to be identified. The Laplacian matrix of the forensic features is determined, and based on the Laplacian matrix, an evaluation matrix and at least one feature block corresponding to the forensic features are determined. Based on the at least one feature block and the evaluation matrix, a feature block weight map of the forensic features is determined. Based on the feature block weight map, it is determined whether the image to be identified is a forged image. This technical solution reduces the complexity of feature extraction from the image to be identified, retains a large amount of image detail in the image to be identified, improves the fineness of the image features, and thus improves the accuracy and robustness of forged image identification of the image to be identified.

[0089] Example 3

[0090] Figure 3A This is a flowchart of a forged image recognition method provided in Embodiment 3 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the operation of determining the forensic features of the image to be recognized.

[0091] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.

[0092] like Figure 3A As shown, the method includes:

[0093] S301. Using a VGG network, feature extraction is performed on the image to be recognized to obtain at least two convolutional features.

[0094] S302. Use at least two convolutional features as image features of the image to be identified.

[0095] S303. Perform dimensionality reduction processing on the image features to obtain the evidence-gathering features of the image to be identified.

[0096] Optionally, the image features are subjected to dimensionality reduction processing to obtain the evidence-gathering features of the image to be identified, including: performing the same convolution operation on at least two convolution features respectively to obtain at least two processed convolution features; merging the at least two processed convolution features to obtain merged convolution features; performing a convolution operation on the merged convolution features to obtain processed merged convolution features; and processing the processed merged convolution features using a convolutional neural network to obtain the evidence-gathering features of the image to be identified.

[0097] Among them, the merged convolutional feature can be a convolutional feature formed by merging at least two convolutional features, and can be represented in matrix or vector form.

[0098] Specifically, the same convolutional layer is used to perform the same convolution operation on at least two convolutional features to obtain at least two processed convolutional features; activation operations are performed on the at least two processed convolutional features, and the activated at least two processed convolutional features are added together to obtain merged convolutional features; activation operations are performed on the merged convolutional features, and they are processed using a convolutional neural network to obtain the evidence-gathering features of the image to be identified; the evidence-gathering features are then normalized. It should be noted that the activation method and the convolutional neural network can be at least one of the existing technologies, and this invention does not limit them.

[0099] In one specific embodiment, at least two convolutional features can be feature maps of size 56*56*128 from the second output of the VGG network; the convolutional layer can be a 1*1*128 convolutional layer; the activation operation can be performed using the ReLU function (Linear Rectification function); the normalization operation can be performed using the softmax function, resulting in a dimension of 128 pixels * 128 pixels for the obtained evidence features.

[0100] Figure 3B This is a schematic diagram illustrating the process of obtaining forensic characteristics. For example... Figure 3BAs shown, this includes convolutional feature 1 and convolutional feature 2. Using the same convolutional layer, convolutional features are performed to obtain processed convolutional feature 1 and processed convolutional feature 2. The ReLU function is then used to activate the convolutional features, resulting in activated processed convolutional feature 1 and activated processed convolutional feature 2. The activated processed convolutional feature 1 and activated processed convolutional feature 2 are added together to obtain a merged convolutional feature. A convolutional layer that processes the convolutional features is then used to perform a convolutional operation on the merged convolutional feature, resulting in a processed merged convolutional feature. The merged convolutional feature is then activated, resulting in an activated merged convolutional feature. Finally, a convolutional neural network is used to process the activated merged convolutional feature to obtain the evidence-gathering feature.

[0101] It is understandable that by adopting the above technical solution, performing the same convolution operation on at least two convolutional features respectively, and merging the at least two processed convolutional features into a merged convolutional feature; performing a convolution operation on the merged convolutional feature, and using a convolutional neural network to process the processed merged convolutional feature to obtain the evidence features of the image to be identified, the complexity of obtaining the evidence features of the image to be identified can be reduced, and a large number of image details in the image to be identified are preserved in the evidence features, improving the fineness of the evidence features, thereby improving the accuracy and robustness of the forgery image identification of the image to be identified.

[0102] S304. Determine the Laplace matrix of the evidence-gathering features, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering features and at least one feature block corresponding to the evidence-gathering features.

[0103] S305. Based on at least one feature block and the evaluation matrix, determine the feature block weight map of the evidence collection features.

[0104] S306. Based on the feature block weight map, determine whether the image to be identified is a forged image.

[0105] The technical solution of this invention employs a VGG network to extract features from the image to be identified, obtaining at least two convolutional features. These at least two convolutional features are then used as image features of the image to be identified. The image features are then subjected to dimensionality reduction processing to obtain forensic features of the image to be identified. The Laplacian matrix of the forensic features is determined, and based on the Laplacian matrix, an evaluation matrix and at least one feature block corresponding to the forensic features are determined. Based on the at least one feature block and the evaluation matrix, a feature block weight map of the forensic features is determined. Based on the feature block weight map, it is determined whether the image to be identified is a forged image. This technical solution reduces the complexity of obtaining forensic features from the image to be identified and retains a large amount of image detail in the forensic features, improving the finesse of the forensic features and thus enhancing the accuracy and robustness of forged image identification of the image to be identified.

[0106] Example 4

[0107] Figure 4 This is a flowchart of a forged image recognition method provided in Embodiment 4 of the present invention. Based on the above embodiments, this embodiment optimizes and improves the evaluation matrix of the evidence collection features and the determination operation of at least one feature block corresponding to the evidence collection features.

[0108] Furthermore, the process of "determining the evaluation matrix of the evidence-gathering feature and at least one feature block corresponding to the evidence-gathering feature based on the Laplace matrix" is refined into "determining at least one eigenvalue of the Laplace matrix and the eigenvector corresponding to the eigenvalue; determining the evaluation matrix of the evidence-gathering feature based on the at least one eigenvalue and the eigenvector corresponding to the eigenvalue; and determining at least one feature block corresponding to the evidence-gathering feature based on the eigenvector corresponding to the eigenvalue," thereby improving the operation of determining the evaluation matrix of the evidence-gathering feature and at least one feature block corresponding to the evidence-gathering feature.

[0109] It should be noted that for any parts not described in detail in the embodiments of the present invention, please refer to the description in the foregoing embodiments.

[0110] like Figure 4 As shown, the method includes:

[0111] S401. Extract features from the image to be recognized to obtain the image features of the image to be recognized.

[0112] S402. Perform dimensionality reduction processing on the image features to obtain the evidence-gathering features of the image to be identified.

[0113] S403. Determine at least one eigenvalue of the Laplacian matrix and the corresponding eigenvector.

[0114] In this embodiment, the Laplace matrix is ​​a normalized Laplace matrix.

[0115] Specifically, based on the order of the Laplace matrix, at least one eigenvalue of the Laplace matrix is ​​determined. Based on the at least one eigenvalue of the Laplace matrix, the corresponding eigenvector is determined. It should be noted that the eigenvalues ​​and eigenvectors of the Laplace matrix can be determined using at least one method from existing techniques, and this invention does not limit this method.

[0116] S404. Determine the evaluation matrix of the evidence-gathering features based on at least one eigenvalue and the corresponding eigenvector.

[0117] Optionally, based on the feature vector corresponding to the feature value, at least one feature block corresponding to the evidence feature is determined, including: subtracting the feature vector corresponding to the feature value from the feature value to obtain the difference result; and generating an evaluation matrix of the evidence feature based on the difference result corresponding to at least one feature value.

[0118] The difference result can be the feature vector obtained by subtracting the corresponding feature value from each element in the feature vector.

[0119] Specifically, the difference between each element in the eigenvector corresponding to the eigenvalue and the eigenvalue is calculated to obtain the difference result; the matrix composed of the eigenvectors corresponding to each eigenvalue is used as the evaluation matrix for evidence features.

[0120] In one specific embodiment, if the Laplacian matrix is ​​an N*N square matrix, where N is any non-zero natural number, take N-dimensional eigenvectors corresponding to the N eigenvalues; subtract each element of the N eigenvectors from its corresponding eigenvalue to obtain N difference results; and use the N*N matrix formed by the N difference results as the evaluation matrix. It should be noted that the dimension of the evaluation matrix is ​​the same as the dimension of the evidence features.

[0121] It is understandable that by adopting the above technical solution, an evaluation matrix of evidence-gathering features is generated based on the difference between each feature vector and its corresponding feature value. This improves the accuracy of the correlation between points in the evidence-gathering features represented by the element values ​​of the evaluation matrix, and further improves the accuracy of the feature block weight map of the evidence-gathering features determined based on the feature blocks and the evaluation matrix.

[0122] S405. Based on the feature vector corresponding to the feature value, determine at least one feature block corresponding to the evidence feature.

[0123] Optionally, based on the feature vectors corresponding to the feature values, at least one feature block corresponding to the evidence-gathering feature is determined, including: clustering at least one feature vector to obtain a clustering result; and segmenting the evidence-gathering feature based on the clustering result to obtain at least one feature block.

[0124] The clustering results can characterize the category to which each point in the evidence features belongs.

[0125] Specifically, at least one feature vector is arranged into a clustering matrix, and each row of the clustering matrix is ​​clustered. Based on the category to which each row belongs in the clustering matrix, the category of each pixel in the evidence feature is determined; the region consisting of points of the same category in the evidence feature is divided into a feature block.

[0126] It should be noted that the clustering algorithm can be at least one of the existing technologies, and this invention does not limit it. For example, the clustering algorithm can be the K-Means algorithm.

[0127] Understandably, by adopting the above technical solution, the evidence features are segmented according to the clustering results to obtain at least one feature block, which improves the accuracy of the feature block, thereby improving the accuracy of the feature weight map, and improving the accuracy of forged image recognition of the image to be identified based on the feature block weight map.

[0128] S406. Based on at least one feature block and the evaluation matrix, determine the feature block weight map of the evidence collection features.

[0129] S407. Based on the feature block weight map, determine whether the image to be identified is a forged image.

[0130] The technical solution of this invention involves determining at least one eigenvalue of a Laplacian matrix and its corresponding eigenvector; determining an evaluation matrix for the evidence-gathering feature based on the at least one eigenvalue and its corresponding eigenvector; and determining at least one feature block corresponding to the evidence-gathering feature based on the eigenvector. By employing this technical solution, an evaluation matrix for the corresponding evidence-gathering feature is determined based on the eigenvalue and eigenvector of the Laplacian matrix, and at least one feature block is determined based on the evaluation matrix. This technical solution improves the accuracy of the feature blocks by determining the evaluation matrix and thus the accuracy of the feature weight map, thereby improving the accuracy of forged image recognition of the image to be identified based on the feature block weight map.

[0131] Example 5

[0132] Figure 5 This is a schematic diagram of a forged image recognition device according to Embodiment 5 of the present invention. This embodiment is applicable to situations where a forged image is identified from an image to be recognized. The forged image device can be implemented in hardware and / or software and can be configured in an electronic device.

[0133] like Figure 5 As shown, the device includes: an image feature determination module 501, an evidence feature determination module 502, an evaluation matrix determination module 503, a weight map determination module 504, and a forged image recognition module 505, wherein...

[0134] The image feature determination module 501 is used to extract features from the image to be identified, thereby obtaining the image features of the image to be identified.

[0135] The evidence feature determination module 502 is used to perform dimensionality reduction processing on image features to obtain the evidence features of the image to be identified;

[0136] The evaluation matrix determination module 503 is used to determine the Laplace matrix of the evidence-gathering feature, and based on the Laplace matrix, determine the evaluation matrix of the evidence-gathering feature and at least one feature block corresponding to the evidence-gathering feature.

[0137] The weight map determination module 504 is used to determine the feature block weight map of the evidence-gathering features based on at least one feature block and an evaluation matrix.

[0138] The forged image recognition module 505 is used to determine whether the image to be recognized is a forged image based on the feature block weight map.

[0139] This invention employs an image feature determination module to extract features from the image to be identified, obtaining image features of the image to be identified; an evidence feature determination module performs dimensionality reduction processing on the image features to obtain evidence features of the image to be identified; an evaluation matrix determination module determines the Laplacian matrix of the evidence features, and based on the Laplacian matrix, determines the evaluation matrix of the evidence features and at least one feature block corresponding to the evidence features; a weight map determination module determines the feature block weight map of the evidence features based on at least one feature block and the evaluation matrix; and a forged image identification module determines whether the image to be identified is a forged image based on the feature block weight map. The technical solution of this invention, by determining the evaluation matrix and feature blocks of the evidence features of the image to be identified, and determining the feature block weight map, identifies the image to be identified. The image recognition process has low complexity, improving the accuracy and robustness of forged image identification.

[0140] Optionally, the image feature determination module 501 includes:

[0141] The convolutional feature acquisition unit is used to extract features from the image to be recognized using the VGG network, and obtain at least two convolutional features.

[0142] An image feature determination unit is used to use at least two convolutional features as image features of the image to be identified.

[0143] Optionally, the forensic feature determination module 502 includes:

[0144] The processed feature acquisition unit is used to perform the same convolution operation on at least two convolutional features respectively to obtain at least two processed convolutional features;

[0145] The feature merging unit is used to merge at least two processed convolutional features to obtain merged convolutional features;

[0146] The processed merged feature acquisition unit is used to perform convolution operations on the merged convolutional features to obtain the processed merged convolutional features.

[0147] The evidence feature acquisition unit is used to process the merged convolutional features using a convolutional neural network to obtain the evidence features of the image to be identified.

[0148] Optionally, the evaluation matrix determination module 503 includes:

[0149] An eigenvalue determination unit is used to determine at least one eigenvalue of the Laplacian matrix and the corresponding eigenvector.

[0150] The evaluation matrix determination unit is used to determine the evaluation matrix of the evidence-gathering features based on at least one feature value and the feature vector corresponding to the feature value.

[0151] The feature block determination unit is used to determine at least one feature block corresponding to the evidence-gathering feature based on the feature vector corresponding to the feature value.

[0152] Optionally, the evaluation matrix determination unit includes:

[0153] The difference result determination sub-unit is used to subtract the eigenvector corresponding to the eigenvalue from the eigenvalue to obtain the difference result;

[0154] The evaluation matrix generation sub-unit is used to generate an evaluation matrix of evidence-gathering features based on the difference results corresponding to at least one feature value.

[0155] Optionally, the forged image recognition module 505 includes:

[0156] The forged image recognition unit is used to determine that the image to be recognized is a forged image if the weight of a block in the feature block weight map is less than the weight threshold.

[0157] The forged image recognition device provided in the embodiments of the present invention can execute the forged image recognition method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the forged image recognition method.

[0158] Example 6

[0159] Figure 6 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0160] like Figure 6 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0161] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0162] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as forgery image recognition methods.

[0163] In some embodiments, the forgery image recognition method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the forgery image recognition method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the forgery image recognition method by any other suitable means (e.g., by means of firmware).

[0164] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0165] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0166] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0167] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0168] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0169] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0170] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0171] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for identifying forged images, characterized in that, include: Feature extraction is performed on the image to be identified to obtain the image features of the image to be identified; The image features are subjected to dimensionality reduction processing to obtain the forensic features of the image to be identified; Determine the Laplace matrix of the evidence-gathering features; Determine at least one eigenvalue of the Laplacian matrix and the eigenvector corresponding to the eigenvalue; The evaluation matrix of the evidence-gathering feature is determined based on the at least one feature value and the feature vector corresponding to the feature value; Based on the feature vector corresponding to the feature value, at least one feature block corresponding to the evidence collection feature is determined; the feature block is a block composed of pixels with the same or similar features in the evidence collection feature; Based on the at least one feature block and the evaluation matrix, determine the feature block weight map of the evidence-gathering feature; Based on the feature block weight map, it is determined whether the image to be identified is a forged image.

2. The method according to claim 1, characterized in that, The step of extracting features from the image to be identified to obtain the image features of the image to be identified includes: The VGG network is used to extract features from the image to be identified, resulting in at least two convolutional features; The at least two convolutional features are used as image features of the image to be identified.

3. The method according to claim 2, characterized in that, The step of performing dimensionality reduction processing on the image features to obtain the forensic features of the image to be identified includes: Perform the same convolution operation on each of the at least two convolutional features to obtain at least two processed convolutional features; At least two processed convolutional features are merged to obtain merged convolutional features; Perform a convolution operation on the merged convolution features to obtain the processed merged convolution features; The processed merged convolutional features are processed using a convolutional neural network to obtain the evidence-gathering features of the image to be identified.

4. The method according to claim 1, characterized in that, The step of determining the evaluation matrix of the evidence-gathering features based on the at least one feature value and the feature vector corresponding to the feature value includes: The difference between the eigenvector corresponding to the eigenvalue and the eigenvalue is obtained by subtracting the eigenvalue; An evaluation matrix for the evidence-gathering features is generated based on the difference results corresponding to at least one feature value.

5. The method according to claim 1, characterized in that, The step of determining at least one feature block corresponding to the evidence-gathering feature based on the feature vector corresponding to the feature value includes: Cluster at least one feature vector to obtain the clustering result; Based on the clustering results, the evidence-gathering features are segmented to obtain at least one feature block.

6. The method according to claim 1, characterized in that, The step of determining whether the image to be identified is a forged image based on the feature block weight map includes: If there is a block weight in the feature block weight map that is less than the weight threshold, then the image to be identified is determined to be a fake image.

7. A forged image recognition device, characterized in that, include: The image feature determination module is used to extract features from the image to be identified, thereby obtaining the image features of the image to be identified. The evidence feature determination module is used to perform dimensionality reduction processing on the image features to obtain the evidence features of the image to be identified; An evaluation matrix determination module is used to determine the Laplacian matrix of the evidence-gathering feature, and based on the Laplacian matrix, determine the evaluation matrix of the evidence-gathering feature and at least one feature block corresponding to the evidence-gathering feature; The weight map determination module is used to determine the feature block weight map of the evidence-gathering feature based on the at least one feature block and the evaluation matrix. The forged image recognition module is used to determine whether the image to be recognized is a forged image based on the feature block weight map. The evaluation matrix determination module includes: An eigenvalue determination unit is used to determine at least one eigenvalue of the Laplacian matrix and the corresponding eigenvector. The evaluation matrix determination unit is used to determine the evaluation matrix of the evidence-gathering features based on at least one feature value and the feature vector corresponding to the feature value. The feature block determination unit is used to determine at least one feature block corresponding to the evidence feature based on the feature vector corresponding to the feature value; the feature block is a block composed of pixels with the same or similar features in the evidence feature.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the forgery image recognition method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for identifying forged images as described in any one of claims 1-6.

Images