Multi-domain joint-driven remote sensing data tampering detection and evaluation method

Abstract

The invention belongs to the field of remote sensing images, and relates to a multi-domain joint-driven remote sensing data tampering detection and evaluation method which is used for supporting detection and evaluation of tampering phenomena in remote sensing images. The method comprises the following steps: constructing a multi-domain authenticity reference portrait covering four-dimensional characteristics of an attribute domain, a digital domain, a physical domain and a semantic domain of a real remote sensing image, designing a multi-domain joint evaluation method, and calculating a joint evaluation approximation degree between an individual description portrait of a to-be-detected remote sensing image and the authenticity reference portrait; and authenticity evaluation and discrimination of the remote sensing image in multiple dimensions are realized.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing images, and relates to a multi-domain joint-driven remote sensing data tampering detection and evaluation method for supporting detection and evaluation of tampering phenomena in remote sensing images. Background Art

[0002] At present, there are a large number of tampered remote sensing images on the Internet. There are various methods of tampering and forgery, including traditional tampering and forgery methods such as splicing, copy-paste, and removal, as well as deep tampering and forgery methods. The tampering and forgery scenes cover different types such as airports and ports, which seriously affect the authenticity and reliability of Internet remote sensing image information, and further affect public opinion's correct understanding of remote sensing images.

[0003] In recent years, many researchers have studied the task of remote sensing image tampering detection and proposed a series of remote sensing image tampering detection methods based on deep neural networks. Related research is often limited to detecting tampering phenomena in a single domain, such as attribute domain, digital domain, physical domain, and semantic domain. However, the detection method in a single domain can only be effective when there is obvious tampering in the domain, and it is not possible to effectively complete the tampering detection of remote sensing images. Therefore, it is urgent to realize a multi-domain joint remote sensing image tampering detection and evaluation method. Summary of the invention

[0004] The technical problem to be solved by the present invention is that the current single-field detection cannot effectively complete the tampering detection and evaluation of remote sensing images. A multi-domain joint-driven remote sensing data tampering detection and evaluation method is proposed. By constructing a multi-domain authenticity benchmark portrait covering the four-dimensional characteristics of the real remote sensing image in the attribute domain, digital domain, physical domain, and semantic domain, a multi-domain joint evaluation method is designed, and the joint evaluation approximation between the individual description portrait of the remote sensing image to be detected and the authenticity benchmark portrait is calculated, so as to realize the authenticity evaluation and discrimination of the remote sensing image in multiple dimensions.

[0005] The technical solution adopted by the present invention is:

[0006] A multi-domain joint-driven remote sensing data tampering detection and evaluation method includes the following steps:

[0007] (1) Construct a multi-domain true benchmark image of a real remote sensing image. The multi-domain true benchmark image is stored in the form of a knowledge graph, including attribute domain benchmark image, digital domain benchmark image, physical domain benchmark image, and semantic domain benchmark image. The true benchmark image of each domain is connected to an entity node named after the real remote sensing image, so that the multi-domain true benchmark image is a whole in the knowledge graph.

[0008] (2) Extracting multi-domain individual description portraits of the remote sensing image to be detected. The multi-domain individual description portraits are stored in the form of knowledge graphs, including attribute domain individual description portraits, digital domain individual description portraits, physical domain individual description portraits, and semantic domain individual description portraits. The individual description portraits of each domain are connected to the entity node named after the remote sensing image to be detected, so that the multi-domain individual description portraits are a whole in the knowledge graph.

[0009] (3) The extracted multi-domain individual description portrait and the multi-domain true benchmark portrait are calculated using a multi-domain joint evaluation method to calculate the joint evaluation approximation; and a tampering judgment threshold M is set. When the calculated joint evaluation approximation is less than M, it is determined that the remote sensing image to be detected has been tampered with.

[0010] Furthermore, in step (1):

[0011] The attribute domain benchmark image describes the basic parameters of the real remote sensing image, including image source, sensor type, data level, resolution, pixel size, geographic attributes and compression rate; the parameter name is stored in the atlas relationship, and the remote sensing image name and parameter content are stored in the atlas entity;

[0012] The digital domain benchmark image describes the consistency characteristics of digital signals generated and retained by real remote sensing images during production, dissemination and storage, including noise distribution difference characteristics, frequency energy distribution difference characteristics and gradient change difference characteristics; the feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities;

[0013] The physical domain benchmark image describes the physical imaging regularity characteristics commonly found in real remote sensing images, including grayscale intensity characteristics, shadow direction characteristics, color interval characteristics, noise interval characteristics, and boundary artifact characteristics. The feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities.

[0014] The semantic domain benchmark portrait describes the semantic logical relationship between scenes and targets and between targets in real remote sensing images, including docking, deployment and appearance; the scenes and targets are stored as graph entities, and the semantic logical relationship is stored as graph relationship.

[0015] Furthermore, in step (2):

[0016] The individual description portraits in the attribute domain are extracted from the metadata file of the remote sensing image, and the parameter contents of the parameter names such as image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate are obtained by text parsing. The parameter names are stored in the atlas relationship, and the remote sensing image name and parameter contents are stored in the atlas entity.

[0017] The digital domain individual description portrait uses LEE filtering, Mel cepstral coefficients and co-occurrence matrix to extract noise distribution difference features, frequency energy distribution difference features and gradient change difference features from the remote sensing image to be detected, and stores the feature type as the image's graph relationship, and the feature extraction result is stored as the image's graph entity;

[0018] The physical domain individual description portrait uses color histogram, binary segmentation, noise estimation method and two-stream R-CNN network model to extract grayscale intensity, shadow direction, color range, noise range and boundary artifacts of the remote sensing image to be detected; the feature name is stored in the atlas relationship, and the remote sensing image name and feature parameters are stored in the atlas entity;

[0019] The individual description portrait in the semantic domain extracts the scenes and targets appearing in the remote sensing image to be detected, as well as the semantic logical relationships between targets, through image semantic understanding, target detection and object classification methods, including target positions, target models, relative relationships and changes; the scenes and targets are stored as graph entities, and the semantic logical relationships are stored as graph relationships.

[0020] Furthermore, the joint evaluation approximation in step (3) is calculated by the weighted sum of the domain approximation and domain coverage of each domain. The calculation process of the domain approximation and domain coverage is:

[0021] The method for calculating the attribute domain similarity is: using transR to calculate the embedded vectors of the attribute domain individual description portrait and the attribute domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the attribute domain individual description portrait and the embedded vector of the attribute domain benchmark portrait as the attribute domain similarity; the method for calculating the digital domain similarity is: using transR to calculate the embedded vectors of the digital domain individual description portrait and the digital domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the digital domain individual description portrait and the embedded vector of the digital domain benchmark portrait as the digital domain similarity; the method for calculating the physical domain similarity is: using transR to calculate the embedded vectors of the physical domain individual description portrait and the physical domain real benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the physical domain individual description portrait and the embedded vector of the physical domain benchmark portrait as the physical domain similarity; the method for calculating the semantic domain similarity is: using transR to calculate the embedded vectors of the semantic domain individual description portrait and the semantic domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the semantic domain individual description portrait and the embedded vector of the semantic domain benchmark portrait as the semantic domain similarity;

[0022] The calculation method of domain coverage is as follows: traverse all local sub-portraits in the multi-domain real benchmark portrait, and use transR to calculate the embedded vectors of all local sub-portraits in the multi-domain real benchmark portrait; use transR to calculate the embedded vector of the multi-domain individual description portrait, and calculate the local sub-portrait of the multi-domain real benchmark portrait with the largest approximation to the embedded vector of the multi-domain individual description portrait, and name it the maximum circumscribed local sub-portrait; calculate the intersection ratio of the maximum circumscribed local sub-portrait with the attribute domain benchmark portrait, digital domain benchmark portrait, physical domain benchmark portrait and semantic domain benchmark portrait, which is the domain coverage of each domain.

[0023] Compared with the background technology, the present invention has the following advantages:

[0024] The present invention proposes a multi-domain joint-driven remote sensing data tampering detection and evaluation method. By modeling benchmark portraits and individual description portraits in different domains, and using a multi-domain joint evaluation method to calculate the joint proximity as a benchmark for whether the remote sensing image to be detected has been tampered with, the problem of existing tampering detection methods that only detect from a single domain dimension, consider a single type of feature dimension, and are not fully adapted to tampering phenomena is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is the overall process framework design diagram of the present invention. DETAILED DESCRIPTION

[0026] The specific implementation of the present invention is described below in conjunction with the accompanying drawings so that those skilled in the art can better understand the present invention. It should be noted that in the following description, when the detailed description of known functions and designs may dilute the main content of the present invention, these descriptions will be omitted here.

[0027] Figure 1 It is a principle flow chart of a specific implementation of the multi-domain joint driven remote sensing image tampering detection and evaluation method of the present invention.

[0028] In this embodiment, if Figure 1 The multi-domain joint driven remote sensing data tampering detection and evaluation method shown includes the following steps:

[0029] (1) Construct a multi-domain true benchmark image of a real remote sensing image. The multi-domain true benchmark image is stored in the form of a knowledge graph, including attribute domain benchmark image, digital domain benchmark image, physical domain benchmark image, and semantic domain benchmark image. The true benchmark image of each domain is connected to an entity node named after the real remote sensing image, so that the multi-domain true benchmark image is a whole in the knowledge graph. Among them:

[0030] The attribute domain benchmark image describes the basic parameters of real remote sensing images, including image source, sensor type, data level, resolution, pixel size, geographic attributes and compression rate; the parameter names are stored in atlas relationships, and the remote sensing image names and parameter contents are stored in atlas entities; for example, the "Gaofen-2 Satellite Remote Sensing Image" entity is connected to the "1m" entity through the "resolution" relationship.

[0031] The digital domain benchmark portrait describes the consistency characteristics of digital signals generated and retained by real remote sensing images during the production, dissemination and storage process, including noise distribution difference characteristics, frequency energy distribution difference characteristics and gradient change difference characteristics; the feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities; for example, the "optical remote sensing image" entity is connected to the "0.32" entity through the "noise distribution difference characteristic" relationship.

[0032] The physical domain benchmark portrait describes the physical imaging regularity characteristics commonly found in real remote sensing images, including grayscale intensity characteristics, shadow direction characteristics, color range characteristics, noise range characteristics, and boundary artifact characteristics. The feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities. For example, the "Gaofen-2 Satellite Remote Sensing Image" entity is connected to the "1db" entity through the "noise range" relationship.

[0033] The semantic domain benchmark portrait describes the semantic logical relationship between scenes and targets and between targets in real remote sensing images, including docking, deployment and appearance; the scenes and targets are stored as graph entities, and the semantic logical relationships are stored as graph relationships; for example, the "aircraft carrier" entity is connected to the "port" base through the "docked" relationship.

[0034] (2) Extract multi-domain individual description portraits of the remote sensing image to be detected. The multi-domain individual description portraits are stored in the form of knowledge graph, including attribute domain individual description portraits, digital domain individual description portraits, physical domain individual description portraits, and semantic domain individual description portraits; the individual description portraits of each domain are connected to the entity node named after the remote sensing image to be detected, so that the multi-domain individual description portraits are a whole in the knowledge graph. Among them:

[0035] The individual description portraits in the attribute domain are extracted through the metadata file of the remote sensing image, and the parameter contents of the parameter names such as image source, sensor type, data level, resolution, pixel size, geographic attributes and compression rate are obtained by text parsing. The parameter names are stored in atlas relationships, and the remote sensing image names and parameter contents are stored in atlas entities.

[0036] The digital domain individual description portrait extracts the noise distribution difference features, frequency energy distribution difference features and gradient change difference features of the remote sensing image to be detected through LEE filtering, Mel-frequency cepstral coefficients and co-occurrence matrix, stores the feature type as the spectral relationship of the portrait, and stores the feature extraction result as the spectral entity of the portrait.

[0037] The physical domain individual description portrait extracts the grayscale intensity, shadow direction, color range, noise range and boundary artifacts of the remote sensing image to be detected through color histogram, binary segmentation, noise estimation method and two-stream R-CNN network model; the feature name is stored in the atlas relationship, and the remote sensing image name and feature parameters are stored in the atlas entity.

[0038] The individual description portrait in the semantic domain extracts the scenes and targets appearing in the remote sensing image to be detected, as well as the semantic logical relationships between targets, through image semantic understanding, target detection and object classification methods, including target positions, target models, relative relationships and changes; the scenes and targets are stored as graph entities, and the semantic logical relationships are stored as graph relationships.

[0039] (3) The extracted multi-domain individual description portrait and the multi-domain true benchmark portrait are calculated using a multi-domain joint evaluation method to calculate the joint evaluation approximation; and a tampering judgment threshold M is set. When the calculated joint evaluation approximation is less than M, it is determined that the remote sensing image to be detected has been tampered with.

[0040] The joint evaluation approximation is calculated by the weighted sum of the domain approximation and domain coverage of each domain. The calculation process of domain approximation and domain coverage is as follows:

[0041] The method for calculating the attribute domain similarity is: using transR to calculate the embedded vectors of the attribute domain individual description portrait and the attribute domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the attribute domain individual description portrait and the embedded vector of the attribute domain benchmark portrait as the attribute domain similarity; the method for calculating the digital domain similarity is: using transR to calculate the embedded vectors of the digital domain individual description portrait and the digital domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the digital domain individual description portrait and the embedded vector of the digital domain benchmark portrait as the digital domain similarity; the method for calculating the physical domain similarity is: using transR to calculate the embedded vectors of the physical domain individual description portrait and the physical domain real benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the physical domain individual description portrait and the embedded vector of the physical domain benchmark portrait as the physical domain similarity; the method for calculating the semantic domain similarity is: using transR to calculate the embedded vectors of the semantic domain individual description portrait and the semantic domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the semantic domain individual description portrait and the embedded vector of the semantic domain benchmark portrait as the semantic domain similarity;

[0042] The calculation method of domain coverage is as follows: traverse all local sub-portraits in the multi-domain real benchmark portrait, and use transR to calculate the embedded vectors of all local sub-portraits in the multi-domain real benchmark portrait; use transR to calculate the embedded vector of the multi-domain individual description portrait, and calculate the local sub-portrait of the multi-domain real benchmark portrait with the largest approximation to the embedded vector of the multi-domain individual description portrait, and name it the maximum circumscribed local sub-portrait; calculate the intersection ratio of the maximum circumscribed local sub-portrait with the attribute domain benchmark portrait, digital domain benchmark portrait, physical domain benchmark portrait and semantic domain benchmark portrait, which is the domain coverage of each domain.

[0043] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, it should be clear that the present invention is not limited to the scope of the specific embodiments. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A multi-domain joint-driven remote sensing data tampering detection and evaluation method, characterized in that: The following steps are involved: (1) Construct a multi-domain true benchmark image of a real remote sensing image. The multi-domain true benchmark image is stored in the form of a knowledge graph, including attribute domain benchmark image, digital domain benchmark image, physical domain benchmark image, and semantic domain benchmark image. The real benchmark images of each domain are connected to the entity nodes named after the real remote sensing images, so that the real benchmark images of multiple domains are a whole in the knowledge graph; (2) Extracting multi-domain individual description portraits of the remote sensing image to be detected. The multi-domain individual description portraits are stored in the form of knowledge graphs, including attribute domain individual description portraits, digital domain individual description portraits, physical domain individual description portraits, and semantic domain individual description portraits. The individual description portraits of each domain are connected to the entity node named after the remote sensing image to be detected, so that the multi-domain individual description portraits are a whole in the knowledge graph. (3) The extracted multi-domain individual description portrait and the multi-domain true benchmark portrait are calculated using a multi-domain joint evaluation method to calculate the joint evaluation approximation; and a tampering judgment threshold M is set. When the calculated joint evaluation approximation is less than M, it is determined that the remote sensing image to be detected has been tampered with.

2. According to the multi-domain joint-driven remote sensing data tampering detection and evaluation method of claim 1, it is characterized in that: In step (1): The attribute domain benchmark image describes the basic parameters of the real remote sensing image, including image source, sensor type, data level, resolution, pixel size, geographic attributes and compression rate; the parameter name is stored in the atlas relationship, and the remote sensing image name and parameter content are stored in the atlas entity; The digital domain benchmark image describes the consistency characteristics of digital signals generated and retained by real remote sensing images during production, dissemination and storage, including noise distribution difference characteristics, frequency energy distribution difference characteristics and gradient change difference characteristics; the feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities; The physical domain benchmark image describes the physical imaging regularity characteristics commonly found in real remote sensing images, including grayscale intensity characteristics, shadow direction characteristics, color interval characteristics, noise interval characteristics, and boundary artifact characteristics. The feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities. The semantic domain benchmark portrait describes the semantic logical relationship between scenes and targets and between targets in real remote sensing images, including docking, deployment and appearance; the scenes and targets are stored as graph entities, and the semantic logical relationship is stored as graph relationship.

3. According to the multi-domain joint driven remote sensing data tampering detection and evaluation method of claim 1, it is characterized in that: In step (2): The individual description portraits in the attribute domain are extracted from the metadata file of the remote sensing image, and the parameter contents of the parameter names such as image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate are obtained by text parsing. The parameter names are stored in the atlas relationship, and the remote sensing image name and parameter contents are stored in the atlas entity. The digital domain individual description portrait uses LEE filtering, Mel cepstral coefficients and co-occurrence matrix to extract noise distribution difference features, frequency energy distribution difference features and gradient change difference features from the remote sensing image to be detected, and stores the feature type as the image's graph relationship, and the feature extraction result is stored as the image's graph entity; The physical domain individual description portrait extracts the grayscale intensity, shadow direction, color range, noise range and boundary artifacts of the remote sensing image to be detected through color histogram, binary segmentation, noise estimation method and two-stream R-CNN network model; The feature names are stored using atlas relationships, and the remote sensing image names and feature parameters are stored using atlas entities; The semantic domain individual description portrait extracts the scenes and targets that appear in the remote sensing image to be detected, as well as the semantic logical relationship between targets, including target location, target model, relative relationship and change status, through image semantic understanding, target detection and object classification methods; The scenes and targets are stored using graph entities, and the semantic logical relationships are stored using graph relationships.

4. According to the multi-domain joint driven remote sensing data tampering detection and evaluation method of claim 1, it is characterized in that: In step (3), the joint evaluation approximation is calculated by the weighted sum of the domain approximation and domain coverage of each domain. The calculation process of domain approximation and domain coverage is: The method for calculating the attribute domain similarity is: using transR to calculate the embedded vectors of the attribute domain individual description portrait and the attribute domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the attribute domain individual description portrait and the embedded vector of the attribute domain benchmark portrait as the attribute domain similarity; the method for calculating the digital domain similarity is: using transR to calculate the embedded vectors of the digital domain individual description portrait and the digital domain benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the digital domain individual description portrait and the embedded vector of the digital domain benchmark portrait as the digital domain similarity; the method for calculating the physical domain similarity is: using transR to calculate the embedded vectors of the physical domain individual description portrait and the physical domain real benchmark portrait respectively, and using Euclidean distance to calculate the similarity between the embedded vector of the physical domain individual description portrait and the embedded vector of the physical domain benchmark portrait as the physical domain similarity; The semantic domain similarity calculation method is as follows: transR is used to calculate the embedded vectors of the semantic domain individual description portrait and the semantic domain benchmark portrait respectively, and the Euclidean distance is used to calculate the similarity between the embedded vector of the semantic domain individual description portrait and the embedded vector of the semantic domain benchmark portrait as the semantic domain similarity; The domain coverage is calculated as follows: traverse all local sub-portraits in the multi-domain true benchmark portrait, and use transR to calculate the embedded vectors of all local sub-portraits in the multi-domain true benchmark portrait; use transR to calculate the embedded vector of the multi-domain individual description portrait, and calculate the local sub-portrait of the multi-domain true benchmark portrait with the greatest approximation to the embedded vector of the multi-domain individual description portrait, and name it the maximum circumscribed local sub-portrait; Calculate the intersection ratio of the largest external local sub-portrait with the attribute domain benchmark portrait, digital domain benchmark portrait, physical domain benchmark portrait and semantic domain benchmark portrait, which is the domain coverage of each domain.

Images