A multi-domain joint driving remote sensing data tampering detection evaluation method

Abstract

The present application belongs to the field of remote sensing images, and relates to a multi-domain joint driving remote sensing data tampering detection and evaluation method for supporting detection and evaluation of tampering phenomena in remote sensing images. The present application builds a multi-domain authenticity benchmark image covering four-dimensional characteristics of real remote sensing images in attribute domain, digital domain, physical domain and semantic domain, designs a multi-domain joint evaluation method, calculates the joint evaluation approximation degree between the individual description image of the remote sensing image to be detected and the authenticity benchmark image, and realizes true or false evaluation and discrimination of the remote sensing image in multiple dimensions.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of remote sensing images, and relates to a multi-domain joint driving remote sensing data tampering detection evaluation method for supporting detection and evaluation of tampering phenomena in remote sensing images. BACKGROUND

[0002] At present, there are a large number of tampered remote sensing images in the Internet, and tampering and falsification methods are various, including traditional tampering and falsification methods such as splicing, copy-paste, removal and deep tampering and falsification methods, and tampering and falsification scenes cover different types such as airports and ports, which seriously affect the authenticity and reliability of Internet remote sensing image information, and further affect the correct understanding of public opinion on remote sensing images.

[0003] In recent years, many workers have studied the remote sensing image tampering detection task, and have proposed a series of remote sensing image tampering detection methods based on deep neural networks, and related researches are often limited to detecting tampering phenomena in a single domain such as attribute domain, digital domain, physical domain and semantic domain. The detection method in a single domain can only take effect when there is obvious tampering appearance in the domain, and cannot effectively complete the tampering detection of remote sensing images, so it is urgent to realize a multi-domain joint remote sensing image tampering detection evaluation method. SUMMARY

[0004] The technical problem to be solved by the application is that the current single-domain detection cannot effectively complete the tampering detection and evaluation of remote sensing images, and a multi-domain joint driving remote sensing data tampering detection evaluation method is proposed, a multi-domain authenticity benchmark image covering four-dimensional features of attribute domain, digital domain, physical domain and semantic domain of real remote sensing images is constructed, a multi-domain joint evaluation method is designed, a joint evaluation approximation degree between individual description images of the remote sensing images to be detected and the authenticity benchmark images is calculated, and real or fake evaluation and discrimination of remote sensing images in multiple dimensions are realized.

[0005] The technical scheme adopted by the application is:

[0006] A multi-domain joint driving remote sensing data tampering detection evaluation method, comprising the following steps:

[0007] (1) Constructing a multi-domain real benchmark image of real remote sensing images, the multi-domain real benchmark image is stored in the form of a knowledge graph, including an attribute domain benchmark image, a digital domain benchmark image, a physical domain benchmark image and a semantic domain benchmark image; the real benchmark images of each domain are connected to an entity node with the real remote sensing image as the name, so that the multi-domain real benchmark image is a whole in the knowledge graph;

[0008] (2) extract the multi-domain individual description image of the remote sensing image to be detected, the multi-domain individual description image is stored in the form of a knowledge graph, including attribute domain individual description image, digital domain individual description image, physical domain individual description image and semantic domain individual description image; the individual description images of each domain are connected to the entity node with the remote sensing image to be detected as the name, so that the multi-domain individual description image is a whole in the knowledge graph;

[0009] (3) calculate the joint evaluation approximation degree by using the multi-domain joint evaluation method for the extracted multi-domain individual description image and the multi-domain real benchmark image; and set a tampering judgment threshold M, when the calculated joint evaluation approximation degree is less than M, it is determined that the remote sensing image to be detected has tampering phenomenon.

[0010] Further, in step (1):

[0011] The attribute domain benchmark image describes the basic parameters of the real remote sensing image, including image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate; wherein the parameter name is stored by using the graph relationship, and the remote sensing image name and parameter content are stored by using the graph entity;

[0012] The digital domain benchmark image describes the digital signal consistency features generated and preserved in the production, transmission and storage process of the real remote sensing image, including noise distribution difference feature, frequency energy distribution difference feature and gradient change difference feature; wherein the feature name is stored by using the graph relationship, and the remote sensing image name and feature parameter are stored by using the graph entity;

[0013] The physical domain benchmark image describes the physical imaging law features commonly existing in the real remote sensing image, including gray intensity feature, shadow direction feature, color interval feature, noise interval feature and boundary artifact feature; wherein the feature name is stored by using the graph relationship, and the remote sensing image name and feature parameter are stored by using the graph entity;

[0014] The semantic domain benchmark image describes the semantic logical relationship between scenes and targets and between targets in the real remote sensing image, including parking, deployment and appearance; wherein the scene and target are stored by using the graph entity, and the semantic logical relationship is stored by using the graph relationship.

[0015] Further, in step (2):

[0016] The attribute domain individual description image is extracted by the metadata file of the remote sensing image, and the parameter content with the parameter name of image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate is acquired by using text analysis; wherein the parameter name is stored by using the graph relationship, and the remote sensing image name and parameter content are stored by using the graph entity;

[0017] The digital domain individual description image extracts noise distribution difference features, frequency energy distribution difference features and gradient change difference features of the remote sensing image to be detected through LEE filtering, mel-frequency cepstral coefficients and a co-occurrence matrix, stores feature types as atlas relationships of the image, and stores feature extraction results as atlas entities of the image;

[0018] The physical domain individual description image extracts gray intensity, shadow direction, color interval, noise interval and boundary artifacts of the remote sensing image to be detected through a color histogram, binary segmentation, noise estimation method and double-flow R-CNN network model, wherein feature names are stored by atlas relationships, and remote sensing image names and feature parameters are stored by atlas entities;

[0019] The semantic domain individual description image extracts scene and target appearing in the remote sensing image to be detected and semantic logical relationships between targets, including target position, target model, relative relationship and change, through image semantic understanding, target detection and feature classification methods, wherein scenes and targets are stored by atlas entities, and semantic logical relationships are stored by atlas relationships.

[0020] Further, the joint approximation degree in step (3) is calculated by a weighted sum of domain approximation degrees and domain coverage degrees of each domain, and the calculation process of the domain approximation degree and the domain coverage degree is as follows:

[0021] The attribute domain approximation degree calculation method is that transR is used to calculate embedded vectors of the attribute domain individual description image and the attribute domain reference image respectively, and the Euclidean distance is used to calculate the approximation degree of the embedded vector of the attribute domain individual description image and the embedded vector of the attribute domain reference image as the attribute domain approximation degree; the digital domain approximation degree calculation method is that transR is used to calculate embedded vectors of the digital domain individual description image and the digital domain reference image respectively, and the Euclidean distance is used to calculate the approximation degree of the embedded vector of the digital domain individual description image and the embedded vector of the digital domain reference image as the digital domain approximation degree; the physical domain approximation degree calculation method is that transR is used to calculate embedded vectors of the physical domain individual description image and the physical domain reference image respectively, and the Euclidean distance is used to calculate the approximation degree of the embedded vector of the physical domain individual description image and the embedded vector of the physical domain reference image as the physical domain approximation degree; and the semantic domain approximation degree calculation method is that transR is used to calculate embedded vectors of the semantic domain individual description image and the semantic domain reference image respectively, and the Euclidean distance is used to calculate the approximation degree of the embedded vector of the semantic domain individual description image and the embedded vector of the semantic domain reference image as the semantic domain approximation degree;

[0022] The calculation method of domain coverage is: traversing all local sub-images in the multi-domain real benchmark image, using transR to calculate the embedded vectors of all local sub-images in the multi-domain real benchmark image; using transR to calculate the embedded vectors of the multi-domain individual description image, calculating the multi-domain real benchmark image local sub-image with the largest embedding vector approximation degree with the multi-domain individual description image, and naming it as the largest circumscribed local sub-image; calculating the intersection ratio of the largest circumscribed local sub-image and the attribute domain benchmark image, the digital domain benchmark image, the physical domain benchmark image and the semantic domain benchmark image, that is, the domain coverage of each domain.

[0023] Compared with the background art, the present application has the following advantages:

[0024] The present application provides a multi-domain joint driving remote sensing data tampering detection and evaluation method, which models benchmark images and individual description images in different domains, and uses a multi-domain joint evaluation method to calculate a joint approximation degree as a benchmark for detecting whether a remote sensing image is tampered with, solving the problem that existing tampering detection methods only detect from a single domain dimension, consider a single feature dimension type, and are not comprehensive in adapting to tampering phenomena. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is the overall process framework design diagram of the present application. DETAILED DESCRIPTION

[0026] The specific embodiments of the present application will be described below with reference to the accompanying drawings, so that those skilled in the art can better understand the present application. It should be particularly noted that in the following description, when the detailed description of known functions and designs may obscure the main content of the present application, these descriptions will be omitted here.

[0027] Figure 1 is the principle process framework diagram of a specific implementation of the multi-domain joint driving remote sensing image tampering detection and evaluation method of the present application.

[0028] In this embodiment, as shown in Figure 1 the multi-domain joint driving remote sensing data tampering detection and evaluation method, the following steps are included:

[0029] (1) Constructing a multi-domain real benchmark image of a real remote sensing image, the multi-domain real benchmark image is stored in the form of a knowledge graph, including an attribute domain benchmark image, a digital domain benchmark image, a physical domain benchmark image and a semantic domain benchmark image; the real benchmark images of each domain are connected to the entity node with the real remote sensing image as the name, so that the multi-domain real benchmark image is a whole in the knowledge graph. Among them:

[0030] The attribute domain reference image describes the basic parameters of the real remote sensing image, including image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate; wherein the parameter name is stored by graph relationship, and the remote sensing image name and parameter content are stored by graph entity; for example, the "high resolution two satellite remote sensing image" entity is connected to the "1m" entity by the "resolution" relationship.

[0031] The digital domain reference image describes the digital signal consistency features generated and retained in the production, transmission and storage process of the real remote sensing image, including noise distribution difference feature, frequency energy distribution difference feature and gradient change difference feature; wherein the feature name is stored by graph relationship, and the remote sensing image name and feature parameter are stored by graph entity; for example, the "optical remote sensing image" entity is connected to the "0.32" entity by the "noise distribution difference feature" relationship.

[0032] The physical domain reference image describes the physical imaging rule features commonly existing in the real remote sensing image, including gray intensity feature, shadow direction feature, color interval feature, noise interval feature and boundary artifact feature; wherein the feature name is stored by graph relationship, and the remote sensing image name and feature parameter are stored by graph entity; for example, the "high resolution two satellite remote sensing image" entity is connected to the "1db" entity by the "noise interval" relationship.

[0033] The semantic domain reference image describes the semantic logical relationship between scenes and targets and between targets in the real remote sensing image, including parking, deployment and appearance; wherein the scene and target are stored by graph entity, and the semantic logical relationship is stored by graph relationship; for example, the "aircraft carrier" entity is connected to the "port" base by the "parking" relationship.

[0034] (2) Extract the multi-domain individual description image of the remote sensing image to be detected, and the multi-domain individual description image is stored in the form of a knowledge graph, including attribute domain individual description image, digital domain individual description image, physical domain individual description image and semantic domain individual description image; the individual description images of each domain are connected to the entity node with the remote sensing image to be detected as the name, so that the multi-domain individual description image is a whole in the knowledge graph. Wherein:

[0035] The attribute domain individual description image is extracted by the metadata file of the remote sensing image, and the parameter name is acquired by text analysis, including image source, sensor type, data level, resolution, pixel size, geographic attribute and compression rate; wherein the parameter name is stored by graph relationship, and the remote sensing image name and parameter content are stored by graph entity.

[0036] The digital domain individual description image extracts noise distribution difference features, frequency energy distribution difference features and gradient change difference features of the remote sensing image to be detected through LEE filtering, mel-frequency cepstral coefficients and a co-occurrence matrix, stores feature types as atlas relationships of the image and stores feature extraction results as atlas entities of the image.

[0037] The physical domain individual description image extracts gray intensity, shadow direction, color interval, noise interval and boundary artifact of the remote sensing image to be detected through a color histogram, binary segmentation, noise estimation method and a two-stream R-CNN network model, wherein feature names are stored by atlas relationships and remote sensing image names and feature parameters are stored by atlas entities.

[0038] The semantic domain individual description image extracts scene and target appearing in the remote sensing image to be detected and semantic logical relationships between targets, including target position, target model, relative relationship and change, by image semantic understanding, target detection and feature classification method, wherein scenes and targets are stored by atlas entities and semantic logical relationships are stored by atlas relationships.

[0039] (3) The multi-domain joint evaluation method is used to calculate a joint evaluation approximation degree between the extracted multi-domain individual description image and the multi-domain real benchmark image, and a tampering judgment threshold M is set, and when the calculated joint evaluation approximation degree is less than M, it is determined that the remote sensing image to be detected has tampering phenomenon.

[0040] The joint evaluation approximation degree is calculated by weighted sum of domain approximation degrees and domain coverage degrees of each domain, and the calculation process of the domain approximation degrees and the domain coverage degrees is as follows:

[0041] The attribute domain approximation degree calculation method is: using transR to calculate the embedded vectors of the attribute domain individual description image and the attribute domain reference image respectively, and using Euclidean distance to calculate the approximation degree of the embedded vectors of the attribute domain individual description image and the embedded vectors of the attribute domain reference image as the attribute domain approximation degree; the digital domain approximation degree calculation method is: using transR to calculate the embedded vectors of the digital domain individual description image and the digital domain reference image respectively, and using Euclidean distance to calculate the approximation degree of the embedded vectors of the digital domain individual description image and the embedded vectors of the digital domain reference image as the digital domain approximation degree; the physical domain approximation degree calculation method is: using transR to calculate the embedded vectors of the physical domain individual description image and the physical domain reference image respectively, and using Euclidean distance to calculate the approximation degree of the embedded vectors of the physical domain individual description image and the embedded vectors of the physical domain reference image as the physical domain approximation degree; the semantic domain approximation degree calculation method is: using transR to calculate the embedded vectors of the semantic domain individual description image and the semantic domain reference image respectively, and using Euclidean distance to calculate the approximation degree of the embedded vectors of the semantic domain individual description image and the embedded vectors of the semantic domain reference image as the semantic domain approximation degree.

[0042] The domain coverage calculation method is: traversing all local sub-images in the multi-domain reference image, using transR to calculate the embedded vectors of all local sub-images in the multi-domain reference image; using transR to calculate the embedded vectors of the multi-domain individual description image, calculating the local sub-image of the multi-domain reference image with the largest approximation degree with the embedded vectors of the multi-domain individual description image, and naming it as the largest circumscribed local sub-image; calculating the intersection ratio of the largest circumscribed local sub-image and the attribute domain reference image, the digital domain reference image, the physical domain reference image and the semantic domain reference image, which is the domain coverage of each domain.

[0043] Although the above describes the specific embodiments of the present application in order to facilitate the understanding of the present application by those skilled in the art, it should be clear that the present application is not limited to the scope of the specific embodiments, and for those skilled in the art, it is obvious that various changes are within the spirit and scope of the present application defined and determined by the appended claims, and all the inventions utilizing the concept of the present application are within the scope of protection.

Claims

1. A multi-domain jointly driven method for detecting and evaluating remote sensing data tampering, characterized in that, Includes the following steps: (1) Construct a multi-domain real benchmark profile of real remote sensing images. The multi-domain real benchmark profile is stored in the form of a knowledge graph, including attribute domain benchmark profile, digital domain benchmark profile, physical domain benchmark profile and semantic domain benchmark profile. Each domain's real baseline profile is connected to an entity node named after the real remote sensing image, making the multi-domain real baseline profiles a whole in the knowledge graph; (2) Extract multi-domain individual description profiles of the remote sensing image to be detected. The multi-domain individual description profiles are stored in the form of a knowledge graph, including attribute domain individual description profiles, digital domain individual description profiles, physical domain individual description profiles and semantic domain individual description profiles. The individual description profiles of each domain are connected to entity nodes named after the remote sensing image to be detected, so that the multi-domain individual description profiles are a whole in the knowledge graph. (3) The extracted multi-domain individual description profile and the multi-domain real benchmark profile are evaluated using a multi-domain joint evaluation method to calculate the joint evaluation similarity; and a tampering judgment threshold M is set. When the calculated joint evaluation similarity is less than M, it is determined that the remote sensing image to be detected has been tampered with.

2. The remote sensing data tampering detection and evaluation method driven by multi-domain joint operation according to claim 1, characterized in that, In step (1): The attribute domain baseline profile describes the basic parameters of the real remote sensing image, including image source, sensor type, data level, resolution, pixel size, geographic attributes, and compression rate; the parameter names are stored using map relationships, and the remote sensing image name and parameter content are stored using map entities. The digital domain baseline profile describes the consistency characteristics of digital signals generated and retained in the production, transmission and storage of real remote sensing images, including noise distribution differences, frequency energy distribution differences and gradient change differences; feature names are stored using spectral relationships, and remote sensing image names and feature parameters are stored using spectral entities. The physical domain baseline profile describes the physical imaging characteristics that are common in real remote sensing images, including grayscale intensity characteristics, shadow direction characteristics, color range characteristics, noise range characteristics, and boundary artifact characteristics; feature names are stored using spectral relationships, and remote sensing image names and feature parameters are stored using spectral entities. The semantic domain baseline profile describes the semantic logical relationships between scenes and targets, as well as between targets, in real remote sensing images, including docking, deployment, and appearance; scenes and targets are stored using graph entities, and semantic logical relationships are stored using graph relationships.

3. The remote sensing data tampering detection and evaluation method driven by multi-domain joint operation according to claim 1, characterized in that, In step (2): Individual description profiles of attribute domains are extracted from the metadata files of remote sensing images. The parameter content, named image source, sensor type, data level, resolution, pixel size, geographic attribute, and compression rate, is obtained by text parsing. The parameter names are stored using map relationships, while the remote sensing image name and parameter content are stored using map entities. The digital domain individual description profile extracts noise distribution difference features, frequency energy distribution difference features, and gradient change difference features from the remote sensing image to be detected by LEE filtering, Mel-frequency cepstral coefficients, and co-occurrence matrix. The feature types are stored as the spectral relationships of the profile, and the feature extraction results are stored as the spectral entities of the profile. Physical domain individual description profiles extract grayscale intensity, shadow direction, color range, noise range, and boundary artifacts from the remote sensing images to be detected using color histograms, binary segmentation, noise estimation methods, and a two-stream R-CNN network model. The feature names are stored using spectral relationships, while the remote sensing image names and feature parameters are stored using spectral entities. Semantic domain individual description profiles extract the semantic logical relationships between scenes and targets, as well as between targets, in the remote sensing images to be detected through image semantic understanding, target detection, and land cover classification methods. This includes target location, target type, relative relationship, and changes. Scenes and targets are stored using graph entities, while semantic logical relationships are stored using graph relationships.

4. The remote sensing data tampering detection and evaluation method driven by multi-domain joint operation according to claim 1, characterized in that, In step (3), the joint evaluation approximation is calculated by weighted sum of the domain approximation and domain coverage of each domain. The calculation process for domain approximation and domain coverage is as follows: The attribute domain approximation calculation method is as follows: The embedded vectors of the attribute domain individual description portrait and the attribute domain baseline portrait are calculated using transR, and the approximation between the embedded vectors of the attribute domain individual description portrait and the embedded vectors of the attribute domain baseline portrait are calculated using Euclidean distance, which is used as the attribute domain approximation. The digital domain approximation calculation method is as follows: The embedded vectors of the digital domain individual description portrait and the digital domain baseline portrait are calculated using transR, and the approximation between the embedded vectors of the digital domain individual description portrait and the embedded vectors of the digital domain baseline portrait are calculated using Euclidean distance, which is used as the digital domain approximation. The physical domain approximation calculation method is as follows: The embedded vectors of the physical domain individual description portrait and the physical domain true baseline portrait are calculated using transR, and the approximation between the embedded vectors of the physical domain individual description portrait and the embedded vectors of the physical domain baseline portrait are calculated using Euclidean distance, which is used as the physical domain approximation. The semantic domain approximation calculation method is as follows: transR is used to calculate the embedded vectors of the semantic domain individual description profile and the semantic domain baseline profile respectively, and Euclidean distance is used to calculate the approximation between the embedded vectors of the semantic domain individual description profile and the embedded vectors of the semantic domain baseline profile, which is used as the semantic domain approximation. The domain coverage is calculated as follows: traverse all local sub-portfolios in the multi-domain real baseline portrait, use transR to calculate the embedded vector of all local sub-portfolios in the multi-domain real baseline portrait; use transR to calculate the embedded vector of the multi-domain individual description portrait, and calculate the local sub-portfolio of the multi-domain real baseline portrait that has the largest approximation with the embedded vector of the multi-domain individual description portrait, and name it the maximum bounding local sub-portfolio. The intersection ratio of the largest enclosing local sub-image with the attribute domain baseline image, the numeric domain baseline image, the physical domain baseline image, and the semantic domain baseline image is calculated, which is the domain coverage of each domain.

Images