Remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion

Abstract

The invention provides a remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion. The method comprises the following steps: acquiring attribute information of a remote sensing target and category information of the remote sensing target detected by a plurality of sensors; fusing the attribute information detected by each sensor to obtain fused attribute information; inputting the fusion attribute information into a remote sensing target knowledge graph to obtain a first classification result; determining an error type of the remote sensing target knowledge graph according to the first classification result and the category information; and error correction processing is performed on the remote sensing target knowledge graph according to the error type and the preset error correction rule, so that the remote sensing target can be identified by using a plurality of remote sensing source data, and the blank in the field of current remote sensing target knowledge graph error correction is filled up. Different characteristics of the target object for different sensors are fully fused, the attribute information of the target is recorded in richer dimensions, and the target recognition rate is improved.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion. Background Art

[0002] Knowledge graph is a structured semantic knowledge base used to quickly describe concepts and their relationships in the physical world. Knowledge graph can aggregate a large amount of information, data and link relationships into knowledge, making information resources easier to calculate, understand and evaluate, and enabling rapid response and reasoning of knowledge. Remote sensing target recognition has always had the problems of rich background information, large amount of data, and difficulty in rapid recognition. Establishing a remote sensing knowledge base to support spatiotemporal big data mining is a convenient method. Therefore, research on the establishment of remote sensing target knowledge graph has emerged. At present, there are research results on the establishment of remote sensing information knowledge graph, but the constructed knowledge graph will inevitably have some loopholes.

[0003] In the actual process of building knowledge graphs, it is found that the quality of real-world data is usually low, which seriously affects the quality of data analysis and mining results and leads to unreliable results. Data collection and acquisition often introduce data errors, such as missing values, input errors, inconsistent formats, duplicate data for the same real-world entity, and data that violates business and data integrity rules. The meaning represented by these erroneous data does not match the real world, so the final decision is easily affected by these erroneous data, resulting in wrong decisions or deviations. There is currently no good solution.

[0004] At present, the knowledge graph error correction algorithm is not oriented to the field of remote sensing data. Due to the characteristics of remote sensing data itself, the existing algorithms cannot be simply transplanted directly. The existing knowledge graph completion only targets the problem of completing triples with some missing parts, and there is no good algorithm for supplementing and correcting the knowledge graph for newly emerging knowledge. Summary of the invention

[0005] In order to solve the above problems existing in the prior art, the present invention provides a remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion. Specifically, it includes:

[0006] In a first aspect, the present invention provides a remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion, comprising:

[0007] Acquire attribute information and category information of remote sensing targets detected by multiple sensors;

[0008] Fuse the attribute information detected by each sensor to obtain the fused attribute information of the remote sensing target;

[0009] Inputting the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain the first classification result;

[0010] Determine the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target;

[0011] The remote sensing target knowledge graph is corrected according to its error types and preset error correction rules.

[0012] In a second aspect, the present invention also provides a remote sensing target knowledge graph dynamic error correction device based on multi-source sensor fusion, comprising:

[0013] An acquisition module, used to acquire attribute information and category information of remote sensing targets detected by multiple sensors;

[0014] A processing module is used to fuse the attribute information detected by each sensor to obtain the fused attribute information of the remote sensing target;

[0015] The processing module is further used to input the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result;

[0016] The processing module is further used to determine the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target;

[0017] The processing module is also used to perform error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules.

[0018] In a third aspect, the present invention further provides an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0019] Memory, used to store computer programs;

[0020] The processor is used to implement any method provided in the first aspect when executing a program stored in the memory.

[0021] In a fourth aspect, the present invention provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, any one of the methods provided in the first aspect is implemented.

[0022] Beneficial effects of the present invention:

[0023] The remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion provided by the present invention obtains the attribute information of the remote sensing target detected by multiple sensors and the category information of the remote sensing target; fuses the attribute information detected by each sensor to obtain the fused attribute information of the remote sensing target; inputs the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result; determines the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target; performs error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules, and can use multiple remote sensing source data to identify the remote sensing target, filling the gap in the current remote sensing target knowledge graph error correction field. By using multi-source remote sensing data as the input data of the error correction algorithm, the different characteristics of the target object for different sensors are fully integrated, and the attribute information of each target is recorded in a richer dimension, thereby improving the target recognition rate.

[0024] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A schematic diagram of a process flow of a remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion provided by the present invention;

[0026] Figure 2 A schematic diagram of a remote sensing target knowledge graph provided by the present invention;

[0027] Figure 3 A schematic flow chart of another remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion provided by the present invention;

[0028] Figure 4 A schematic diagram of a flow chart of another remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion provided by the present invention;

[0029] Figure 5 A schematic diagram of the structure of a remote sensing target knowledge graph dynamic error correction device based on multi-source sensor fusion provided by the present invention. DETAILED DESCRIPTION

[0030] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0031] In the text information knowledge graph error correction, researchers use the same data to input the same knowledge graph multiple times to obtain the output results each time. If the multiple results are not consistent, it is considered that the knowledge graph is wrong, and the result with the most occurrences is considered to be the correct result. For the correction of knowledge graphs, most of the existing solutions are knowledge graph completion. The knowledge graph contains a large number of fact triples (head entity, relationship, tail entity). When a triple is missing, the relevant algorithm for knowledge graph completion can be used. Knowledge graph completion model For example, the knowledge graph embedding model embeds entities and relationships as low-dimensional representations into the knowledge graph, and defines a scoring function on the embedding space to measure the rationality of the triple. Generally speaking, valid triples should score higher than invalid triples. However, in the face of completely missing triples, the knowledge completion algorithm cannot be used to correct the knowledge graph.

[0032] In order to solve the problems existing in the prior art, the present invention provides a dynamic error correction method for remote sensing target knowledge graph based on multi-source sensor fusion. The method first identifies the error type of the remote sensing target knowledge graph, and then corrects it in a corresponding manner based on the error type.

[0033] like Figure 1 As shown, the method includes:

[0034] S101. Acquire attribute information and category information of remote sensing targets detected by multiple sensors.

[0035] Optionally, the attribute information of the remote sensing target includes one or more of shape outline, color, motion, spectrum, spatial spectrum, and dimensional information.

[0036] Sensors include but are not limited to optical sensors, radar sensors, infrared sensors, microwave radiometers, laser radars, etc. Optical sensors include but are not limited to multispectral, hyperspectral and hyperspectral sensors. The category information of remote sensing targets is the real category.

[0037] S102: Fusing the attribute information detected by each sensor to obtain fused attribute information of the remote sensing target.

[0038] The attribute information of remote sensing targets obtained by different sensors is fused, and the fused attribute information is used as the input data of the remote sensing target knowledge graph.

[0039] S103: Input the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result.

[0040] For example, a knowledge graph such as Figure 2As shown in the figure. Specific objects such as "forest" and "water body" are entities in the remote sensing target knowledge graph, and "shape" and "color" represent the association between nodes, which are relationships in the knowledge graph. "Optical sensor -> detection -> forest -> color -> green -> broad-leaved forest" is a continuous sequence from one entity to another, which means that the forest is detected by the optical sensor, its color and other characteristics are analyzed, and the tree species are classified by type.

[0041] S104: Determine the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target.

[0042] S105. Perform error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules.

[0043] Exemplarily, the error types of remote sensing target knowledge graph include first-type errors and second-type errors. The first-type error is that the recognition results obtained by different sensor signals are not exactly the same, and the existing knowledge graph needs to be modified and new path information needs to be added. The second-type error is that the identified target belongs to the missing information in the knowledge graph. In this case, the knowledge graph lacks the attributes or category information of this target, and new content needs to be added to the knowledge graph.

[0044] Accordingly, for determining the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target in S104 in the above embodiment, optionally, see Figure 3 , according to the first classification result and the category information of the remote sensing target, the error type of the remote sensing target knowledge graph is determined, including:

[0045] S301. Compare the first classification result with the category information of the remote sensing target. If the first classification is inconsistent with the category information, input the attribute information detected by each sensor into the remote sensing target knowledge graph to obtain the second classification result corresponding to each attribute information.

[0046] Specifically, if the first classification is inconsistent with the category information, it is determined that there is an error in the remote sensing target knowledge graph and it needs to be corrected; if the first classification is consistent with the category information, it is determined that there is no error in the remote sensing target knowledge graph and it does not need to be corrected.

[0047] S302, determine whether the second classification results are not completely the same, if so, execute S303, if not, execute S304.

[0048] S303: Determine that the error type of the remote sensing target knowledge graph is a first-class error.

[0049] S304. Determine that the error type of the remote sensing target knowledge graph is a second type error.

[0050] Furthermore, with respect to S105 in the above-mentioned embodiment, error correction processing is performed on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules. In a possible implementation method, when the error type of the remote sensing target knowledge graph is a first-class error, error correction processing is performed on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules, including: based on the knowledge graph completion algorithm, completing the attribute information of the triples corresponding to the remote sensing target in the remote sensing target knowledge graph.

[0051] Optionally, the attribute information of the triples corresponding to the remote sensing targets in the remote sensing target knowledge graph is completed, including modifying the path information in the remote sensing target knowledge graph, or adding new path information in the remote sensing target knowledge graph.

[0052] Specifically, a knowledge graph completion algorithm, such as the TranE algorithm, is used. It converts the original entities (targets, attributes, etc.) and relationships ("color", "shape", etc.) of the knowledge graph into low-dimensional vectors, and regards the relationship as a translation vector from the head entity to the tail entity. For example, in "forest + color = green", the "color" relationship vector can explain the gap between the "forest" and "green" vectors. And by optimizing the objective function corresponding to the algorithm, the head entity vector, relationship vector, and tail entity are closed as much as possible. Therefore, if any element in a triple in the remote sensing target knowledge graph is missing, the algorithm will find the element that is most likely to complete the vector closure; or if there is an error in the triple, the algorithm will select the correct entity that is closer to the vector, thereby updating the knowledge graph.

[0053] Further, with respect to S105 in the above embodiment, error correction processing is performed on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and the preset error correction rule. In a possible implementation, see Figure 4 , when the error type of the remote sensing target knowledge graph is the second type of error, the remote sensing target knowledge graph is corrected according to the error type of the remote sensing target knowledge graph and the preset error correction rules, including:

[0054] S401. Determine whether there is erroneous path information in the remote sensing target knowledge graph based on the second classification results. If so, execute S402; otherwise, execute S403.

[0055] S402: Delete incorrect path information.

[0056] S403. Based on reinforcement learning, determine the missing information of the remote sensing target knowledge graph, and determine new triples related to the remote sensing target based on the missing information.

[0057] S404, adding the new triples into the remote sensing target knowledge graph.

[0058] Optionally, based on reinforcement learning, the missing information of the remote sensing target knowledge graph is determined, and new triples related to the remote sensing target are determined based on the missing information, including: based on the DeepPath algorithm, a reinforcement learning model is constructed according to each second classification result, and the missing information of the remote sensing target knowledge graph is determined through the reinforcement learning model, and new triples related to the remote sensing target are determined based on the missing information.

[0059] Specifically, the current knowledge graph and the missing category information are used as input data, and the DeepPath algorithm is used for path exploration. The algorithm models the path search and planning problem in the knowledge graph as a reinforcement learning problem, and uses the policy network as an intelligent agent to perform reasoning in the knowledge graph. The policy network outputs the probability of taking different paths based on the existing relationships between nodes, and selects the next step relationship of the current node based on the probability. In this process, if the path accurately reflects the categories of the old data and the newly filled data, a reward is given, and feedback is given to the policy network to adjust the network parameters. At the end of the knowledge graph update, the maximum accumulated reward is obtained.

[0060] Since path-based remote sensing target knowledge graph reasoning requires finding a multi-hop path from the head entity to the tail entity in the remote sensing target knowledge graph, the process of establishing this path can be regarded as a Markov decision process, so reinforcement learning can be used to model this Markov decision process. Using the idea of ​​reinforcement learning when correcting remote sensing target knowledge graphs can create a knowledge graph with a simpler path.

[0061] The remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion provided by the present invention obtains the attribute information of the remote sensing target detected by multiple sensors and the category information of the remote sensing target; fuses the attribute information detected by each sensor to obtain the fused attribute information of the remote sensing target; inputs the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result; determines the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target; performs error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules, and can use multiple remote sensing source data to identify the remote sensing target, filling the gap in the current remote sensing target knowledge graph error correction field. By using multi-source remote sensing data as the input data of the error correction algorithm, the different characteristics of the target object for different sensors are fully integrated, and the attribute information of each target is recorded in a richer dimension, thereby improving the target recognition rate.

[0062] Figure 5 A schematic diagram of a remote sensing target knowledge graph dynamic error correction device based on multi-source sensor fusion provided by the present invention is shown in FIG. Figure 5As shown, the device comprises:

[0063] The acquisition module 51 is used to acquire the attribute information and category information of the remote sensing targets detected by multiple sensors.

[0064] The processing module 52 is used to fuse the attribute information detected by each sensor to obtain the fused attribute information of the remote sensing target.

[0065] The processing module 52 is also used to input the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result.

[0066] The processing module 52 is further used to determine the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target.

[0067] The processing module 52 is also used to perform error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules.

[0068] The present invention also provides a structure of an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus.

[0069] Memory, used to store computer programs;

[0070] The processor is used to implement the steps provided in the above method embodiment when executing the program stored in the memory.

[0071] The communication interface is used for communication between the above electronic device and other devices.

[0072] The method provided in the embodiment of the present invention can be applied to electronic devices. Specifically, the electronic device can be: a desktop computer, a portable computer, an intelligent mobile terminal, a server, etc. This is not limited here, and any electronic device that can implement the present invention belongs to the protection scope of the present invention.

[0073] The present invention also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps provided in the above method embodiment are implemented.

[0074] As for the device / electronic device / storage medium embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the specific content and beneficial effects and other related matters can be referred to the partial description of the method embodiment.

[0075] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0076] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A remote sensing target knowledge graph dynamic error correction method based on multi-source sensor fusion, characterized in that: include: Acquiring attribute information of remote sensing targets detected by multiple sensors and category information of the remote sensing targets; Fusing the attribute information detected by each of the sensors to obtain fused attribute information of the remote sensing target; Inputting the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result; Determining an error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target; Error correction processing is performed on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules.

2. The method according to claim 1, characterized in that The step of determining the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target includes: Comparing the first classification result with the category information of the remote sensing target, if the first classification is inconsistent with the category information, respectively inputting the attribute information detected by each sensor into the remote sensing target knowledge graph to obtain a second classification result corresponding to each attribute information; Determine whether the second classification results are not completely the same. If so, determine that the error type of the remote sensing target knowledge graph is a first type error. Otherwise, determine that the error type of the remote sensing target knowledge graph is a second type error.

3. The method according to claim 2, characterized in that When the error type of the remote sensing target knowledge graph is a first type error, The error correction processing of the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and the preset error correction rule includes: Based on the knowledge graph completion algorithm, the attribute information of the triples corresponding to the remote sensing target in the remote sensing target knowledge graph is completed.

4. The method according to any one of claims 2-3, characterized in that: When the error type of the remote sensing target knowledge graph is a type II error, The error correction processing of the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and the preset error correction rule includes: S41, determining whether there is erroneous path information in the remote sensing target knowledge graph according to each of the second classification results, if so, executing S42, if not, executing S43; S42, deleting the erroneous path information; S43, determining missing information of the remote sensing target knowledge graph based on reinforcement learning, and determining a new triplet related to the remote sensing target based on the missing information; S44, adding the new triplet into the remote sensing target knowledge graph.

5. The method according to claim 4, characterized in that The method of determining the missing information of the remote sensing target knowledge graph based on reinforcement learning, and determining a new triplet related to the remote sensing target based on the missing information, includes: Based on the DeepPath algorithm, a reinforcement learning model is constructed according to each of the second classification results, and the missing information of the remote sensing target knowledge graph is determined through the reinforcement learning model, and new triples related to the remote sensing target are determined based on the missing information.

6. The method according to claim 5, characterized in that The attribute information of the remote sensing target includes one or more of shape outline, color, motion, spectrum, spatial spectrum, and dimensional information.

7. A remote sensing target knowledge graph dynamic error correction device based on multi-source sensor fusion, characterized in that: include: An acquisition module, used to acquire attribute information of remote sensing targets detected by multiple sensors and category information of the remote sensing targets; A processing module, used for fusing the attribute information detected by each of the sensors to obtain fused attribute information of the remote sensing target; The processing module is further used to input the fused attribute information of the remote sensing target into the remote sensing target knowledge graph to obtain a first classification result; The processing module is further used to determine the error type of the remote sensing target knowledge graph according to the first classification result and the category information of the remote sensing target; The processing module is also used to perform error correction processing on the remote sensing target knowledge graph according to the error type of the remote sensing target knowledge graph and preset error correction rules.

8. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-6 when executing a program stored in a memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images