Knowledge graph generation method, device and storage medium based on SAR satellite

Abstract

This application discloses a method, device, and storage medium for generating a knowledge graph based on SAR satellites, including: using a SAR satellite to collect SAR images corresponding to a target area and constructing a coordinate system corresponding to the SAR images; using the SAR images and the coordinate system to determine multiple time series vectors corresponding to respective synthetic aperture lengths; determining a time series vector sequence corresponding to the multiple time series vectors, inputting the time series vector sequence into a joint extraction model, and determining multiple entities corresponding to the target area and the entity relationships between the entities; and constructing a knowledge graph corresponding to the target area using the multiple entities and the multiple entity relationships, wherein nodes in the knowledge graph represent entities and edges represent entity relationships. This method significantly improves the understandability, computability, and applicability of SAT images, making it easier for operators to analyze and utilize SAR images.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and storage medium for generating a knowledge graph based on SAR satellites. Background Art

[0002] SAR images are high-resolution radar images acquired through active microwave remote sensing technology. Unlike optical sensors, SAR satellites do not rely on sunlight. Instead, they create images by emitting electromagnetic waves and receiving backscattered signals from the target area.

[0003] Because SAR images are produced by emitting electromagnetic waves rather than optical imaging, SAR satellites can operate around the clock and are not restricted by lighting conditions. Furthermore, the microwave signals emitted by SAR satellites can penetrate clouds and smoke, making them suitable for use in most areas. However, SAR images also have many drawbacks. For example, because SAR uses slant-range imaging, geometric distortions such as perspective contraction, overlap, and shadows can occur in SAR images, making them difficult to interpret. Consequently, the pixel-level features of SAR images cannot be directly used by operators, requiring professionals familiar with radar scattering mechanisms and the electromagnetic properties of ground objects. This means that operators cannot directly visualize and analyze SAR images; they must undergo specialized processing before they can be used, making it extremely inconvenient for operators to analyze and utilize SAR images.

[0004] Publication number CN113779267A, titled "An Intent-Based Onboard Intelligent Mission Decision-Making Method," consists of two submodules. Module 1 is a machine learning-based intent parameter instantiation modeling approach: This involves expanding the input formatted intent into a structured intent with necessary parameters such as observation period and spatial resolution. The expanded parameters can be obtained using a machine learning model, according to specified rules, or a combination of machine learning models and rules. Module 2 is a template-based decomposition of user intent: This approach queries the knowledge graph based on intent and target type to generate different payload task lists.

[0005] Publication number CN118897688A is titled "A Software-Defined Satellite Error Correction Method Based on a Large Model Knowledge Graph." The knowledge graph contains relationships between satellite subsystems, historical failure cases, and correction solutions. When a fault is detected, the system can quickly locate the source and generate optimized correction suggestions based on similar historical cases and inference rules.

[0006] Currently, no effective solution has been proposed to the technical problem that operators cannot directly analyze SAR images visually and need to use them after professional processing, which is extremely inconvenient for operators to analyze and use SAR images. Summary of the Invention

[0007] The embodiments of the present disclosure provide a method, device, and storage medium for generating a knowledge graph based on SAR satellites, so as to at least solve the technical problem in the prior art that operators cannot directly visualize and analyze SAR images and need to use them after professional processing, which is extremely inconvenient for operators to analyze and use SAR images.

[0008] According to one aspect of an embodiment of the present disclosure, a method for generating a knowledge graph based on a SAR satellite is provided, comprising: using a SAR satellite to collect a SAR image corresponding to a target area, and constructing a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents a synthetic aperture length related to the time series, and the ordinate of the coordinate system represents a slant distance; using the SAR image and based on the coordinate system, determining a plurality of time series vectors corresponding to each synthetic aperture length, wherein the time series vector is used to indicate the distribution of signal intensities of electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensities of electromagnetic wave backscatter corresponding to different slant distances; determining a time series vector sequence corresponding to the plurality of time series vectors, and inputting the time series vector sequence into a joint extraction model, and determining a plurality of entities corresponding to the target area and entity relationships between the entities; and constructing a knowledge graph corresponding to the target area using the plurality of entities and the plurality of entity relationships, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0009] According to another aspect of an embodiment of the present disclosure, a storage medium is further provided, the storage medium including a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0010] According to another aspect of an embodiment of the present disclosure, a knowledge graph generation device based on a SAR satellite is also provided, including: a SAR image acquisition module, used to use a SAR satellite to acquire a SAR image corresponding to a target area, and construct a coordinate system corresponding to the SAR image, wherein the horizontal coordinate of the coordinate system represents a synthetic aperture length related to the time series, and the vertical coordinate of the coordinate system represents a slant distance; a time series vector determination module, used to use the SAR image and the coordinate system to determine multiple time series vectors corresponding to each synthetic aperture length, wherein the time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances; an entity and entity relationship extraction module, used to determine a time series vector sequence corresponding to the multiple time series vectors, and input the time series vector sequence into a joint extraction model, and determine multiple entities corresponding to the target area and the entity relationships between the entities; and a knowledge graph construction module, used to use the multiple entities and the multiple entity relationships to construct a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0011] According to another aspect of an embodiment of the present disclosure, a knowledge graph generation device based on a SAR satellite is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: using a SAR satellite to collect a SAR image corresponding to a target area, and constructing a coordinate system corresponding to the SAR image, wherein the horizontal coordinate of the coordinate system represents a synthetic aperture length related to the time series, and the vertical coordinate of the coordinate system represents the slant distance; using the SAR image and based on the coordinate system, determining multiple time series vectors corresponding to each synthetic aperture length, wherein the time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances; determining a time series vector sequence corresponding to the multiple time series vectors, and inputting the time series vector sequence into a joint extraction model, and determining multiple entities corresponding to the target area and the entity relationships between the entities; and using the multiple entities and the multiple entity relationships, constructing a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0012] The present application provides a knowledge graph generation method based on SAR satellites. First, the processor uses a SAR satellite to collect SAR images corresponding to the target area and constructs a coordinate system corresponding to the SAR image. Then, the processor uses the SAR image and, based on the coordinate system, determines multiple time series vectors corresponding to each synthetic aperture length. Furthermore, the processor determines a time series vector sequence corresponding to the multiple time series vectors, inputs the time series vector sequence into a joint extraction model, and determines multiple entities corresponding to the target area and the entity relationships between the entities. Finally, the processor uses the multiple entities and the multiple entity relationships to construct a knowledge graph corresponding to the target area.

[0013] Referring to the above, it can be seen that, unlike the prior art, this application does not directly utilize SAR images. Instead, taking into account that the synthetic aperture length in the SAR image is related to time and that the SAR satellite is continuously moving in the azimuth direction (corresponding to the synthetic aperture length), a coordinate system is constructed with the synthetic aperture length related to the time series as the horizontal coordinate and the slant distance as the vertical coordinate. Based on the SAR image and the constructed coordinate system, multiple time series vectors corresponding to the synthetic aperture length are determined. Furthermore, when the time series vector sequence is determined, the time series vector sequence is input into a pre-trained joint extraction model to extract the entities and entity relationships corresponding to the target area. Furthermore, when the entities and entity relationships corresponding to the target area are extracted, the multiple entities and multiple entity relationships corresponding to the target area are used to construct a knowledge graph corresponding to the target area.

[0014] Once the knowledge graph corresponding to the target area is determined, the operator can directly analyze the target area using the visualized knowledge graph, significantly improving the comprehensibility, computability, and applicability of SAR images and facilitating their analysis and utilization. This addresses the existing technical issue of operators being unable to directly visualize and analyze SAR images, requiring specialized processing before they can be used, which is extremely inconvenient for operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and constitute a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation of the present disclosure. In the drawings:

[0016] Figure 1 This is a schematic diagram of a communication connection system between a SAR satellite and a terminal device according to Example 1 of the present application;

[0017] Figure 2A1 is a schematic diagram of the hardware architecture of the SAR satellite according to Example 1 of the present application;

[0018] Figure 2B This is a schematic diagram of the hardware architecture of the terminal device according to Example 1 of the present application;

[0019] Figure 3 1 is a flow chart of the method for generating a knowledge graph based on SAR satellites according to Example 1 of the present application;

[0020] Figure 4 is a schematic diagram of a knowledge graph generating device based on SAR satellite according to Example 2 of the present application;

[0021] Figure 5 This is a schematic diagram of the SAR satellite-based knowledge graph generation device described in Example 3 of the present application. DETAILED DESCRIPTION

[0022] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present disclosure.

[0023] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0024] Example 1

[0025] According to this embodiment, a method embodiment for generating a knowledge graph based on SAR images is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0026] Figure 1 A schematic diagram of a communication connection system between a SAR satellite and a terminal device according to this embodiment is shown. The system includes a terminal device 10, a ground system 20, and a SAR satellite 30. The terminal device 10 sends an instruction to generate a knowledge graph corresponding to a target area to the SAR satellite 30 via the ground system 20. The processor of the SAR satellite 30 is configured to receive and respond to the instruction to construct a knowledge graph corresponding to the target area, acquire a SAR image corresponding to the target area, and construct a coordinate system corresponding to the SAR image. The processor (referring to the processor of the SAR satellite 30, the same applies below) is further configured to use the SAR image and the coordinate system to determine multiple time series vectors corresponding to various synthetic aperture lengths. The processor is further configured to determine a sequence of time series vectors corresponding to the multiple time series vectors, input the sequence of time series vectors into a joint extraction model, and determine multiple entities corresponding to the target area and the entity relationships between the entities. The processor is further configured to construct a knowledge graph corresponding to the target area using the multiple entities and the multiple entity relationships.

[0027] Figure 2A It further shows Figure 1 Schematic diagram of the hardware architecture of the SAR satellite 30. Figure 2A As shown, the SAR satellite 30 includes an integrated electronic system, which includes: a processor, a memory, a bus management module and a communication interface. The memory is connected to the processor, so that the processor can access the memory, read the program instructions stored in the memory, read data from the memory or write data to the memory. The bus management module is connected to the processor and is also connected to a bus such as a CAN bus. The processor can communicate with the onboard peripherals connected to the bus through the bus managed by the bus management module. In addition, the processor is also connected to devices such as cameras, star sensors, measurement and control transponders, and data transmission equipment via the communication interface. It can be understood by those skilled in the art that Figure 2A The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2A More or fewer components than shown, or with Figure 2A Different configurations shown.

[0028] Figure 2B It further shows Figure 1 Schematic diagram of the hardware architecture of the terminal device 10. Figure 2BAs shown, the terminal device 10 may include one or more processors (the processor may include, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA) or other processing device), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, transmission device, and input / output interface are connected to the processor via a bus. In addition, it may also include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those skilled in the art will understand that Figure 2B The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 2B More or fewer components than shown, or with Figure 2B Different configurations shown.

[0029] It should be noted that Figure 2A and Figure 2B The one or more processors and / or other data processing circuits shown in the figure may generally be referred to herein as "data processing circuitry." The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuitry may be a single, independent processing module, or may be fully or partially integrated into any of the other components of the computing device. As discussed in the embodiments of the present disclosure, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0030] Figure 2A and Figure 2B The memory shown in the figure can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the SAR satellite-based knowledge graph generation method in the embodiment of the present disclosure. The processor executes the software programs and modules stored in the memory to perform various functional applications and data processing, thereby implementing the SAR satellite-based knowledge graph generation method for the above-mentioned application. The memory can include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory.

[0031] It should be noted that, in some optional embodiments, the above Figure 2A and Figure 2B The devices shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware and software elements. Figure 2A and Figure 2B This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the apparatus described above.

[0032] Under the above operating environment, according to the first aspect of this embodiment, a method for generating a knowledge graph based on SAR satellite is provided. Figure 2A The processor implementation of the SAR satellite 30 shown in FIG. Figure 3 A schematic diagram of the process is shown in FIG. Figure 3 As shown, the method includes:

[0033] S302: Using a SAR satellite to collect a SAR image corresponding to the target area, and constructing a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents a synthetic aperture length related to the time series, and the ordinate of the coordinate system represents a slant distance;

[0034] S304: Determine, using the SAR image and the coordinate system, a plurality of time series vectors corresponding to respective synthetic aperture lengths, wherein the time series vectors are used to indicate the distribution of electromagnetic wave backscattering signal intensities corresponding to different slant distances, and each element in the time series vectors is used to represent the electromagnetic wave backscattering signal intensities corresponding to different slant distances;

[0035] S306: Determine a time series vector sequence corresponding to the multiple time series vectors, input the time series vector sequence into a joint extraction model, and determine multiple entities corresponding to the target area and entity relationships between the entities; and

[0036] S308: Utilize multiple entities and multiple entity relationships to construct a knowledge graph corresponding to the target area, wherein nodes in the knowledge graph represent entities, and edges represent entity relationships.

[0037] Specifically, first, the operator uses the terminal device 10 and sends an instruction to the SAR satellite 30 through the ground system 20 to build a knowledge graph corresponding to the target area. The processor of the SAR satellite 30 receives and responds to the instruction, collects a SAR image corresponding to the target area, and constructs a coordinate system corresponding to the SAR image (S302). The flight direction of the SAR satellite 30 is the azimuth direction, and each row in the SAR image represents the distance (i.e., synthetic aperture length) traveled from the SAR satellite 30 emitting electromagnetic waves to the receiving of returned electromagnetic waves when the SAR satellite 30 flies in that direction. And the distance is related to time. The specific calculation formula is: .in, L represents the synthetic aperture length, v The speed at which electromagnetic waves propagate in the air. t The side-view direction of the SAR satellite 30 is taken as the slant range direction, and each column in the SAR image represents the slant range between the SAR satellite 30 and the ground area when the SAR satellite 30 is imaging the ground area.

[0038] The processor then uses the SAR image and a coordinate system to determine the time-series vectors corresponding to each synthetic aperture length (S304). Specifically, a coordinate system is constructed with the time-series-related synthetic aperture length as the abscissa and the slant distance as the ordinate. The processor then determines the electromagnetic backscatter signal intensity corresponding to each unique synthetic aperture length and slant distance. In practice, SAR images are grayscale images, and the electromagnetic backscatter signal intensity corresponding to each unique synthetic aperture length and slant distance is equivalent to the pixel value in the SAR image. Specifically, a greater signal intensity corresponds to a larger pixel value, while a smaller signal intensity corresponds to a smaller pixel value.

[0039] Specifically, based on the SAR image and the coordinate system, the synthetic aperture length in the SAR image can be determined. , slant distance And each combination of synthetic aperture length and slant distance determines a unique signal strength. For example, and signal strength correspond, and signal strength correspond, and signal strength correspond,..., and signal strength correspond. and signal strength correspond, and signal strength correspond, and signal strength correspond,..., and signal strength Corresponding. And so on. and signal strength correspond, and signal strength correspond, and signal strength correspond,..., and signal strength correspond.

[0040] And because of the synthetic aperture length and time Corresponding, so the moment With time series vector Corresponding, and . Synthetic aperture length and time Corresponding, so the moment With time series vector Corresponding, and And so on. Synthetic aperture length and time Corresponding, so the moment With time series vector Corresponding, and .

[0041] The processor can then determine the timing vector The time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances.

[0042] It is worth noting that, since the SAR satellite 30 moves continuously in azimuth and the unit synthetic aperture length of the SAR satellite 30 moving in azimuth is the same, multiple time series vectors can be generated based on each synthetic aperture length in the SAR image.

[0043] When the processor determines multiple time-series vectors, it further determines a time-series vector sequence corresponding to the multiple time-series vectors, inputs the time-series vector sequence into a joint extraction model, and determines multiple entities corresponding to the target area and the entity relationships between each entity (S306). Specifically, the joint extraction model includes an entity recognition network and an entity relationship extraction network. The entity recognition network is used to determine the entity type label sequence corresponding to each time-series vector from the time-series vector sequence, thereby determining multiple entities corresponding to the target area. The entity recognition network includes a first LSTM unit and a CRF layer. The entity relationship extraction network is used to determine the entity relationship label sequence corresponding to each time-series vector sequence from the time-series vector sequence, thereby determining multiple entity relationships corresponding to the target area. The entity relationship extraction network includes a second LSTM unit, a local attention mechanism, a fully connected layer, and a softmax classifier. The above content will be described in detail later, so it will not be repeated here.

[0044] Finally, the processor constructs a knowledge graph corresponding to the target area using the multiple entities and the multiple entity relationships (S308). That is, the processor represents the nodes in the knowledge graph as entities and the edges as entity relationships, and constructs a knowledge graph corresponding to the target area.

[0045] As described in the background, because SAR images are produced by emitting electromagnetic waves rather than optical imaging, SAR satellites can operate around the clock and are not restricted by lighting conditions. Furthermore, the microwave signals emitted by SAR satellites can penetrate clouds and dust, making them suitable for use in most areas. However, SAR images also have many drawbacks. For example, because SAR uses slant-range imaging, geometric distortions such as perspective contraction, overlap, and shadows can occur in SAR images, making them difficult to interpret. Consequently, the pixel-level features of SAR images cannot be directly used by operators, requiring specialized personnel familiar with radar scattering mechanisms and the electromagnetic properties of ground objects. In other words, operators cannot directly visualize and analyze SAR images; they must undergo specialized processing before they can be used, making it extremely inconvenient for operators to analyze and utilize SAR images.

[0046] In view of this, the present application provides a method for generating a knowledge graph based on SAR satellites. Referring to the above-described content, it can be seen that, unlike the prior art, the present application does not directly utilize SAR images. Instead, taking into account that the synthetic aperture length in SAR images is related to time and that the SAR satellite is continuously moving in the azimuth direction (corresponding to the synthetic aperture length), a coordinate system is constructed with the synthetic aperture length related to the time series as the horizontal coordinate and the slant distance as the vertical coordinate. Based on the SAR image and the constructed coordinate system, multiple time series vectors corresponding to the synthetic aperture length are determined. Furthermore, once the sequence of time series vectors is determined, the sequence of time series vectors is input into a pre-trained joint extraction model to extract entities and entity relationships corresponding to the target area. Furthermore, once the entities and entity relationships corresponding to the target area are extracted, the knowledge graph corresponding to the target area is constructed using multiple entities and entity relationships that are mutually exclusive with the target area.

[0047] Once the knowledge graph corresponding to the target area is determined, the operator can directly analyze the target area using the visualized knowledge graph, significantly improving the comprehensibility, computability, and applicability of SAR images and facilitating their analysis and utilization. This addresses the existing technical issue of operators being unable to directly visualize and analyze SAR images, requiring specialized processing before they can be used, which is extremely inconvenient for operators.

[0048] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entities corresponding to the target area includes: inputting the time series vector sequence into the first LSTM unit, and using the first LSTM unit to output a first enhanced time series vector sequence; inputting the first enhanced time series vector sequence into the CRF unit, and using the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes probabilities corresponding to each entity type; and determining multiple entities corresponding to the target area based on the entity type label sequence corresponding to each time series vector.

[0049] Specifically, the joint extraction model includes entity extraction and entity relationship extraction from a time series vector sequence. The entity recognition network includes a first LSTM unit and a CRF unit. The first LSTM unit can resolve long-distance dependencies and capture complete contextual features. Therefore, when a time series vector sequence is input to the first LSTM unit, a first enhanced time series vector sequence can be output.

[0050] Then, the first enhanced time series vector sequence is input into the CRF unit, and the CRF unit is used to select the optimal label path through global optimization, thereby obtaining the entity type label sequence corresponding to the time series vector sequence. The entity type label sequence includes the probability corresponding to each entity type. For example, the time series vector sequence includes multiple time series vectors The entity type label sequence includes multiple entity type label vectors ,in Represents a time series vector The probability corresponding to each entity type; Represents a time series vector The probability corresponding to each entity type; and so on; Represents a time series vector The probability corresponding to each entity type.

[0051] For example, the entity type label vector , represents the probability corresponding to entity type 1, represents the probability corresponding to entity type 2, ..., represents the probability corresponding to entity type r.

[0052] And when each probability in each entity type label vector is greater than a preset threshold, the entity type of the corresponding time series vector is determined. The probability corresponding to entity type 2 Greater than the preset threshold , so that entity type 2 is the time series vector The corresponding entity type.

[0053] For another example, the entity type label vector The probability corresponding to entity type 2 and the probability corresponding to entity type 3 Greater than the preset threshold , so that entity type 2 and entity type 3 are time series vectors The corresponding entity type.

[0054] Therefore, when the processor determines the entity type corresponding to each time series vector, multiple entity types corresponding to the target area can be determined.

[0055] Optionally, the joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer and a softmax classifier, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entity relationships corresponding to the target area includes: inputting the entities and time series vector sequences corresponding to each time series vector into the second LSTM unit and the local attention mechanism layer, and outputting a second enhanced time series vector sequence; inputting the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and outputting an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and determining multiple entity relationships corresponding to the target area based on the entity relationship label sequence corresponding to each time series vector.

[0056] Specifically, the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier. The second LSTM unit can solve the long-distance dependency problem and capture complete contextual features. The local attention mechanism layer can automatically identify and weight the local semantic units around the entity pair, rather than all the words in the entire sentence, thereby more efficiently capturing the semantic clues around the entity by limiting the scope of attention. Therefore, when the processor inputs the entities and time series vector sequences corresponding to each time series vector into the second LSTM unit and the local attention mechanism layer, it can output a second enhanced time series vector sequence.

[0057] Furthermore, the processor inputs the second enhanced time series vector sequence into the fully connected layer, and the fully connected layer maps the high-level features to the specific entity relationship category space, thereby outputting the entity relationship vector sequence corresponding to the time series vector sequence. Finally, the processor inputs the entity relationship vector sequence corresponding to the time series vector sequence into the softmax classifier, so that the softmax classifier can output the entity relationship label sequence corresponding to the entity relationship vector sequence. The entity relationship label sequence includes the probabilities corresponding to each entity relationship type. For example, the time series vector sequence includes multiple time series vectors. The entity relationship label sequence includes multiple entity relationship label vectors ,in Represents a time series vector The probability corresponding to each entity relationship type; Represents a time series vector The probability corresponding to each entity relationship type; and so on; Represents a time series vector The probability corresponding to each entity relationship type.

[0058] For example, the entity relationship label vector , represents the probability corresponding to entity relationship type 1, represents the probability corresponding to entity relationship type 2, ..., Represents the probability corresponding to the entity relationship type s.

[0059] And when each probability in each entity relationship label vector is greater than a preset threshold, the entity relationship type of the corresponding time series vector is determined. The probability corresponding to entity relationship type 2 Greater than the preset threshold , so that entity relationship type 2 is related to the time series vector The corresponding entity relationship type.

[0060] For another example, the entity relationship label vector The probability corresponding to entity relationship type 2 and the probability corresponding to entity relationship type 3 Greater than the preset threshold , so that entity relationship type 2 and entity relationship type 3 are related to the time series vector The corresponding entity relationship type.

[0061] Therefore, when the processor determines the entity relationship type corresponding to each time series vector, it can determine multiple entity relationship types corresponding to the target area.

[0062] Optionally, the operation of constructing a knowledge graph corresponding to the target area using multiple entities and multiple entity relationships includes: generating multiple triples based on multiple entities and entity relationships between each entity; and constructing a knowledge graph corresponding to the target area based on the multiple triples.

[0063] Specifically, when the entities and entity relationships corresponding to each time series vector are determined, the entity and entity relationship descriptions are combined into triples. Corresponding to the entity and entities , and the time series vector Corresponding to entity relationship , then the triple is .

[0064] Based on the above operations, the processor can determine the multiple triplets corresponding to each time series vector in the time series vector sequence. Among them, each time series vector can have one triple or multiple triples.

[0065] Therefore, the above method can construct a knowledge graph corresponding to the target area, which is convenient for operators to analyze and use the SAR images corresponding to the target area based on the knowledge graph.

[0066] Therefore, according to the first aspect of this embodiment, a technical effect is achieved in which an operator can use a visualized knowledge graph to analyze and utilize the SAR image corresponding to the target area.

[0067] In addition, reference Figure 1 As shown, according to a second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, a processor executes any one of the above methods.

[0068] Therefore, according to this embodiment, a technical effect is achieved in which an operator can use a visualized knowledge graph to analyze and utilize SAR images corresponding to a target area.

[0069] It should be noted that for the aforementioned method embodiments, for simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that the present invention is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0070] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0071] Example 2

[0072] Figure 4 FIG4 shows a SAR satellite-based knowledge graph generation device 400 according to this embodiment, which corresponds to the method according to embodiment 1. Figure 4 As shown, the device includes: a SAR image acquisition module 410, which is used to use a SAR satellite to acquire a SAR image corresponding to the target area and construct a coordinate system corresponding to the SAR image, wherein the horizontal axis of the coordinate system represents a synthetic aperture length related to the time series, and the vertical axis of the coordinate system represents a slant distance; a time series vector determination module 420, which is used to use the SAR image and the coordinate system to determine a plurality of time series vectors corresponding to each synthetic aperture length, wherein the time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances; an entity and entity relationship extraction module 430, which is used to determine a time series vector sequence corresponding to the plurality of time series vectors, and input the time series vector sequence into a joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationships between the entities; and a knowledge graph construction module 440, which is used to use the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0073] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the entity and entity relationship extraction module 430 includes: a first enhancement module, used to input the time series vector sequence into the first LSTM unit, and use the first LSTM unit to output a first enhanced time series vector sequence; a first label sequence output module, used to input the first enhanced time series vector sequence into the CRF unit, and use the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes probabilities corresponding to each entity type; and an entity determination module, used to determine multiple entities corresponding to the target area based on the entity type label sequences corresponding to each time series vector.

[0074] Optionally, the joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer and a softmax classifier, and the entity and entity relationship extraction module 430 includes: a second enhancement module, which is used to input the time series vector sequence into the second LSTM unit and the local attention mechanism layer, and output a second enhanced time series vector sequence; a second label sequence output module, which is used to input the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and output an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes a probability corresponding to each entity relationship type; and an entity relationship determination module, which is used to determine multiple entity relationships corresponding to the target area based on the entity relationship label sequence corresponding to each time series vector.

[0075] Optionally, the knowledge graph construction module 440 includes: a triple generation module for generating multiple triples based on multiple entities and entity relationships between each entity; and a knowledge graph construction sub-module for constructing a knowledge graph corresponding to the target area based on multiple triples.

[0076] Therefore, according to this embodiment, a technical effect is achieved in which an operator can use a visualized knowledge graph to analyze and utilize SAR images corresponding to a target area.

[0077] Example 3

[0078] Figure 5 FIG2 shows a SAR satellite-based knowledge graph generation device 500 according to this embodiment, which corresponds to the method according to embodiment 1. Figure 5As shown, the device includes: a processor 510; and a memory connected to the processor 510, for providing the processor 510 with instructions for processing the following processing steps: using a SAR satellite to collect a SAR image corresponding to the target area, and constructing a coordinate system corresponding to the SAR image, wherein the horizontal axis of the coordinate system represents a synthetic aperture length related to the time series, and the vertical axis of the coordinate system represents a slant distance; using the SAR image and based on the coordinate system, determining multiple time series vectors corresponding to each synthetic aperture length, wherein the time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscatter corresponding to different slant distances; determining a time series vector sequence corresponding to the multiple time series vectors, and inputting the time series vector sequence into a joint extraction model, and determining multiple entities corresponding to the target area and the entity relationships between the entities; and using the multiple entities and the multiple entity relationships, constructing a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0079] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entities corresponding to the target area includes: inputting the time series vector sequence into the first LSTM unit, and using the first LSTM unit to output a first enhanced time series vector sequence; inputting the first enhanced time series vector sequence into the CRF unit, and using the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes probabilities corresponding to each entity type; and determining multiple entities corresponding to the target area based on the entity type label sequence corresponding to each time series vector.

[0080] Optionally, the joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer and a softmax classifier, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entity relationships corresponding to the target area includes: inputting the entities and time series vector sequences corresponding to each time series vector into the second LSTM unit and the local attention mechanism layer, and outputting a second enhanced time series vector sequence; inputting the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and outputting an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and determining multiple entity relationships corresponding to the target area based on the entity relationship label sequence corresponding to each time series vector.

[0081] Optionally, the operation of constructing a knowledge graph corresponding to the target area using multiple entities and multiple entity relationships includes: generating multiple triples based on multiple entities and entity relationships between each entity; and constructing a knowledge graph corresponding to the target area based on the multiple triples.

[0082] Therefore, according to this embodiment, a technical effect is achieved in which an operator can use a visualized knowledge graph to analyze and utilize SAR images corresponding to a target area.

[0083] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0084] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0085] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0086] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0087] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0088] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), a mobile hard drive, a magnetic disk, or an optical disk.

[0089] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A knowledge graph generation method based on SAR satellite, characterized in that: include: Using a SAR satellite to collect a SAR image corresponding to the target area, and constructing a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents a synthetic aperture length associated with a time sequence, and the ordinate of the coordinate system represents a slant distance; Determining, using the SAR image and according to the coordinate system, a plurality of time series vectors corresponding to respective synthetic aperture lengths, wherein the time series vectors are used to indicate distributions of electromagnetic wave backscatter signal intensities corresponding to different slant distances, and each element in the time series vectors is used to represent the electromagnetic wave backscatter signal intensities corresponding to the different slant distances; Determining a time series vector sequence corresponding to the multiple time series vectors, inputting the time series vector sequence into a joint extraction model, and determining multiple entities corresponding to the target area and entity relationships between the entities; as well as A knowledge graph corresponding to the target area is constructed using the multiple entities and multiple entity relationships, wherein the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

2. The method according to claim 1, characterized in that The joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entities corresponding to the target area includes: Inputting the time series vector sequence into the first LSTM unit, and using the first LSTM unit to output a first enhanced time series vector sequence; Inputting the first enhanced time series vector sequence into the CRF unit, and using the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes a probability corresponding to each entity type; and Based on the entity type label sequences corresponding to the respective time series vectors, a plurality of entities corresponding to the target area are determined.

3. The method according to claim 2, characterized in that The joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier, and the time series vector sequence is input into the joint extraction model, and the operation of determining multiple entity relationships corresponding to the target area includes: Input the entities corresponding to each time series vector and the time series vector sequence into the second LSTM unit and the local attention mechanism layer, and output a second enhanced time series vector sequence; Inputting the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and outputting an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes a probability corresponding to each entity relationship type; and Based on the entity relationship label sequences corresponding to the respective time series vectors, a plurality of entity relationships corresponding to the target area are determined.

4. The method according to claim 3, characterized in that The operation of constructing a knowledge graph corresponding to the target area using the multiple entities and the multiple entity relationships includes: generating multiple triples based on the multiple entities and the entity relationships between the entities; and Based on the multiple triples, a knowledge graph corresponding to the target area is constructed.

5. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 4.

6. A knowledge graph generation device based on SAR satellite, characterized in that: include: a SAR image acquisition module, configured to acquire a SAR image corresponding to the target area using a SAR satellite, and construct a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents a synthetic aperture length associated with a time sequence, and the ordinate of the coordinate system represents a slant distance; a time series vector determination module, configured to determine, using the SAR image and according to the coordinate system, a plurality of time series vectors corresponding to respective synthetic aperture lengths, wherein the time series vectors are configured to indicate a distribution of electromagnetic wave backscatter signal intensities corresponding to different slant distances, and each element in the time series vectors is configured to represent an electromagnetic wave backscatter signal intensities corresponding to the different slant distances; an entity and entity relationship extraction module, configured to determine a time series vector sequence corresponding to the plurality of time series vectors, input the time series vector sequence into a joint extraction model, and determine a plurality of entities corresponding to the target region and entity relationships between the entities; as well as A knowledge graph construction module is used to use the multiple entities and multiple entity relationships to construct a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

7. The device according to claim 6, characterized in that The joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the entity and entity relationship extraction module includes: a first enhancement module, configured to input the time series vector sequence into the first LSTM unit, and output a first enhanced time series vector sequence using the first LSTM unit; a first label sequence output module, configured to input the first enhanced time series vector sequence into the CRF unit, and use the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes a probability corresponding to each entity type; and The entity determination module is configured to determine a plurality of entities corresponding to the target area based on entity type label sequences corresponding to the respective time series vectors.

8. The device according to claim 7, characterized in that The joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer and a softmax classifier, and the entity and entity relationship extraction module includes: A second enhancement module is used to input the time series vector sequence into the second LSTM unit and the local attention mechanism layer, and output a second enhanced time series vector sequence; a second label sequence output module, configured to input the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and output an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes a probability corresponding to each entity relationship type; and The entity relationship determination module is configured to determine a plurality of entity relationships corresponding to the target area based on the entity relationship label sequences corresponding to the respective time series vectors.

9. The device according to claim 8, characterized in that The knowledge graph construction module includes: a triple generation module for generating a plurality of triples based on the plurality of entities and the entity relationships between the entities; and The knowledge graph construction submodule is used to construct a knowledge graph corresponding to the target area based on the multiple triples.

10. A knowledge graph generation device based on SAR satellite, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following steps: Using a SAR satellite to collect a SAR image corresponding to the target area, and constructing a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents a synthetic aperture length associated with a time sequence, and the ordinate of the coordinate system represents a slant distance; Determining, using the SAR image and according to the coordinate system, a plurality of time series vectors corresponding to respective synthetic aperture lengths, wherein the time series vectors are used to indicate distributions of electromagnetic wave backscatter signal intensities corresponding to different slant distances, and each element in the time series vectors is used to represent the electromagnetic wave backscatter signal intensities corresponding to the different slant distances; Determining a time series vector sequence corresponding to the multiple time series vectors, inputting the time series vector sequence into a joint extraction model, and determining multiple entities corresponding to the target area and entity relationships between the entities; as well as A knowledge graph corresponding to the target area is constructed using the multiple entities and multiple entity relationships, wherein the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

Images