Knowledge graph generation method and device based on SAR satellite and storage medium

Abstract

The invention discloses a knowledge graph generation method and device based on an SAR satellite and a storage medium, and the method comprises the steps: collecting an SAR image corresponding to a target region through the SAR satellite, and constructing a coordinate system corresponding to the SAR image; determining a plurality of time sequence vectors corresponding to each synthetic aperture length by using the SAR image according to the coordinate system; determining a time sequence vector sequence corresponding to the plurality of time sequence vectors, inputting the time sequence vector sequence into the joint extraction model, and determining a plurality of entities corresponding to the target area and entity relationships among the entities; and utilizing the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, nodes in the knowledge graph representing the entities, and edges representing the entity relationships. The technical effects of remarkably improving the understandability, the computability and the application of the SAT image and facilitating the analysis and utilization of the SAR image by the operator are achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a method, device and storage medium for generating a knowledge graph based on SAR satellites. Background Art

[0002] SAR images are high-resolution radar images acquired through active microwave remote sensing technology. Unlike optical sensors, SAR satellites do not rely on sunlight, but instead create images by emitting electromagnetic waves and receiving backscattered signals from the target area.

[0003] Since SAR images are not based on optical imaging but are formed by emitting electromagnetic waves, SAR satellites can work around the clock and are not restricted by lighting conditions. In addition, the microwave signals emitted by SAR satellites can penetrate clouds and smoke, and are suitable for most areas. However, SAR images also have many disadvantages. For example, since SAR uses slant range imaging, SAR images have geometric distortions such as perspective contraction, overlap, and shadows, which makes interpretation difficult. Therefore, the pixel-level features of SAR images do not support direct use by operators, and professionals are required to be familiar with radar scattering mechanisms and electromagnetic characteristics of ground objects. In other words, operators cannot directly visualize and analyze SAR images, and they need to be professionally processed before they can use SAR images, which is extremely inconvenient for operators to analyze and use SAR images.

[0004] The publication number is CN113779267A, and the name is an intent-based on-board intelligent task decision-making method. It includes two sub-modules. Module one is the instantiation modeling of intent parameters based on machine learning: by expanding the input formatted intent into a structured intent with necessary parameters such as observation period and spatial resolution. The extended parameters can be obtained by using a machine learning model, according to specified rules, or by combining machine learning models and rules; Module two is the user intent decomposition based on template library reasoning: querying the knowledge graph according to the intent and target type to obtain different payload task lists.

[0005] The publication number is CN118897688A, and the name is A method for guiding software-defined satellite error correction based on a large model knowledge graph. The knowledge graph contains the relationship between satellite subsystems, historical failure cases and correction solutions. When a fault is detected, the system can quickly locate the source of the fault and generate optimized correction suggestions based on similar historical cases and reasoning rules.

[0006] With regard to the technical problem in the above-mentioned prior art that operators cannot directly visualize and analyze SAR images and need to use SAR images after professional processing, which is extremely inconvenient for operators to analyze and use SAR images, no effective solution has been proposed so far. Summary of the invention

[0007] Embodiments of the present disclosure provide a method, apparatus, and storage medium for generating a knowledge graph based on an SAR satellite, so as to at least solve the technical problem in the prior art that operators cannot directly visualize and analyze SAR images and need to process them professionally before they can utilize the SAR images, which is extremely inconvenient for operators to analyze and utilize the SAR images.

[0008] According to one aspect of the embodiments of the present disclosure, there is provided a method for generating a knowledge graph based on an SAR satellite, including: using the SAR satellite to collect SAR images corresponding to a target area, and constructing a coordinate system corresponding to the SAR images, where the abscissa of the coordinate system represents the synthetic aperture length related to time series, and the ordinate of the coordinate system represents the slant range; using the SAR images and according to the coordinate system, determining a plurality of time series vectors corresponding to each synthetic aperture length, where the time series vectors are used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; determining a time series vector sequence corresponding to the plurality of time series vectors, inputting the time series vector sequence into a joint extraction model, and determining a plurality of entities corresponding to the target area and the entity relationships between the entities; and using the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, where the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0009] According to another aspect of the embodiments of the present disclosure, there is also provided a storage medium, where the storage medium includes a stored program, and when the program runs, the above-mentioned method is executed by a processor.

[0010] According to another aspect of the embodiments of the present disclosure, there is also provided a knowledge graph generation device based on a SAR satellite, including: a SAR image acquisition module, configured to acquire a SAR image corresponding to a target area by using the SAR satellite and construct a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents the synthetic aperture length related to the time sequence, and the ordinate of the coordinate system represents the slant range; a time sequence vector determination module, configured to determine a plurality of time sequence vectors corresponding to each synthetic aperture length by using the SAR image and according to the coordinate system, wherein the time sequence vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time sequence vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; an entity and entity relationship extraction module, configured to determine a time sequence vector sequence corresponding to the plurality of time sequence vectors, input the time sequence vector sequence into a joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationships between the entities; and a knowledge graph construction module, configured to construct a knowledge graph corresponding to the target area by using the plurality of entities and the plurality of entity relationships, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0011] According to another aspect of the embodiments of the present disclosure, there is also provided a knowledge graph generation device based on a SAR satellite, including: a processor; and a memory connected to the processor, configured to provide instructions for the processor to perform the following processing steps: acquiring a SAR image corresponding to a target area by using the SAR satellite and constructing a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents the synthetic aperture length related to the time sequence, and the ordinate of the coordinate system represents the slant range; determining a plurality of time sequence vectors corresponding to each synthetic aperture length by using the SAR image and according to the coordinate system, wherein the time sequence vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time sequence vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; determining a time sequence vector sequence corresponding to the plurality of time sequence vectors, inputting the time sequence vector sequence into a joint extraction model, and determining a plurality of entities corresponding to the target area and the entity relationships between the entities; and constructing a knowledge graph corresponding to the target area by using the plurality of entities and the plurality of entity relationships, wherein the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0012] The present application provides a method for generating a knowledge graph based on an SAR satellite. First, a processor uses the SAR satellite to collect an SAR image corresponding to a target area and constructs a coordinate system corresponding to the SAR image. Then, the processor uses the SAR image and based on the coordinate system, determines a plurality of temporal vectors corresponding to each synthetic aperture length. Further, the processor determines a temporal vector sequence corresponding to the plurality of temporal vectors, inputs the temporal vector sequence into a joint extraction model, and determines a plurality of entities corresponding to the target area and the entity relationships between each entity. Finally, the processor uses the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area.

[0013] As can be seen from the above content, different from the prior art, the present application does not directly use the SAR image. Instead, considering that the synthetic aperture length in the SAR image is related to time and the SAR satellite moves continuously in the azimuth direction (corresponding to the synthetic aperture length), a coordinate system is constructed with the synthetic aperture length related to time as the abscissa and the slant range as the ordinate, and based on the SAR image and the constructed coordinate system, a plurality of temporal vectors corresponding to the synthetic aperture length are determined. Further, in the case of determining the temporal vector sequence, the temporal vector sequence is input into a pre-trained joint extraction model, and the entities and entity relationships corresponding to the target area can be extracted. And in the case of extracting the entities and entity relationships corresponding to the target area, a knowledge graph corresponding to the target area is constructed using the plurality of entities and the plurality of entity relationships corresponding to the target area.

[0014] Thus, in the case of determining the knowledge graph corresponding to the target area, the operator can directly analyze the target area using the visual knowledge graph, thereby significantly improving the comprehensibility, computability, and applicability of the SAR image, and facilitating the operator to analyze and utilize the SAR image. Furthermore, it solves the technical problem in the prior art that the operator cannot directly and visually analyze the SAR image and needs to process it professionally before being able to utilize the SAR image, which is extremely inconvenient for the operator to analyze and utilize the SAR image. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings described herein are used to provide a further understanding of the present disclosure and form a part of this application. The illustrative embodiments of the present disclosure and their descriptions are used to explain the present disclosure and do not constitute an improper limitation to the present disclosure. In the drawings: Figure 1 is a schematic diagram for implementing the communication connection system between the SAR satellite and the terminal device according to Embodiment 1 of the present application; Figure 2A is a schematic hardware architecture diagram of the SAR satellite according to Embodiment 1 of the present application; Figure 2BIt is a schematic diagram of the hardware architecture of the terminal device according to Embodiment 1 of the present application; Figure 3 It is a schematic flowchart of a method for generating a knowledge graph based on SAR satellites according to Embodiment 1 of the present application; Figure 4 It is a schematic diagram of a device for generating a knowledge graph based on SAR satellites according to Embodiment 2 of the present application; Figure 5 It is a schematic diagram of a device for generating a knowledge graph based on SAR satellites according to Embodiment 3 of the present application. Detailed implementation manners

[0016] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present disclosure.

[0017] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order different from those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0018] Embodiment 1 According to this embodiment, a method embodiment for generating a knowledge graph based on SAR images is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that here.

[0019] Figure 1A schematic diagram of the communication connection system between the SAR satellite and the terminal device according to this embodiment is shown. The system includes: a terminal device 10, a ground system 20, and a SAR satellite 30. Among them, the terminal device 10 sends an instruction to generate a knowledge graph corresponding to the target area to the SAR satellite 30 through the ground system 20. The processor of the SAR satellite 30 is used to receive and respond to the instruction to construct a knowledge graph corresponding to the target area, collect a SAR image corresponding to the target area, and construct a coordinate system corresponding to the SAR image. The processor (referring to the processor of the above-mentioned SAR satellite 30, the same hereinafter) is also used to use the SAR image and determine a plurality of timing vectors corresponding to each synthetic aperture length according to the coordinate system. The processor is also used to determine a timing vector sequence corresponding to the plurality of timing vectors, input the timing vector sequence into the joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationships between the entities. The processor is also used to construct a knowledge graph corresponding to the target area by using the plurality of entities and the plurality of entity relationships.

[0020] Figure 2A Further shown is Figure 1 a schematic diagram of the hardware architecture of the SAR satellite 30 in. Refer to Figure 2A As shown, the SAR satellite 30 includes an integrated electronic system, and the integrated electronic system includes: a processor, a memory, a bus management module, and a communication interface. Among them, the memory is connected to the processor, so that the processor can access the memory, read the program instructions stored in the memory, read data from the memory, or write data to the memory. The bus management module is connected to the processor and is also connected to a bus such as a CAN bus. Thus, the processor can communicate with the on-board peripherals connected to the bus through the bus managed by the bus management module. In addition, the processor is also communicatively connected to devices such as a camera, a star sensor, a TT&C transponder, and a data transmission device via the communication interface. Those of ordinary skill in the art can understand that Figure 2A the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the SAR satellite may also include more or fewer components than Figure 2A shown in, or have a configuration different from Figure 2A shown in.

[0021] Figure 2B Further shown is Figure 1 a schematic diagram of the hardware architecture of the terminal device 10 in. Refer to Figure 2BAs shown, the terminal device 10 may include one or more processors (the processors may include, but are not limited to, processing devices such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, a transmission device for communication functions, and an input / output interface. The memory, the transmission device, and the input / output interface are connected to the processor via a bus. In addition, it may further include: a display, a keyboard, and a cursor control device connected to the input / output interface. Those of ordinary skill in the art can understand that Figure 2B the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the ground system may further include more or fewer components than Figure 2B shown therein, or have a different configuration from Figure 2B shown.

[0022] It should be noted that Figure 2A and Figure 2B one or more processors and / or other data processing circuits shown in

[0023] Figure 2A and Figure 2B are generally referred to as "data processing circuits" herein. The data processing circuit may be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuit may be a single independent processing module, or be incorporated in whole or in part into any one of other elements in the computing device. As involved in the embodiments of the present disclosure, the data processing circuit is a kind of processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0024] It should be noted here that in some alternative embodiments, the above-mentioned Figure 2A and Figure 2B devices may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. It should be pointed out that Figure 2A and Figure 2B is only an example of a specific specific instance and is intended to show the types of components that may exist in the above-mentioned devices.

[0025] Under the above operating environment, according to the first aspect of this embodiment, a method for generating a knowledge graph based on a SAR satellite is provided. This method is implemented by the processor of the SAR satellite 30 shown in Figure 2A As shown. Figure 3 The flow schematic diagram of this method is shown. Referring to Figure 3 As shown, this method includes: S302: Use the SAR satellite to collect a SAR image corresponding to the target area, and construct a coordinate system corresponding to the SAR image, where the abscissa of the coordinate system represents the synthetic aperture length related to the time series, and the ordinate of the coordinate system represents the slant range; S304: Use the SAR image and according to the coordinate system, determine a plurality of time series vectors corresponding to each synthetic aperture length, where the time series vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time series vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; S306: Determine a time series vector sequence corresponding to the plurality of time series vectors, input the time series vector sequence into the joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationships between the entities; and S308: Use the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, where the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0026] Specifically, first, the operator uses the terminal device 10 and sends an instruction to construct a knowledge graph corresponding to the target area to the SAR satellite 30 through the ground system 20. The processor of the SAR satellite 30 receives and responds to this instruction, collects a SAR image corresponding to the target area, and constructs a coordinate system corresponding to the SAR image (S302). Among them, the flight direction of the SAR satellite 30 is the azimuth direction, and each row in the SAR image represents the distance (i.e., the synthetic aperture length) that the SAR satellite 30 travels when emitting electromagnetic waves to receiving the returned electromagnetic waves in this direction. And among them, this distance is related to time. The specific calculation formula is: . Among them, L represents the synthetic aperture length, v represents the propagation speed of electromagnetic waves in the air, t represents the time elapsed from emitting electromagnetic waves to receiving the returned electromagnetic waves. Taking the side-looking direction of the SAR satellite 30 as the slant range direction, and each column in the SAR image represents the slant range between the SAR satellite 30 and the ground area when imaging the ground area.

[0027] Then, the processor uses the SAR image and determines a timing vector corresponding to each synthetic aperture length according to the coordinate system (S304). Specifically, in the case where a coordinate system is constructed with the synthetic aperture length related to timing as the abscissa and the slant range as the ordinate, the signal intensity of the electromagnetic wave backscattering corresponding to the unique synthetic aperture length and the slant range is determined. In an actual situation, the SAR image is a grayscale image, and the signal intensity of the electromagnetic wave backscattering corresponding to the unique synthetic aperture length and the slant range is equivalent to the pixel value in the SAR image. That is, the greater the signal intensity, the greater the pixel value; the smaller the signal intensity, the smaller the pixel value.

[0028] Specifically, based on the SAR image and the coordinate system, it is possible to determine the synthetic aperture length included in the SAR image , the slant range . And each combination of the synthetic aperture length and the slant range determines a unique signal intensity. For example, corresponds to the signal intensity , corresponds to the signal intensity , corresponds to the signal intensity ,..., corresponds to the signal intensity . corresponds to the signal intensity , corresponds to the signal intensity , corresponds to the signal intensity ,..., corresponds to the signal intensity . And so on. corresponds to the signal intensity , corresponds to the signal intensity , corresponds to the signal intensity ,..., corresponds to the signal intensity .

[0029] And since the synthetic aperture length corresponds to the time , so the time corresponds to the timing vector , and . The synthetic aperture length corresponds to the time , so the time corresponds to the timing vector , and . And so on. The synthetic aperture length corresponds to the time corresponds, so at all times and the timing vector corresponds, and .

[0030] Thus, the processor can determine the timing vector . The timing vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the timing vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges.

[0031] It should be noted that since the SAR satellite 30 moves continuously in the azimuth direction and the unit synthetic aperture length of the SAR satellite 30 moving in the azimuth direction is the same, multiple timing vectors can be generated based on each synthetic aperture length in the SAR image.

[0032] When the processor determines multiple timing vectors, it further determines a timing vector sequence corresponding to the multiple timing vectors, inputs the timing vector sequence into the joint extraction model, and determines multiple entities corresponding to the target area and the entity relationships between each entity (S306). Specifically, the joint extraction model includes an entity recognition network and an entity relationship extraction network. Among them, the entity recognition network is used to determine the entity type label sequence corresponding to each timing vector from the timing vector sequence, so as to determine multiple entities corresponding to the target area. And among them, the entity recognition network includes a first LSTM unit and a CRF layer. The entity relationship extraction network is used to determine the entity relationship label sequence corresponding to each timing vector sequence from the timing vector sequence, so as to determine multiple entity relationships corresponding to the target area. And among them, the entity relationship extraction network includes a second LSTM unit, a local attention mechanism, a fully connected layer, and a softmax classifier. The above content will be described in detail later, so it will not be elaborated here.

[0033] Finally, the processor constructs a knowledge graph corresponding to the target area by using multiple entities and multiple entity relationships (S308). That is, the processor represents the nodes in the knowledge graph as entities, represents the edges as entity relationships, and constructs a knowledge graph corresponding to the target area.

[0034] As described in the background art, since SAR images do not rely on optical imaging but image by emitting electromagnetic waves, SAR satellites can work all-weather and are not restricted by lighting conditions. In addition, the microwave signals emitted by SAR satellites can penetrate clouds and soot and are applicable to most regions. However, SAR images also have many disadvantages. For example, due to the slant-range imaging method used in SAR, geometric distortions such as perspective contraction, layover, and shadow exist in SAR images, and the interpretation difficulty is relatively high. Therefore, the pixel-level features of SAR images do not support direct use by operators and require professionals to be familiar with radar scattering mechanisms and the electromagnetic characteristics of ground objects. That is, operators cannot directly and visually analyze SAR images and need to process them professionally before they can use SAR images, which is extremely inconvenient for operators to analyze and utilize SAR images.

[0035] In view of this, the present application provides a method for generating a knowledge graph based on SAR satellites. And referring to the above content, different from the prior art, the present application does not directly use SAR images. Instead, considering that the synthetic aperture length in SAR images is related to time and the SAR satellite moves continuously in the azimuth direction (corresponding to the synthetic aperture length), a coordinate system is constructed with the synthetic aperture length related to time series as the abscissa and the slant range as the ordinate. Based on the SAR image and the constructed coordinate system, a plurality of time series vectors corresponding to the synthetic aperture length are determined. Further, in the case of determining the time series vector sequence, the time series vector sequence is input into a pre-trained joint extraction model, and then the entities and entity relationships corresponding to the target area can be extracted. And in the case of extracting the entities and entity relationships corresponding to the target area, a knowledge graph corresponding to the target area is constructed by using a plurality of entities and a plurality of entity relationships that are multi-win with the target area.

[0036] Thus, in the case of determining the knowledge graph corresponding to the target area, operators can directly analyze the target area with the visual knowledge graph, thereby significantly improving the comprehensibility, computability, and applicability of SAR images and facilitating the analysis and utilization of SAR images by operators. Furthermore, it solves the technical problem in the prior art that operators cannot directly and visually analyze SAR images and need to process them professionally before they can use SAR images, which is extremely inconvenient for operators to analyze and utilize SAR images.

[0037] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and inputting the time series vector sequence into the joint extraction model and determining the operations of multiple entities corresponding to the target region include: inputting the time series vector sequence into the first LSTM unit and using the first LSTM unit to output a first enhanced time series vector sequence; inputting the first enhanced time series vector sequence into the CRF unit and using the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, where the entity type label sequence includes probabilities corresponding to each entity type; and determining multiple entities corresponding to the target region based on the entity type label sequence corresponding to each time series vector.

[0038] Specifically, the joint extraction model includes entity extraction and entity relationship extraction of the time series vector sequence. The entity recognition network includes a first LSTM unit and a CRF unit. The first LSTM unit can solve the long-distance dependence problem and capture complete context features. Thus, when inputting the time series vector sequence into the first LSTM unit, a first enhanced time series vector sequence can be output.

[0039] After that, input the first enhanced time series vector sequence into the CRF unit, and use the CRF unit to select the optimal label path through global optimization, so as to obtain an entity type label sequence corresponding to the time series vector sequence. The entity type label sequence includes probabilities corresponding to each entity type. For example, the time series vector sequence includes multiple time series vectors . The entity type label sequence includes multiple entity type label vectors , where represents the probability corresponding to the time series vector and each entity type; represents the probability corresponding to the time series vector and each entity type; and so on; represents the probability corresponding to the time series vector and each entity type.

[0040] For example, the entity type label vector , represents the probability corresponding to entity type 1, represents the probability corresponding to entity type 2,..., represents the probability corresponding to entity type r.

[0041] And when each probability in each entity type label vector is greater than a preset threshold, determine the entity type corresponding to the corresponding time series vector. For example, in the entity type label vector the probability corresponding to entity type 2 Greater than a preset threshold , so that entity type 2 is the entity type corresponding to the time series vector .

[0042] For another example, the probability corresponding to entity type 2 in the entity type label vector and the probability corresponding to entity type 3 are greater than a preset threshold , so that entity type 2 and entity type 3 are the entity types corresponding to the time series vector .

[0043] Therefore, when the processor determines the entity types corresponding to each time series vector, it can determine multiple entity types corresponding to the target area.

[0044] Optionally, the joint extraction model includes an entity relationship extraction network, where the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier. And the operation of inputting the time series vector sequence into the joint extraction model and determining multiple entity relationships corresponding to the target area includes: inputting the entities corresponding to each time series vector and the time series vector sequence into the second LSTM unit and the local attention mechanism layer, and outputting a second enhanced time series vector sequence; inputting the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and outputting an entity relationship label sequence corresponding to each time series vector, where the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and determining multiple entity relationships corresponding to the target area based on the entity relationship label sequence corresponding to each time series vector.

[0045] Specifically, the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier. The second LSTM unit can solve the long-distance dependence problem and capture complete context features. The local attention mechanism layer can automatically identify and weight the local semantic units around the entity pair, rather than all the words in the whole sentence, so as to more efficiently capture the semantic clues around the entity by restricting the attention range. Therefore, when the processor inputs the entities corresponding to each time series vector and the time series vector sequence into the second LSTM unit and the local attention mechanism layer, it can output a second enhanced time series vector sequence.

[0046] ​Further, the processor inputs the second enhanced timing vector sequence into the fully connected layer, and the fully connected layer maps the high-level features to the specific entity relationship category space, so as to output the entity relationship vector sequence corresponding to the timing vector sequence. Finally, the processor inputs the entity relationship vector sequence corresponding to the timing vector sequence into the softmax classifier, so that the softmax classifier can output the entity relationship label sequence corresponding to the entity relationship vector sequence. The entity relationship label sequence includes the probabilities corresponding to each entity relationship type. For example, the timing vector sequence includes multiple timing vectors . The entity relationship label sequence includes multiple entity relationship label vectors , where represents the probability corresponding to the timing vector and each entity relationship type; represents the probability corresponding to the timing vector and each entity relationship type; and so on; represents the probability corresponding to the timing vector and each entity relationship type.

[0047] For example, the entity relationship label vector , represents the probability corresponding to entity relationship type 1, represents the probability corresponding to entity relationship type 2,... represents the probability corresponding to entity relationship type s.

[0048] And when each probability in each entity relationship label vector is greater than a preset threshold, the entity relationship type corresponding to the corresponding timing vector is determined. For example, the probability corresponding to entity relationship type 2 in the entity relationship label vector is greater than the preset threshold , so that entity relationship type 2 is the entity relationship type corresponding to the timing vector .

[0049] For another example, the probability corresponding to entity relationship type 2 and the probability corresponding to entity relationship type 3 in the entity relationship label vector are greater than the preset threshold , so that entity relationship type 2 and entity relationship type 3 are the entity relationship types corresponding to the timing vector .

[0050] Thus, when the processor determines the entity relationship types corresponding to each timing vector, multiple entity relationship types corresponding to the target region can be determined.

[0051] Optionally, the operation of constructing a knowledge graph corresponding to the target area by using multiple entities and multiple entity relationships includes: generating multiple triples based on the multiple entities and the entity relationships between the entities; and constructing a knowledge graph corresponding to the target area based on the multiple triples.

[0052] Specifically, in the case of determining the entities and entity relationships corresponding to each time series vector, the entities and the entity relationship descriptions are combined into triples. For example, the time series vector corresponds to the entity and the entity , and the time series vector corresponds to the entity relationship , then the triple is .

[0053] Thus, based on the above operations, the processor can determine multiple triples corresponding to each time series vector in the time series vector sequence . Among them, each time series vector may have one triple or multiple triples.

[0054] Thus, a knowledge graph corresponding to the target area can be constructed in the above manner, facilitating the operator to analyze and utilize the SAR image corresponding to the target area according to the knowledge graph.

[0055] Thus, according to the first aspect of this embodiment, the technical effect of facilitating the operator to analyze and utilize the SAR image corresponding to the target area by using the visual knowledge graph is achieved.

[0056] In addition, as shown in Figure 1 , according to the second aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein, when the program runs, the method described in any one of the above is executed by a processor.

[0057] Thus, according to this embodiment, the technical effect of facilitating the operator to analyze and utilize the SAR image corresponding to the target area by using the visual knowledge graph is achieved.

[0058] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0059] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in various embodiments of the present invention.

[0060] Embodiment 2 Figure 4 Fig. shows a knowledge graph generation device 400 based on an SAR satellite according to the present embodiment. The device corresponds to the method according to Embodiment 1. Refer to Figure 4 As shown, the device includes: an SAR image acquisition module 410, configured to use an SAR satellite to acquire an SAR image corresponding to a target area and construct a coordinate system corresponding to the SAR image, where the abscissa of the coordinate system represents the synthetic aperture length related to the time sequence, and the ordinate of the coordinate system represents the slant range; a time sequence vector determination module 420, configured to use the SAR image and according to the coordinate system, determine a plurality of time sequence vectors corresponding to each synthetic aperture length, where the time sequence vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time sequence vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; an entity and entity relationship extraction module 430, configured to determine a time sequence vector sequence corresponding to the plurality of time sequence vectors, input the time sequence vector sequence into a joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationship between each entity; and a knowledge graph construction module 440, configured to use the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, where the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0061] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the entity and entity relationship extraction module 430 includes: a first enhancement module for inputting a sequence of temporal vectors into the first LSTM unit and outputting a first enhanced sequence of temporal vectors using the first LSTM unit; a first label sequence output module for inputting the first enhanced sequence of temporal vectors into the CRF unit and outputting an entity type label sequence corresponding to the sequence of temporal vectors using the CRF unit, where the entity type label sequence includes probabilities corresponding to each entity type; and an entity determination module for determining a plurality of entities corresponding to the target region based on the entity type label sequences corresponding to each temporal vector.

[0062] Optionally, the joint extraction model includes an entity relationship extraction network, where the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier, and the entity and entity relationship extraction module 430 includes: a second enhancement module for inputting a sequence of temporal vectors into the second LSTM unit and the local attention mechanism layer and outputting a second enhanced sequence of temporal vectors; a second label sequence output module for inputting the second enhanced sequence of temporal vectors into the fully connected layer and the softmax classifier and outputting an entity relationship label sequence corresponding to each temporal vector, where the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and an entity relationship determination module for determining a plurality of entity relationships corresponding to the target region based on the entity relationship label sequences corresponding to each temporal vector.

[0063] Optionally, the knowledge graph construction module 440 includes: a triple generation module for generating a plurality of triples based on a plurality of entities and the entity relationships between each entity; and a knowledge graph construction sub-module for constructing a knowledge graph corresponding to the target region based on the plurality of triples.

[0064] Thus, according to this embodiment, the technical effect of facilitating the operator to analyze and utilize the SAR image corresponding to the target region using the visual knowledge graph is achieved.

[0065] Embodiment 3 Figure 5 Shows a knowledge graph generation device 500 based on a SAR satellite according to this embodiment, and the device corresponds to the method according to Embodiment 1. Refer to Figure 5As shown, the device includes: a processor 510; and a memory connected to the processor 510 for providing instructions for the processor 510 to process the following steps: acquiring a SAR image corresponding to a target area by using a SAR satellite and constructing a coordinate system corresponding to the SAR image, where the abscissa of the coordinate system represents the synthetic aperture length related to the time sequence, and the ordinate of the coordinate system represents the slant range; determining a plurality of time sequence vectors corresponding to each synthetic aperture length by using the SAR image and according to the coordinate system, where the time sequence vectors are used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time sequence vectors is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges; determining a time sequence vector sequence corresponding to the plurality of time sequence vectors, inputting the time sequence vector sequence into a joint extraction model, and determining a plurality of entities corresponding to the target area and the entity relationships between the entities; and constructing a knowledge graph corresponding to the target area by using the plurality of entities and the plurality of entity relationships, where the nodes in the knowledge graph represent entities, and the edges represent entity relationships.

[0066] Optionally, the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the operation of inputting the time sequence vector sequence into the joint extraction model and determining a plurality of entities corresponding to the target area includes: inputting the time sequence vector sequence into the first LSTM unit and using the first LSTM unit to output a first enhanced time sequence vector sequence; inputting the first enhanced time sequence vector sequence into the CRF unit and using the CRF unit to output an entity type label sequence corresponding to the time sequence vector sequence, where the entity type label sequence includes probabilities corresponding to each entity type; and determining a plurality of entities corresponding to the target area based on the entity type label sequence corresponding to each time sequence vector.

[0067] Optionally, the joint extraction model includes an entity relationship extraction network, where the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier, and the operation of inputting the time sequence vector sequence into the joint extraction model and determining a plurality of entity relationships corresponding to the target area includes: inputting the entities corresponding to each time sequence vector and the time sequence vector sequence into the second LSTM unit and the local attention mechanism layer and outputting a second enhanced time sequence vector sequence; inputting the second enhanced time sequence vector sequence into the fully connected layer and the softmax classifier and outputting an entity relationship label sequence corresponding to each time sequence vector, where the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and determining a plurality of entity relationships corresponding to the target area based on the entity relationship label sequence corresponding to each time sequence vector.

[0068] Optionally, the operation of constructing a knowledge graph corresponding to a target area by using multiple entities and multiple entity relationships includes: generating a plurality of triples based on the multiple entities and the entity relationships between the entities; and constructing a knowledge graph corresponding to the target area based on the plurality of triples.

[0069] Therefore, according to this embodiment, the technical effect of facilitating the operator to analyze and utilize the SAR image corresponding to the target area by using the visual knowledge graph is achieved.

[0070] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0071] In the above embodiments of the present invention, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0072] In several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in an electrical or other form.

[0073] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0074] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0075] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), mobile hard disks, magnetic disks, or optical discs.

[0076] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A method for generating a knowledge graph based on SAR satellites, characterized in that Comprising: Collecting SAR images corresponding to a target area by using an SAR satellite, and constructing a coordinate system corresponding to the SAR images, wherein the abscissa of the coordinate system represents the synthetic aperture length related to the time series, and the ordinate of the coordinate system represents the slant range; Using the SAR images and according to the coordinate system, determining a plurality of time series vectors corresponding to each synthetic aperture length, wherein the time series vectors are used to indicate the distribution of the signal intensities of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time series vectors is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to the different slant ranges; Determining a time series vector sequence corresponding to the plurality of time series vectors, inputting the time series vector sequence into a joint extraction model, and determining a plurality of entities corresponding to the target area and the entity relationships between the entities; And Using the plurality of entities and the plurality of entity relationships to construct a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

2. The method according to claim 1, wherein Wherein the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the operation of inputting the time series vector sequence into the joint extraction model and determining a plurality of entities corresponding to the target area includes: Inputting the time series vector sequence into the first LSTM unit, and using the first LSTM unit to output a first enhanced time series vector sequence; Inputting the first enhanced time series vector sequence into the CRF unit, and using the CRF unit to output an entity type label sequence corresponding to the time series vector sequence, wherein the entity type label sequence includes probabilities corresponding to each entity type; and Based on the entity type label sequence corresponding to each time series vector, determining a plurality of entities corresponding to the target area.

3. The method according to claim 2, characterized in that Wherein the joint extraction model includes an entity relationship extraction network, the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer and a softmax classifier, and the operation of inputting the time series vector sequence into the joint extraction model and determining a plurality of entity relationships corresponding to the target area includes: Inputting the entities corresponding to each time series vector and the time series vector sequence into the second LSTM unit and the local attention mechanism layer, and outputting a second enhanced time series vector sequence; Inputting the second enhanced time series vector sequence into the fully connected layer and the softmax classifier, and outputting an entity relationship label sequence corresponding to each time series vector, wherein the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and Based on the entity relationship label sequence corresponding to each time series vector, determining a plurality of entity relationships corresponding to the target area.

4. The method according to claim 3, characterized in that, The operation of constructing a knowledge graph corresponding to the target area by using the multiple entities and multiple entity relationships includes: generating a plurality of triples based on the multiple entities and the entity relationships between the respective entities; and constructing a knowledge graph corresponding to the target area based on the plurality of triples.

5. A storage medium, characterized in that, The storage medium includes a stored program, wherein the method according to any one of claims 1 to 4 is executed by a processor when the program runs.

6. A knowledge graph generation device based on an SAR satellite, characterized in that Including: An SAR image acquisition module, configured to use an SAR satellite to acquire an SAR image corresponding to a target area, and construct a coordinate system corresponding to the SAR image, wherein the abscissa of the coordinate system represents the synthetic aperture length related to the time sequence, and the ordinate of the coordinate system represents the slant range; A time sequence vector determination module, configured to use the SAR image and according to the coordinate system, determine a plurality of time sequence vectors corresponding to each synthetic aperture length, wherein the time sequence vector is used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the time sequence vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to the different slant ranges; An entity and entity relationship extraction module, configured to determine a time sequence vector sequence corresponding to the plurality of time sequence vectors, input the time sequence vector sequence into a joint extraction model, and determine a plurality of entities corresponding to the target area and the entity relationships between the respective entities; And A knowledge graph construction module, configured to use the multiple entities and multiple entity relationships to construct a knowledge graph corresponding to the target area, wherein the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

7. The device according to claim 6, characterized in that, Wherein the joint extraction model includes an entity recognition network, the entity recognition network includes a first LSTM unit and a CRF unit connected to the first LSTM unit, and the entity and entity relationship extraction module includes: A first enhancement module, configured to input the time sequence vector sequence into the first LSTM unit, and use the first LSTM unit to output a first enhanced time sequence vector sequence; A first label sequence output module, configured to input the first enhanced time sequence vector sequence into the CRF unit, and use the CRF unit to output an entity type label sequence corresponding to the time sequence vector sequence, wherein the entity type label sequence includes probabilities corresponding to each entity type; and An entity determination module, configured to determine a plurality of entities corresponding to the target area based on the entity type label sequences corresponding to the respective time sequence vectors.

8. The device according to claim 7, characterized in that, Wherein the joint extraction model includes an entity relationship extraction network, wherein the entity relationship extraction network includes a second LSTM unit, a local attention mechanism layer, a fully connected layer, and a softmax classifier, and the entity and entity relationship extraction module includes: A second enhancement module, configured to input the time sequence vector sequence into the second LSTM unit and the local attention mechanism layer, and output a second enhanced time sequence vector sequence; A second label sequence output module, configured to input the second enhanced timing vector sequence into the fully connected layer and the softmax classifier, and output an entity relationship label sequence corresponding to each timing vector, where the entity relationship label sequence includes probabilities corresponding to each entity relationship type; and An entity relationship determination module, configured to determine multiple entity relationships corresponding to the target region based on the entity relationship label sequences corresponding to each timing vector.

9. The device according to claim 8, characterized in that, The knowledge graph construction module includes: a triple generation module, configured to generate multiple triples based on the multiple entities and the entity relationships between each entity; and A knowledge graph construction sub-module, configured to construct a knowledge graph corresponding to the target region based on the multiple triples.

10. A knowledge graph generation device based on an SAR satellite, characterized in that, Comprising: A processor; And A memory, connected to the processor, for providing instructions for the processor to perform the following processing steps: Collecting a SAR image corresponding to a target region by using a SAR satellite, and constructing a coordinate system corresponding to the SAR image, where the abscissa of the coordinate system represents the synthetic aperture length related to timing, and the ordinate of the coordinate system represents the slant range; Using the SAR image and according to the coordinate system, determining multiple timing vectors corresponding to each synthetic aperture length, where the timing vectors are used to indicate the distribution of the signal intensity of the electromagnetic wave backscattering corresponding to different slant ranges, and each element in the timing vector is used to represent the signal intensity of the electromagnetic wave backscattering corresponding to the different slant ranges; Determining a timing vector sequence corresponding to the multiple timing vectors, inputting the timing vector sequence into a joint extraction model, and determining multiple entities corresponding to the target region and the entity relationships between each entity; and And Constructing a knowledge graph corresponding to the target region by using the multiple entities and the multiple entity relationships, where the nodes in the knowledge graph represent the entities, and the edges represent the entity relationships.

Images