Method and system for constructing semantic model of space non-cooperative target based on graph model representation

Abstract

The application provides a space non-cooperative target semantic model construction method and system based on a graph model representation, comprising the following steps: step 1: determining the space quadrant of a target ontology through target ontology 3D bounding box vertex identification; step 2: estimating the possibility of each load belonging to each target ontology space quadrant according to the position information of various loads on an image identified by a target recognition deep learning model; step 3: forming a target semantic model under a single view angle and using an undirected graph to describe, thereby forming a target semantic model undirected graph; and step 4: based on a graph matching method, integrating the target semantic model undirected graphs observed from each view to form a target semantic model with consistency under multiple views. The application fuses the non-complete target structure graph models under each view angle through the graph matching mode, and solves the problem of difficult alignment of target information under discrete view angles.

Description

Technical Field

[0001] This invention relates to the field of model building technology, and more specifically, to a method and system for constructing a spatial non-cooperative target semantic model based on graph model representation. Background Technology

[0002] Non-cooperative targets in space are considered to be space targets that cannot be communicated with or controlled. Perception of space targets largely depends on comparing a large number of observation results with existing databases. When faced with targets that do not exist in the database, the database needs to be expanded based on the observation results.

[0003] Patent document CN106874942A (application number: CN201710044816.8) discloses a method for rapidly constructing a target model based on regular expression semantics, belonging to the field of machine vision and pattern recognition. First, the image containing the object to be identified is preprocessed to improve the quality of feature extraction. Then, the local features of the image target are extracted using the Harris feature detection algorithm. Finally, a target matching model for the identified object is described by combining the derived regular expression semantics with a predefined image library. However, this patent cannot solve the existing technical problems.

[0004] Therefore, it is very important to obtain structural information of the target from observations using visible light information. This invention aims to construct a semantic model that can characterize the structural composition of a space non-cooperative target aircraft based on deep learning methods such as target recognition, semantic segmentation, and monocular 6D pose estimation. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for constructing a spatial non-cooperative target semantic model based on graph model representation.

[0006] The spatial non-cooperative target semantic model construction method based on graph model representation provided by the present invention includes:

[0007] Step 1: Determine the spatial quadrant of the target body by identifying the vertices of the 3D bounding box of the target body;

[0008] Step 2: Based on the location information of various payloads on the image identified by the target recognition deep learning model, estimate the probability that each payload belongs to each target body spatial quadrant;

[0009] Step 3: Form a target semantic model from a single perspective and describe it using an undirected graph to form an undirected graph of the target semantic model;

[0010] Step 4: Based on the graph matching method, the undirected graphs of the target semantic model obtained from each view observation are integrated to form a target semantic model with consistency under multiple views.

[0011] Preferably, step 2 includes: dividing the spatial region of the target body into six spatial quadrants corresponding to the six faces of the cuboid, performing semantic segmentation on the component recognition results, calculating the overlap rate between the pixel region where the target component is located and the six spatial quadrants, as well as the geometric visibility of the region, thereby determining the probability of the target component belonging to the six spatial quadrants.

[0012] Preferably, step 3 includes:

[0013] Step 3.1: Establish a spatial description diagram of the target, take the target body as the central node, and set the six surrounding nodes connected to it to represent the six spatial orientations with the center of the body as the origin, which become the spatial quadrant nodes of the non-cooperative target configuration, where adjacent spatial quadrant nodes are connected by edges.

[0014] Step 3.2: For a certain frame of image, the components detected by the component recognition algorithm are assigned to each spatial quadrant node after spatial location calculation. The component node represented by the component type is connected to the spatial quadrant node to which it belongs by an edge, forming a graph description of the target semantic model containing component information.

[0015] Step 3.3: Detect the sequence image, solve the spatial position of the repeating components, assign them to the nodes of each spatial quadrant, and obtain the sequence of graph description of the target semantic model.

[0016] Preferably, when constructing the graph model, the nodes represented by the key loads are given specificity and uniqueness. Specifically, for representative components identified from a portion of the views, a unique correspondence is established between the nodes representing the spatial quadrants in the graph model and their corresponding load nodes. This eliminates the differences in spatial structure detection for a non-cooperative target across multiple views, thereby enabling pre-matching of representative component nodes in the graph model obtained from different observation views and retaining observation results with singularities or uncertainties during the pre-matching process.

[0017] Preferably, graph matching is performed on the target semantic graph model of each view to achieve the fusion of multi-view observation results, specifically as follows:

[0018] Graph matching is based on the similarity of nodes and edges. For a component node, its description includes the component type and identification confidence. At the same time, it takes into account the information of the spatial quadrant nodes that are adjacent to and opposite to the component node, and assigns corresponding weights to each piece of information. In the matching process, inexact graph matching is used to find an optimization problem that maximizes the sum of node similarity and edge similarity.

[0019] Given two graphs G1 = {P1, Q1, G1, H1} and G2 = {P2, Q2, G2, H2}, where P represents the characteristics of a node; Q represents the characteristics of an edge; and G and H represent the correspondence between nodes and edges; define two incidence matrices K. p K q Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges. The expression is:

[0020]

[0021] The first term represents the sum of similarities between matched nodes; the second term represents the sum of similarities between matched edges; and X represents the matching relationship between nodes in the two graphs. This indicates that the i1th node in graph G1 matches the i2th node in graph G2, and the matching relationship between the nodes satisfies the one-to-one correspondence constraint. This indicates that the c1th edge in graph G1 starts from node i1; This indicates that the c1th edge in graph G1 terminates at node j1; and Combined, they represent the correspondence between nodes and edges. This indicates that the c1th edge in graph G1 points from node i1 to node j1. This indicates that the c1th edge in graph G1 does not point from node i1 to node j1;

[0022] The matching degree between the two graph models is calculated using a 24-times enumeration method, and J is obtained. gm The maximum result of (X) is obtained to obtain a suitable matching relationship, and finally a relatively complete target semantic model is formed.

[0023] The spatial non-cooperative target semantic model construction system based on graph model representation provided by the present invention includes:

[0024] Module M1: Determines the spatial quadrant of the target body by recognizing the vertices of the 3D bounding box of the target body;

[0025] Module M2: Based on the location information of various payloads on the image identified by the target recognition deep learning model, estimate the probability that each payload belongs to each target body spatial quadrant;

[0026] Module M3: Forms a target semantic model from a single perspective and describes it using an undirected graph, forming an undirected graph of the target semantic model;

[0027] Module M4: Based on graph matching, this method integrates the undirected graphs of the target semantic model obtained from various views to form a consistent target semantic model across multiple views.

[0028] Preferably, the module M2 includes: dividing the spatial region of the target body into six spatial quadrants corresponding to the six faces of the cuboid, performing semantic segmentation on the component recognition results, calculating the overlap rate between the pixel region where the target component is located and the six spatial quadrants, as well as the geometric visibility of the region, thereby determining the probability of the target component belonging to the six spatial quadrants.

[0029] Preferably, the module M3 includes:

[0030] Module M3.1: Establish the spatial description graph of the target, take the target body as the central node, and set the six surrounding nodes connected to it to represent the six spatial orientations with the center of the body as the origin, which become the spatial quadrant nodes of the non-cooperative target configuration, where adjacent spatial quadrant nodes are connected by edges.

[0031] Module M3.2: For a given frame of image, the components detected by the component recognition algorithm are assigned to various spatial quadrant nodes after spatial location calculation. The component node represented by the component type is connected to the spatial quadrant node to which it belongs by an edge, forming a graph description of the target semantic model containing component information.

[0032] Module M3.3: Detects sequential images, calculates the spatial location of repeating components, assigns them to nodes in each spatial quadrant, and obtains the sequence of graph descriptions of the target semantic model.

[0033] Preferably, when constructing the graph model, the nodes represented by the key loads are given specificity and uniqueness. Specifically, for representative components identified from a portion of the views, a unique correspondence is established between the nodes representing the spatial quadrants in the graph model and their corresponding load nodes. This eliminates the differences in spatial structure detection for a non-cooperative target across multiple views, thereby enabling pre-matching of representative component nodes in the graph model obtained from different observation views and retaining observation results with singularities or uncertainties during the pre-matching process.

[0034] Preferably, graph matching is performed on the target semantic graph model of each view to achieve the fusion of multi-view observation results, specifically as follows:

[0035] Graph matching is based on the similarity of nodes and edges. For a component node, its description includes the component type and identification confidence. At the same time, it takes into account the information of the spatial quadrant nodes that are adjacent to and opposite to the component node, and assigns corresponding weights to each piece of information. In the matching process, inexact graph matching is used to find an optimization problem that maximizes the sum of node similarity and edge similarity.

[0036] Given two graphs G1 = {P1, Q1, G1, H1} and G2 = {P2, Q2, G2, H2}, where P represents the characteristics of a node; Q represents the characteristics of an edge; and G and H represent the correspondence between nodes and edges; define two incidence matrices K. p K q Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges. The expression is:

[0037]

[0038] The first term represents the sum of similarities between matched nodes; the second term represents the sum of similarities between matched edges; and X represents the matching relationship between nodes in the two graphs. This indicates that the i1th node in graph G1 matches the i2th node in graph G2, and the matching relationship between the nodes satisfies the one-to-one correspondence constraint. This indicates that the c1th edge in graph G1 starts from node i1; This indicates that the c1th edge in graph G1 terminates at node j1; and Combined, they represent the correspondence between nodes and edges. This indicates that the c1th edge in graph G1 points from node i1 to node j1. This indicates that the c1th edge in graph G1 does not point from node i1 to node j1;

[0039] The matching degree between the two graph models is calculated using a 24-times enumeration method, and J is obtained. gm The maximum result of (X) is obtained to obtain a suitable matching relationship, and finally a relatively complete target semantic model is formed.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] 1) This invention uses a graph model to represent the target structure composition obtained through reasoning by multiple visual algorithms under multiple views, forming a compact target semantic model expression;

[0042] 2) This invention combines domain knowledge of common spatial target structures and establishes a unique correspondence between nodes representing spatial quadrants in the graphical model and their corresponding load nodes through some representative loads, thereby eliminating the differences in spatial structure detection for a non-cooperative target across multiple views.

[0043] 3) This invention fuses incomplete target structure graph models from various perspectives through graph matching, thus solving the problem of difficult alignment of target information from discrete perspectives. Attached Figure Description

[0044] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0045] Figure 1a and Figure 1b These are the 3D bounding box recognition of the target body and the component recognition images of the target;

[0046] Figure 2a and Figure 2b These are spatial region maps of the target ontology obtained from the 3D bounding boxes of the target ontology.

[0047] Figure 3 A spatial description diagram of the target;

[0048] Figure 4 Image sequences and detection results obtained from multi-view observation;

[0049] Figures 5a-5d A multi-set graph model diagram formed from multiple sets of observation result sequences;

[0050] Figure 6 The graph model is matched and combined from multiple sets of observation results to form a complete target graph model. Detailed Implementation

[0051] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0052] Example 1:

[0053] This invention provides a method for constructing a spatial non-cooperative target semantic model based on graph model representation under multi-view observation. By identifying the vertices of the 3D bounding box of the target ontology, the spatial quadrant of the target ontology is determined. Based on the positional information of various payloads on the image identified by the target recognition deep learning model, the probability of each payload belonging to its respective target ontology spatial quadrant is estimated, forming a target semantic model under a single view. This model is then described using an undirected graph, forming an undirected target semantic model graph. Finally, based on a graph matching method, the undirected target semantic model graphs obtained from each view observation are integrated to form a consistent target semantic model under multiple views.

[0054] The specific steps are as follows:

[0055] Step 1: Identify the 3D bounding box vertices and components of the target body in consecutive frames from a single viewpoint. Perform coarse-grained reasoning on the component mounting areas to form a target semantic model described in an undirected graph format, such as... Figure 1a and Figure 1b As shown.

[0056] Step 2: Divide the spatial region of the aircraft body into six spatial quadrants corresponding to the six faces of the cuboid. Based on the semantic segmentation results in the component identification results, calculate the overlap rate between the pixel region of the target component's combat zone and the six spatial quadrants, as well as the geometric visibility of that region, thereby determining the probability of the target component's affiliation within the six spatial quadrants. Figure 2a and Figure 2b .

[0057] Step 3: After identifying the 3D bounding box of the target body, dividing the body space into regions, and identifying the payload attribution for each frame of the target image, this scheme represents the target semantic model of each frame as an undirected graph structure as follows:

[0058] First, establish a spatial description diagram of the target: The target itself is taken as the central node, and six surrounding nodes connected to it represent the six spatial orientations (up, down, left, right, front, and back) with the target's center as the origin. These become the "spatial quadrant nodes" of the non-cooperative target configuration. Adjacent spatial quadrant nodes are connected by edges, such as... Figure 3 .

[0059] For a given frame of image, the components detected by the component recognition algorithm are assigned to various spatial quadrant nodes after spatial location calculation. The "component node" represented by the component type is connected to the spatial quadrant node to which it belongs by an edge, forming a graph description of the target semantic model containing component information.

[0060] (Taking major, common components as examples, where P represents the sail, A represents the antenna, N represents the engine nozzle, and C represents optical equipment.)

[0061] After detecting the sequence of images and repeating the above-described component spatial location calculation and assignment to nodes in each spatial quadrant, a graph description of the target semantic model is obtained, such as... Figure 4 .

[0062] Step 4: After completing the graphical modeling from various perspectives, this approach considers that most spacecraft follow similar design logic. Therefore, this method combines domain knowledge of key payloads (such as significant design features like dual-sail symmetry design, single antenna, dual antennas, and single main engine) to apply specificity and uniqueness to the nodes represented by key payloads when constructing the graphical model.

[0063] For representative components identified from a subset of perspectives, a unique correspondence is established between the nodes representing spatial quadrants in the graphical model and their corresponding load nodes (e.g., the node P corresponding to the sail is artificially set as node 1 or 2, the node corresponding to the main engine nozzle is artificially set as node 6, and the node corresponding to the antenna is artificially set as node 5, as shown in Figure 5). This can eliminate the differences in spatial structure detection for a non-cooperative target across multiple views.

[0064] Based on this, this method can first perform pre-matching on representative component nodes in the graphical model obtained from different observation perspectives, while retaining the observation results with singularity or uncertainty in the pre-matching process.

[0065] Step 5: Due to the limitations of the observation perspective, the semantic model graph description of each target is not complete, but only contains the component detection results from that perspective. At the same time, the pre-matching process also retains the observation results with singularity or uncertainty.

[0066] Therefore, this solution uses graph matching on the target semantic graph model of each view to achieve the fusion of multi-view observation results:

[0067] Graph matching is based on the similarity of nodes and edges. For component nodes, their descriptions inherently include the component type and identification confidence. Furthermore, information about adjacent and opposite spatial quadrant nodes to the directly connected spatial quadrant node is considered. All of the above are combined to form the information description of each component node, and each piece of information is assigned a corresponding weight S. i The weight is multiplied by the node information to obtain the final node information, which is then used in the similarity calculation of the node information. The expression is:

[0068] {S1*Class,S2*confidence,S3*[Node1{Class,confidence}],...,S7

[0069] *[Node5{Class,confidence}]}

[0070] During the matching process, this method employs inexact graph matching to find an optimization problem that maximizes the sum of node similarity and edge similarity.

[0071] Given two graphs G1 = {P1, Q1, G1, H1} and G2 = {P2, Q2, G2, H2}, and define two incidence matrices K. p ,K q Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges.

[0072]

[0073] The first term represents the sum of similarities between matched nodes, and the second term represents the sum of similarities between matched edges. X represents the matching relationship between nodes in the two graphs. This means that the i1th node in graph G1 matches the i2th node in graph G2, and the matching relationship between nodes must satisfy the one-to-one correspondence constraint.

[0074] In the task addressed by this method, since the target space is divided into 6 mutually perpendicular spaces, the matching degree between the two graph models can be calculated using 24 enumerations, and J can be obtained. gm The result with the largest (X) is used to obtain a suitable matching relationship, ultimately forming a relatively complete target semantic model, such as Figure 6 .

[0075] Example 2:

[0076] The present invention also provides a system for constructing a spatial non-cooperative target semantic model based on graph model representation. The system can be implemented by executing the process steps of the method for constructing a spatial non-cooperative target semantic model based on graph model representation. That is, those skilled in the art can understand the method for constructing a spatial non-cooperative target semantic model based on graph model representation as a preferred embodiment of the system for constructing a spatial non-cooperative target semantic model based on graph model representation.

[0077] The spatial non-cooperative target semantic model construction system based on graph model representation provided by the present invention includes: module M1: determining the spatial quadrant of the target ontology by recognizing the vertices of the 3D bounding box of the target ontology; module M2: estimating the probability that each load belongs to the spatial quadrant of each target ontology based on the position information of various loads on the image identified by the target recognition deep learning model; module M3: forming a target semantic model in a single view and describing it using an undirected graph to form an undirected graph of the target semantic model; module M4: integrating the undirected graphs of the target semantic model obtained from each view observation based on a graph matching method to form a target semantic model with consistency in multiple views.

[0078] The module M2 includes: dividing the spatial region of the target body into six spatial quadrants corresponding to the six faces of the cuboid, performing semantic segmentation on the component recognition results, calculating the overlap rate between the pixel region where the target component is located and the six spatial quadrants, as well as the geometric visibility of the region, thereby determining the probability of the target component belonging to the six spatial quadrants.

[0079] Module M3 includes: Module M3.1: Establishing a spatial description graph of the target, taking the target body as the central node, and setting six surrounding nodes connected to it to represent six spatial orientations with the center of the body as the origin, becoming spatial quadrant nodes of the non-cooperative target configuration, wherein adjacent spatial quadrant nodes are connected by edges; Module M3.2: For a certain frame of image, the components detected by the component recognition algorithm are assigned to each spatial quadrant node after spatial position calculation, and the component nodes represented by component types are connected to their respective spatial quadrant nodes by edges, forming a graph description of the target semantic model containing component information; Module M3.3: Detecting the sequence of images, repeating the spatial position calculation of components, assigning them to each spatial quadrant node, and obtaining a sequence of graph descriptions of the target semantic model.

[0080] When constructing the graph model, the nodes represented by the key loads are given specificity and uniqueness. Specifically, for representative components identified from a portion of the views, a unique correspondence is established between the nodes representing the spatial quadrants in the graph model and their corresponding load nodes. This eliminates the differences in spatial structure detection for a non-cooperative target across multiple views, thereby enabling pre-matching of representative component nodes in the graph model obtained from different observation views and retaining observation results with singularities or uncertainties during the pre-matching process.

[0081] Graph matching is performed on the target semantic graph models of each view to achieve the fusion of multi-view observation results, specifically as follows:

[0082] Graph matching is based on the similarity of nodes and edges. For a component node, its description includes the component type and identification confidence. At the same time, it takes into account the information of the spatial quadrant nodes that are adjacent to and opposite to the component node, and assigns corresponding weights to each piece of information. In the matching process, inexact graph matching is used to find an optimization problem that maximizes the sum of node similarity and edge similarity.

[0083] Given two graphs G1 = {P1, Q1, G1, H1} and G2 = {P2, Q2, G2, H2}, where P represents the characteristics of a node; Q represents the characteristics of an edge; and G and H represent the correspondence between nodes and edges; define two incidence matrices K. p K q Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges. The expression is:

[0084]

[0085] The first term represents the sum of similarities between matched nodes; the second term represents the sum of similarities between matched edges; and X represents the matching relationship between nodes in the two graphs. This indicates that the i1th node in graph G1 matches the i2th node in graph G2, and the matching relationship between the nodes satisfies the one-to-one correspondence constraint. This indicates that the c1th edge in graph G1 starts from node i1; This indicates that the c1th edge in graph G1 terminates at node j1; and Combined, they represent the correspondence between nodes and edges. This indicates that the c1th edge in graph G1 points from node i1 to node j1. This indicates that the c1th edge in graph G1 does not point from node i1 to node j1;

[0086] The matching degree between the two graph models is calculated using a 24-times enumeration method, and J is obtained. gm The maximum result of (X) is obtained to obtain a suitable matching relationship, and finally a relatively complete target semantic model is formed.

[0087] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0088] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for constructing a spatial non-cooperative target semantic model based on graph model representation, characterized in that, include: Step 1: Determine the spatial quadrant of the target body by identifying the vertices of the 3D bounding box of the target body; Step 2: Based on the location information of various payloads on the image identified by the target recognition deep learning model, estimate the probability that each payload belongs to each target body spatial quadrant; Step 3: Form a target semantic model from a single perspective and describe it using an undirected graph to form an undirected graph of the target semantic model; Step 4: Based on the graph matching method, the undirected graphs of the target semantic model obtained from each view observation are integrated to form a target semantic model with consistency under multiple views; Step 3 includes: Step 3.1: Establish a spatial description diagram of the target, take the target body as the central node, and set the six surrounding nodes connected to it to represent the six spatial orientations with the center of the body as the origin, which become the spatial quadrant nodes of the non-cooperative target configuration, where adjacent spatial quadrant nodes are connected by edges. Step 3.2: For a certain frame of image, the components detected by the component recognition algorithm are assigned to each spatial quadrant node after spatial location calculation. The component node represented by the component type is connected to the spatial quadrant node to which it belongs by an edge, forming a graph description of the target semantic model containing component information. Step 3.3: Detect the sequence images, solve the spatial location of the repeating components, and assign them to the nodes of each spatial quadrant to obtain the sequence of graph description of the target semantic model; When constructing the graph model, the nodes represented by the key loads are given specificity and uniqueness. Specifically, for representative components identified from a portion of the views, a unique correspondence is established between the nodes representing the spatial quadrants in the graph model and their corresponding load nodes. This eliminates the differences in spatial structure detection for a non-cooperative target across multiple views, thereby enabling pre-matching of representative component nodes in the graph model obtained from different observation views and retaining observation results with singularities or uncertainties during the pre-matching process.

2. The method for constructing a spatial non-cooperative target semantic model based on graph model representation according to claim 1, characterized in that, Step 2 includes: dividing the spatial region of the target body into six spatial quadrants corresponding to the six faces of the cuboid, performing semantic segmentation on the component recognition results, calculating the overlap rate between the pixel region where the target component is located and the six spatial quadrants, as well as the geometric visibility of the region, thereby determining the probability of the target component belonging to the six spatial quadrants.

3. The method for constructing a spatial non-cooperative target semantic model based on graph model representation according to claim 1, characterized in that, Graph matching is performed on the target semantic graph models of each view to achieve the fusion of multi-view observation results, specifically as follows: Graph matching is based on the similarity of nodes and edges. For a component node, its description includes the component type and identification confidence. At the same time, it takes into account the information of the spatial quadrant nodes that are adjacent to and opposite to the component node, and assigns corresponding weights to each piece of information. In the matching process, inexact graph matching is used to find an optimization problem that maximizes the sum of node similarity and edge similarity. Given two graphs , , where P represents the feature of the node; Q represents the characteristics of an edge; G and H represent the correspondence between nodes and edges; two incidence matrices are defined. , Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges. The expression is: The first term represents the sum of similarities between matched nodes; the second term represents the sum of similarities between matched edges. This indicates the matching relationship between nodes in two graphs. =1 indicates a graph The first in Nodes and graph The first in Each node is matched, and the matching relationship between nodes satisfies a one-to-one correspondence constraint. Representation diagram The first in An edge is formed by nodes To begin; Representation diagram The first in An edge is formed by nodes For the end; Combined, they represent the correspondence between nodes and edges. Representation diagram The first in An edge is formed by nodes Pointing to node , Representation diagram The first in An edge is formed by nodes Not pointing to a node ; The matching degree between the two graph models is calculated using a 24-times enumeration method, and the results are obtained. The goal is to achieve the best possible results, thereby obtaining appropriate matching relationships and ultimately forming a relatively complete target semantic model.

4. A system for constructing a spatial non-cooperative target semantic model based on graph model representation, characterized in that, include: Module M1: Determines the spatial quadrant of the target body by recognizing the vertices of the 3D bounding box of the target body; Module M2: Based on the location information of various payloads on the image identified by the target recognition deep learning model, estimate the probability that each payload belongs to each target body spatial quadrant; Module M3: Forms a target semantic model from a single perspective and describes it using an undirected graph, forming an undirected graph of the target semantic model; Module M4: Based on graph matching, this method integrates the undirected graphs of the target semantic model obtained from various views to form a consistent target semantic model across multiple views. The module M3 includes: Module M3.1: Establish the spatial description graph of the target, take the target body as the central node, and set the six surrounding nodes connected to it to represent the six spatial orientations with the center of the body as the origin, which become the spatial quadrant nodes of the non-cooperative target configuration, where adjacent spatial quadrant nodes are connected by edges. Module M3.2: For a given frame of image, the components detected by the component recognition algorithm are assigned to various spatial quadrant nodes after spatial location calculation. The component node represented by the component type is connected to the spatial quadrant node to which it belongs by an edge, forming a graph description of the target semantic model containing component information. Module M3.3: Detects sequential images, calculates the spatial location of repeating components, assigns them to nodes in each spatial quadrant, and obtains the sequence of graph descriptions of the target semantic model; When constructing the graph model, the nodes represented by the key loads are given specificity and uniqueness. Specifically, for representative components identified from a portion of the views, a unique correspondence is established between the nodes representing the spatial quadrants in the graph model and their corresponding load nodes. This eliminates the differences in spatial structure detection for a non-cooperative target across multiple views, thereby enabling pre-matching of representative component nodes in the graph model obtained from different observation views and retaining observation results with singularities or uncertainties during the pre-matching process.

5. The spatial non-cooperative target semantic model construction system based on graph model representation according to claim 4, characterized in that, The module M2 includes: dividing the spatial region of the target body into six spatial quadrants corresponding to the six faces of the cuboid, performing semantic segmentation on the component recognition results, calculating the overlap rate between the pixel region where the target component is located and the six spatial quadrants, as well as the geometric visibility of the region, thereby determining the probability of the target component belonging to the six spatial quadrants.

6. The spatial non-cooperative target semantic model construction system based on graph model representation according to claim 4, characterized in that, Graph matching is performed on the target semantic graph models of each view to achieve the fusion of multi-view observation results, specifically as follows: Graph matching is based on the similarity of nodes and edges. For a component node, its description includes the component type and identification confidence. At the same time, it takes into account the information of the spatial quadrant nodes that are adjacent to and opposite to the component node, and assigns corresponding weights to each piece of information. In the matching process, inexact graph matching is used to find an optimization problem that maximizes the sum of node similarity and edge similarity. Given two graphs , , where P represents the feature of the node; Q represents the characteristics of an edge; G and H represent the correspondence between nodes and edges; two incidence matrices are defined. , Let represent the similarity between nodes and between edges in two graphs. This transforms the graph matching problem into finding the optimal matching relationship between two graphs, maximizing the sum of the similarities between nodes and edges. The expression is: The first term represents the sum of similarities between matched nodes; the second term represents the sum of similarities between matched edges. This indicates the matching relationship between nodes in two graphs. =1 indicates a graph The first in Nodes and graph The first in Each node is matched, and the matching relationship between nodes satisfies a one-to-one correspondence constraint. Representation diagram The first in An edge is formed by nodes To begin; Representation diagram The first in An edge is formed by nodes For the end; Combined, they represent the correspondence between nodes and edges. Representation diagram The first in An edge is formed by nodes Pointing to node , Representation diagram The first in An edge is formed by nodes Not pointing to a node ; The matching degree between the two graph models is calculated using a 24-times enumeration method, and the results are obtained. The goal is to achieve the best possible results, thereby obtaining appropriate matching relationships and ultimately forming a relatively complete target semantic model.

Images