A multi-machine multi-target matching method based on graph matching

Abstract

The application discloses a kind of multi-machine multi-target matching methods based on graph matching, comprising: using unmanned aerial vehicle A and unmanned aerial vehicle B, at least one image of detection area at the same time is obtained, and detection area includes multiple targets;For each image, input it to DLA34 network, after obtaining feature map, feature map is input to feature extraction head, so that feature extraction head carries out target detection based on the feature map, and extracts the related characteristics of target;Related characteristics include target characteristics and the characteristics of target edge;According to the target detection result and related characteristics corresponding to unmanned aerial vehicle A, and the target detection result and related characteristics corresponding to unmanned aerial vehicle B, affinity matrix is constructed;Based on affinity matrix, the matching result between targets is calculated.The application improves the feature utilization ability, efficiency and robustness of algorithm, especially in the case where the characteristics of the target itself are not clear, more features can be used, and better matching results can still be obtained.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, specifically relating to a multi-machine, multi-target matching method based on graph matching. Background Technology

[0002] Object re-identification (Re-ID) is an important research task in the field of computer vision, aiming to accurately identify and associate the same target object or person using images / videos from different cameras or time periods. Unlike object detection, object re-identification focuses more on the ability to track the same target across cameras, multiple angles, and over long periods. Object re-identification technology has wide applications in video surveillance, pedestrian tracking, and intelligent transportation. It describes the appearance features of the target by extracting feature representations, then compares and matches these features, and finally measures the similarity between different targets through distance calculation to determine whether they are the same target.

[0003] The development of target re-identification technology has brought great convenience and benefits to real-time monitoring, crowd management, and security. However, the problem of multi-drone cognitive consistency refers to the difficulty in achieving unified recognition of the same target during multi-drone collaborative tasks. Cognitive consistency is the foundation of all swarm intelligence tasks. However, with the increasing complexity of drone swarm system architectures and application environments, the effectiveness of using pure target re-identification methods to solve the multi-drone cognitive consistency problem is deteriorating. The simplest way to improve cognitive consistency in multi-drone swarm systems is to increase the size of the acquired images and use more powerful computing devices and models for feature extraction and comparison. However, this method places high demands on the system's workload and is unlikely to significantly improve performance.

[0004] The multi-drone cognitive unification method based on graph matching solves the above problems. Currently, solutions to the multi-drone cognitive unification problem can be divided into three main categories. The first category is image matching methods based on traditional image processing techniques. These methods extract identity features from targets using the SIFT operator or corner points, and then perform a global search and matching of the previously extracted identity features in another image. This method has a low matching success rate and is time-consuming. The second category is target re-identification methods. These methods first use target detection algorithms to obtain all targets and their positions within the field of view, and then use target re-identification algorithms to extract features from each detected target. By comparing the corresponding features between targets in images acquired by different drones, it is determined whether they belong to the same target. This method is simple and straightforward, easy to understand, but has high computational complexity. As the number of drones and targets within the field of view increases, the computational complexity increases quadratically, making it difficult to meet real-time requirements. The third type of method is based on affine transformation. This method first uses target detection algorithms on multiple UAVs to obtain all targets and their positions within the field of view. Then, a tracking algorithm is used to assign each target a globally unique ID. Next, the affine transformation between the photos is solved, and one photo is projected onto another photo through the affine transformation. The matching relationship between targets is obtained based on the affine transformation and the IOU between targets. This method has good performance, but the computation time is long. If there are few matching targets, it is difficult to obtain a good affine transformation solution, which leads to a significant decrease in the algorithm's performance. Summary of the Invention

[0005] To address the aforementioned problems in the existing technology, this invention provides a multi-machine, multi-target matching method based on graph matching. The technical problem to be solved by this invention is achieved through the following technical solution:

[0006] This invention provides a multi-machine, multi-target matching method based on graph matching, comprising:

[0007] Using UAV A and UAV B, at least one image of the detection area is acquired at the same time, and the detection area includes multiple targets;

[0008] For each image, it is input into the DLA34 network to obtain a feature map. The feature map is then input into a feature extraction head so that the feature extraction head can perform target detection based on the feature map and extract relevant features of the target. The relevant features include target features and features of edges between targets.

[0009] Based on the target detection results and related features corresponding to UAV A, and the target detection results and related features corresponding to UAV B, an affinity matrix is ​​constructed.

[0010] Based on the affinity matrix, the matching results between targets are calculated.

[0011] In one embodiment of the present invention, the feature extraction head includes: a target detection head, a target feature head, and an inter-target feature head. The target detection head includes: a center detection unit, a center correction unit, and an edge box prediction unit. The center detection unit, the center correction unit, the edge box prediction unit, the target detection head, the target feature head, and the inter-target feature head all include a convolutional layer with a kernel of 3, a ReLU layer, and a convolutional layer with a kernel of 1 connected in sequence.

[0012] In one embodiment of the present invention, the center detection unit is used to detect the target center point (X). cen ,Y cen ), X cen Y cen These represent the coordinates of the detected target center point on the X and Y axes, respectively.

[0013] The center correction unit is used to predict the correction amount (X, Y) of the target center point on the X and Y axes. reg ,Y reg );

[0014] The bounding box prediction unit is used to predict the bounding box (l, r, u, d) of the target, where l, r, u, and d represent the corrected target center point (X). cen +X reg Y cen +Y reg Distances to the left, right, above, and below the target;

[0015] The target feature header is used to extract target features;

[0016] The target feature header is used to extract the target edge features of the edge connecting the corrected center points of any two targets.

[0017] In one embodiment of the present invention, the step of constructing an affinity matrix based on the target detection results and related features corresponding to UAV A and the target detection results and related features corresponding to UAV B includes:

[0018] Obtain the target detection results for UAV A And the target detection results corresponding to UAV B Let n1, n2, ..., n be the nth digits detected from the image acquired by UAV A. A One goal, Let n1, n2, ..., n be the nth digits detected from the image acquired by UAV B. B One goal;

[0019] According to the goal With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n aff The affinity matrix; where n aff =n A *n B n1 <n i <n A n1 <n p <n A n1 <n j <n B n1 <n q <n B .

[0020] In one embodiment of the present invention, when n C =n E n D =n F At that time, according to the target With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n aff The steps of obtaining the affinity matrix include:

[0021] Obtain the target Target characteristics and objectives The target features are determined, and the similarity between the two is calculated to obtain the position of the first element in the affinity matrix. line, number The elements of the column.

[0022] In one embodiment of the present invention, when n C ≠n E n D ≠n F At that time, according to the target With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n affThe steps of obtaining the affinity matrix include:

[0023] Obtain the target Corrected center point and target The corrected features of the edges connecting the center points to the target edges, and the target With the goal The target edge features of the connected edges are obtained, and the elements located on the diagonal of the affinity matrix are obtained by calculating the similarity between the two.

[0024] In one embodiment of the present invention, the step of calculating the matching result between targets based on the affinity matrix includes:

[0025] Based on the affinity matrix, the size is calculated using the reweighted random walk matching algorithm. The probability matrix; the elements in the probability matrix represent the probability that the target in the row where the element is located matches the target in the column where the element is located;

[0026] Based on the probability matrix, the matching results between targets are obtained using the Hungarian matching algorithm.

[0027] In one embodiment of the present invention, after the step of obtaining the matching results between targets using the Hungarian matching algorithm based on the probability matrix, the method further includes:

[0028] For the matching target pairs in the matching results, determine whether the similarity of their target features is greater than a preset threshold; if yes, no processing is performed; if no, the target pair is deleted from the matching results to obtain the final matching result.

[0029] In one embodiment of the present invention, target pairs in the final matching result are displayed using bounding boxes.

[0030] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0031] This invention provides a multi-machine, multi-target matching method based on graph matching, which solves the problem of poor matching effect in existing technologies. It also improves the feature utilization capability and efficiency of the algorithm, and enhances the robustness of the algorithm to achieve higher matching accuracy. In particular, when the features of the target itself are unclear, more usable features are introduced, and good matching results can still be obtained.

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0033] Figure 1 This is a flowchart of a multi-machine, multi-target matching method based on graph matching provided in an embodiment of the present invention;

[0034] Figure 2 This is a schematic diagram of a multi-machine, multi-target matching method based on graph matching provided in an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of the structure of the DLA34 network provided in an embodiment of the present invention;

[0036] Figure 4 This is a schematic diagram of the feature extraction head provided in an embodiment of the present invention;

[0037] Figure 5 This is a schematic diagram of the target detection results and related features provided in the embodiments of the present invention;

[0038] Figure 6 This is a schematic diagram of the affinity matrix provided in an embodiment of the present invention;

[0039] Figure 7 This is a schematic diagram of the matching results provided in an embodiment of the present invention;

[0040] Figure 8a This is a schematic diagram of the matching results using only target features provided in an embodiment of the present invention;

[0041] Figure 8b This is a schematic diagram of the matching results using only the edge features between targets, provided by an embodiment of the present invention. Detailed Implementation

[0042] The present invention will be further described in detail below with reference to specific embodiments, but the implementation of the present invention is not limited thereto.

[0043] Figure 1 This is a flowchart of a multi-machine, multi-target matching method based on graph matching provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a multi-machine, multi-target matching method based on graph matching provided in an embodiment of the present invention. For example... Figure 1-2 As shown, this embodiment of the invention provides a multi-machine, multi-target matching method based on graph matching, including:

[0044] S1. Using UAV A and UAV B, acquire at least one image of the detection area at the same time. The detection area includes multiple targets.

[0045] S2. For each image, input it into the DLA34 network to obtain a feature map. Then, input the feature map into the feature extraction head so that the feature extraction head can perform target detection based on the feature map and extract relevant features of the target. Relevant features include target features and features of edges between targets.

[0046] S3. Construct an affinity matrix based on the target detection results and related features corresponding to UAV A and UAV B.

[0047] S4. Calculate the matching results between targets based on the affinity matrix.

[0048] It should be noted that the detection area can be photographed at the same time using the cameras built into both Drone A and Drone B, and the shooting angles of Drone A and Drone B are different, thus obtaining one or more consecutive images containing the target.

[0049] In step S2, the images acquired by the two drones are cropped, scaled, and preprocessed to a size of 3*768*1024 before being input into the DLA34 network. Figure 3 This is a schematic diagram of the structure of the DLA34 network provided in an embodiment of the present invention, as shown below. Figure 3 As shown, its network structure consists of six levels, namely levels 0 to 6 as shown in the figure. Each level consists of a base layer or a base layer and a aggregation layer. Specifically, the base layer consists of a convolutional layer with a kernel size of 3, a BN layer, and a ReLU layer. Level 0 consists of a convolutional layer with a kernel size of 3, a BN layer, and a ReLU layer. Level 1 consists of a convolutional layer with a kernel size of 3 and a stride of 2, a BN layer, and a ReLU layer. Starting from level 2, each level adopts a DenseNet structure, internally using a residual structure. Finally, the Root is used to merge the outputs from DenseNet and the residuals. Compared with level 2, level 3 is deeper and wider. Level 4 is the same as above. Finally, level 5 is used for feature integration and output. The feature map size extracted by the DLA34 network is 1*64*152*272.

[0050] In this embodiment, the output of the DLA34 network is connected to a feature extraction head, which is used to learn multiple desired targets, thereby acquiring multiple target information simultaneously. Figure 4 This is a schematic diagram of the feature extraction head provided in an embodiment of the present invention. Optionally, as shown... Figure 4 As shown, the feature extraction head includes: an object detection head, an object feature head, and an inter-object feature head. The object detection head includes: a center detection unit, a center correction unit, and an edge box prediction unit. The center detection unit, the center correction unit, the edge box prediction unit, the object detection head, the object feature head, and the inter-object feature head all include a convolutional layer with a kernel of 3, a ReLU layer, and a convolutional layer with a kernel of 1, which are connected in sequence.

[0051] Optionally, the central detection unit is used to detect the target center point (X).cen ,Y cen ), X cen Y cen These represent the coordinates of the detected target center point on the X and Y axes, respectively.

[0052] The center correction unit is used to predict the correction amount of the target center point on the X and Y axes (X). reg ,Y reg );

[0053] The bounding box prediction unit is used to predict the bounding box (l, r, u, d) of the target, where l, r, u, and d represent the corrected target center point (X). cen +X reg Y cen +Y reg Distances to the left, right, above, and below the target;

[0054] The target feature header is used to extract target features;

[0055] The inter-target feature header is used to extract the inter-target edge features of the edge connecting the corrected center points of any two targets.

[0056] Specifically, this embodiment obtains the target center point location feature map, the target center point offset error feature map, and the feature representation feature map of the distance from the target center point to the target edge box for the target detection task by controlling the number of convolution kernels with a kernel size of 1 in the feature extraction head. The size of these features is 1*1*152*272, 1*2*152*272, and 1*4*152*272, respectively. At the same time, for the feature extraction task, the target feature representation feature map with a size of 1*128*152*272 and the feature representation feature map of the edges between targets are obtained with a size of 1*64*152*272.

[0057] Figure 5 This is a schematic diagram of the target detection results and related features provided in an embodiment of the present invention. Please refer to... Figure 5 In this embodiment, the target center point detected based on the above-mentioned target center point location feature map is denoted as (X). cen ,Y cen ), X cen Y cen Let X and Y represent the coordinates of the target center point on the X and Y axes, respectively. The target center point correction amount predicted based on the above target center point offset error is denoted as (X...). reg ,Y reg ), X reg Y regLet (l, r, u, d) represent the correction amounts for the target center point along the X and Y axes, respectively. The target edge box detected based on the feature map representing the distance from the target center point to the target edge box is denoted as (l, r, u, d), where l, r, u, and d represent the corrected distances from the target center point to the left, right, top, and bottom of the target, respectively. The corrected target center point coordinates are (X, r, u, d). cen +X reg ,Y cen +Y reg The overall representation of the objective is (X). min ,Y min ,X max ,Y max ), X min =Xl,Y min =Yu,X max =X+r,Y max =Y+d. Further, the coordinates on the target feature representation feature map are taken as (X... cen +X reg ,Y cen +Y reg The 128-dimensional vector of the point is used as the feature representation of the target. The feature of the edge between the targets is represented by the set of features of the points. That is, for an edge, the corresponding features of the two target center points of its endpoints and the points that divide the edge between the two center points on the feature representation feature map of the edge between the targets are used as the feature representation of the edge.

[0058] Step S3, the step of constructing the affinity matrix based on the target detection results and related features corresponding to UAV A and UAV B, includes:

[0059] S301. Obtain the target detection results corresponding to UAV A. And the target detection results corresponding to UAV B Let n1, n2, ..., n be the nth digits detected from the image acquired by UAV A. A One goal, Let n1, n2, ..., n be the nth digits detected from the image acquired by UAV B. B One goal;

[0060] S302, According to the objective With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n affThe affinity matrix; where n aff =n A *n B n1 <n i <n A n1 <n p <n A n1 <n j <n B n1 <n q <n B .

[0061] Specifically, when n C =n E n D =n F At that time, according to the target With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n aff The steps for obtaining the affinity matrix include:

[0062] Obtain the target Target characteristics and objectives The target features are determined, and the similarity between the two is calculated to obtain the value located at the th position in the affinity matrix. line, number The elements of the column.

[0063] On the other hand, when n C ≠n E n D ≠n F At that time, according to the target With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size n. aff *n aff The steps for obtaining the affinity matrix include:

[0064] Obtain the target Corrected center point and target The corrected features of the edges connecting the center points to the target edges, and the target With the goal The target edge features of the connected edges are obtained, and the elements located on the diagonal of the affinity matrix are obtained by calculating the similarity between the two.

[0065] To facilitate understanding of this embodiment, the affinity matrix will be explained first.

[0066] Affinity matrices are used to measure the distance or similarity between two points in a space. In computer vision tasks, affinity matrices are typically represented as a weighted graph that treats each pixel as a node and connects each pair of pixels with an edge.

[0067] In this embodiment, all detected targets are treated as points in the affinity matrix. The corrected center points of two targets are connected to form an edge, thereby constructing the affinity matrix. The problem of cognitive consistency among UAVs is solved by solving the matching of points in the affinity matrix. This method cleverly transforms the simple target matching problem into a graph matching problem.

[0068] Optionally, the affinity matrix is ​​constructed using the target detection results, target feature representations, and inter-target edge feature representations extracted through the DLA34 network and feature extraction head. The construction of the affinity matrix first requires determining a metric that makes the result closer to 1 when two variables are similar, and closer to 0 when two variables are different, to represent the relationship between two features. In this invention, the cosine similarity between target features or between inter-target edge features is selectively used as a measure of similarity.

[0069] For each image acquired by a drone, the target detection result is represented as matrix N. det The size is n×4, where n represents the number of detected targets, and each target can be described as (X). min ,Y min ,X max ,Y max ), X min Y min X max Y max These represent the X-axis and Y-axis coordinates of the top-left vertex and the bottom-right vertex of the target bounding box, respectively. The further extracted target features are represented as matrix N. NF The size is n×128, meaning that the representation dimension of each target feature is 128; the edge features between targets are represented as matrix N. EF Size n 2 ×320, where n 2 denoted by , where 320 represents the feature representation of each edge. For example, cosine similarity is used as the metric to construct the affinity matrix, and both target features and inter-target edge features are encoded simultaneously.

[0070] Figure 6This is a schematic diagram of the affinity matrix provided in an embodiment of the present invention. Specifically, the target detection result corresponding to UAV A is represented as follows: The number of targets detected is n A The target detection result corresponding to UAV B is represented as The number of targets detected is n B The constructed affinity matrix is ​​represented as N aff Size n aff *n aff , where n aff =n a *n b .like Figure 6 As shown, during the construction process, N is first sequentially... detA Using the elements in the table as the core, iterate through N one by one. detB The element in the matrix is ​​located at the th position in the affinity matrix. line, number The elements of the row represent the target. The target-to-target edge features of the edges connected by the corrected center points and the target The similarity of the features of the edges between targets connected by the corrected center points; in particular, when n C =n E n D =n F At that time, located in the first line, number The elements of the column are the diagonal elements of the affinity matrix, and these elements can be calculated using the target... Target characteristics and objectives The similarity of the target features is obtained.

[0071] Obviously, in order to achieve cognitive unification among unmanned aerial vehicles, this embodiment will perform graph construction processing on the above-mentioned relevant features. That is, the target itself is regarded as a vertex in the graph, the feature representation of the vertex is represented by the extracted target feature representation, and the relationship between the target is regarded as the edge between two vertices in the graph, the feature representation of the edge is represented by the extracted edge features between the targets, thereby constructing the affinity matrix.

[0072] Step S4, which involves calculating the matching results between targets based on the affinity matrix, includes:

[0073] S401. Based on the affinity matrix, the size is calculated using the reweighted random walk matching algorithm. The probability matrix; the elements in the probability matrix represent the probability that the target in the row corresponding to the current element matches the target in the column corresponding to the current element.

[0074] S402. Based on the probability matrix, the matching results between targets are obtained by using the Hungarian matching algorithm.

[0075] This embodiment selectively uses the RRWM (Reweighted Random Walk Matching) algorithm to solve for the correspondence between targets. RRWM is based on the random walk algorithm and performs matching by calculating similarity. The basic idea is to use the random walk theory in graph theory to perform random walks on the constructed graph (affinity matrix) to calculate the similarity between them.

[0076] Specifically, in step S401, the affinity matrix is ​​first normalized to ensure that the sum of the elements in each row is 1 and the value of each element is greater than 0. Given the affinity matrix W and the reweighting factor α, the transition matrix P is initialized using the affinity matrix, and the probability matrix x is initialized, with the values ​​in x being averaged during initialization. The probability matrix x and the transition matrix P are repeatedly used to perform edge random walk calculations that maintain affinity under the constraint of bidirectional reweighting, thereby calculating the expected probability matrix y. The probability matrix x = (1-α)y + αx is updated using the reweighting factor α and the expected probability matrix y.

[0077] Repeat the above process until x converges, then use the Hungarian matching algorithm to solve for the probability matrix x and obtain the final matching result.

[0078] Of course, in some other embodiments of this application, other graph matching algorithms can also be used in step S401 to solve the matching results between targets, and this embodiment does not limit this.

[0079] Furthermore, after the step of obtaining the matching results between targets using the Hungarian matching algorithm based on the probability matrix, the process also includes:

[0080] For a matching target pair in the matching results, determine whether the similarity of their target features is greater than a preset threshold; if yes, no processing is performed; if no, the target pair is deleted from the matching results to obtain the final matching result.

[0081] Specifically, after solving the probability matrix P using the Hungarian algorithm, the matching result is represented as a similarity matrix. To avoid the situation where different targets are successfully matched, the similarity of the target features of the matched target pairs can also be calculated. If the similarity between the two is less than a preset threshold such as 0.3, they are deleted from the matching result.

[0082] For the target pairs in the final matching results, bounding boxes can be used to display them, allowing users to intuitively obtain the matching results.

[0083] The following simulation experiment further illustrates the multi-machine, multi-objective matching method based on graph matching provided by this invention.

[0084] Specifically, input two images taken simultaneously from the MDMT dataset, but with certain differences in perspective, where at least one identical target exists in both images.

[0085] Figure 7 This is a schematic diagram of the matching results provided in an embodiment of the present invention. Taking two drones as an example, the correspondence between targets appearing in both images can be calculated, such as... Figure 7 As shown in the figure, the analysis shows that the matching results are relatively accurate and make full use of the information in the image. Even if most of the target has left the field of vision, the relationship between the targets can still be used to make connections.

[0086] Figure 8a This is a schematic diagram of the matching results using only target features, provided by an embodiment of the present invention. Figure 8b

[0087] This is a schematic diagram of the matching results using only the inter-target edge features provided in an embodiment of the present invention. Further, the simulation conditions are the same as above, but target features are not input when constructing the affinity matrix; matching is performed separately using both target features and inter-target edge features. The matching results are as follows. Figure 8a , 8b As shown, both cases can solve some problems, but the effect is not as good as when both target features and inter-target edge features are used simultaneously, thus proving the superior effect of the present invention and solving the cognitive unity problem better.

[0088] As can be seen from the above embodiments, the beneficial effects of the present invention are as follows:

[0089] This invention provides a multi-machine, multi-target matching method based on graph matching, which solves the problem of poor matching effect in existing technologies. It also improves the feature utilization capability and efficiency of the algorithm, and enhances the robustness of the algorithm to achieve higher matching accuracy. In particular, when the features of the target itself are unclear, more usable features are introduced, and good matching results can still be obtained.

[0090] In the description of this invention, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. In addition, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0091] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. A multi-machine, multi-objective matching method based on graph matching, characterized in that, include: Using UAV A and UAV B, at least one image of the detection area is acquired at the same time, and the detection area includes multiple targets; For each image, it is input into the DLA34 network to obtain a feature map. The feature map is then input into a feature extraction head so that the feature extraction head can perform target detection based on the feature map and extract relevant features of the target. The relevant features include target features and features of edges between targets. The feature extraction head includes: a target detection head, a target feature head, and an inter-target feature head. The target detection head includes: a center detection unit, a center correction unit, and an edge box prediction unit. The center detection unit is used to detect the target center point. The center correction unit is used to predict the correction amount of the target center point on the X and Y axes. Based on the target detection results and related features corresponding to UAV A, and the target detection results and related features corresponding to UAV B, an affinity matrix is ​​constructed, including: obtaining the target detection results corresponding to UAV A. And the target detection results corresponding to UAV B , These represent the results of image detection based on images acquired by UAV A. One goal, These represent the results of image detection based on image acquired by UAV B. One goal; based on the goal With the goal Relevant characteristics and objectives With the goal Relevant features, calculate the position of the first line, number The elements of the column are used to construct a structure of size. The affinity matrix, specifically includes the following steps: when , At that time, obtain the target Target characteristics and objectives The target features are determined, and the similarity between the two is calculated to obtain the position of the first element in the affinity matrix. line, number The elements of the column; when , At that time, obtain the target Corrected center point and target The corrected features of the edges connecting the center points to the target edges, and the target With the goal The target edge features of the connected edges are obtained, and the elements located on the diagonal of the affinity matrix are obtained by calculating the similarity between the two; wherein, , , , , ; Based on the affinity matrix, the matching results between targets are calculated, including: calculating the size of the target using a reweighted random walk matching algorithm based on the affinity matrix. The probability matrix is ​​obtained; the elements in the probability matrix represent the probability that the target in its own row matches the target in its own column; based on the probability matrix, the matching result between the targets is obtained by using the Hungarian matching algorithm.

2. The multi-machine, multi-target matching method based on graph matching according to claim 1, characterized in that, The center detection unit, center correction unit, edge box prediction unit, target detection head, target feature head, and inter-target feature head all include a convolutional layer with a kernel of 3, a ReLU layer, and a convolutional layer with a kernel of 1 connected in sequence.

3. The multi-machine, multi-target matching method based on graph matching according to claim 2, Its features are, Among them, the target center point is , , These represent the X and Y coordinates of the detected target center point, respectively; the correction values ​​for the target center point on the X and Y axes are... ; The bounding box prediction unit is used to predict the edge box of the target. , , , , These represent the corrected target center points. Distances to the left, right, above, and below the target; The target feature header is used to extract target features; The target feature header is used to extract the target edge features of the edge connecting the corrected center points of any two targets.

4. The multi-machine multi-target matching method based on graph matching according to claim 1, characterized in that, After obtaining the matching results between targets using the Hungarian matching algorithm based on the probability matrix, the method further includes: For the matching target pairs in the matching results, determine whether the similarity of their target features is greater than a preset threshold; if yes, no processing is performed; if no, the target pair is deleted from the matching results to obtain the final matching result.

5. The multi-machine multi-target matching method based on graph matching according to claim 4, characterized in that, The target pairs in the final matching result are displayed using bounding boxes.

Images