Multi-target matching method under cross view angle

Abstract

The invention relates to a multi-target matching method under a cross view angle, and the method comprises the steps: 1, collecting multi-target images at different angles through a plurality of cameras, and obtaining the images of a plurality of targets in different view fields; 2, detecting target images shot at different angles by adopting an intelligent target recognition algorithm, then calculating sight angles of multiple targets in the images under a camera coordinate system, and temporarily adding a label attribute to each detected target; 3, according to the attitude angle of the camera in the reference coordinate system and the conversion relation between the camera coordinate system and the reference coordinate system, the sight angle of the target in the reference coordinate system is calculated; 4, calculating a direction cosine based on the sight angle of the target under the reference coordinate system, then calculating direction vectors of the target under different detection view angles, and at the moment, obtaining direction vectors of the target under different view angles; and step 5, constructing a multi-target matching model, and judging whether the target sight angles under different detection view angles are coplanar and intersected or not.

Description

Technical Field

[0001] The present invention belongs to the technical field of target recognition and matching, and in particular relates to a multi-target matching method under cross-viewing angles. Background Art

[0002] When considering the collaborative detection of multiple targets by multiple aircraft, the detected targets have different scales, shape changes, target occlusion and other problems in different detection fields of view, which makes it impossible to directly match multiple targets under cross-viewing angles through geometric features, which greatly increases the difficulty of determining the consistency of multiple targets under multiple perspectives. However, for multi-target recognition and matching technology under multiple detection perspectives, whether the association between multi-source targets can be found is the key to achieving the correspondence between multiple targets under different perspectives. Since traditional target matching algorithms are affected by factors such as perspective changes, the accuracy of multi-target matching through geometric features when observing targets from different angles is low. Especially in multi-UAV collaborative target detection, multi-target matching technology under different perspectives is a decisive factor in UAV task allocation, so target matching technology under multiple detection perspectives is of great significance.

[0003] Commonly used target association methods are mainly based on two categories: the feature constraint relationship of the target and the spatial constraint relationship of the target. Due to the influence of factors such as angle changes, traditional target association algorithms do not work well under target observation conditions from different perspectives. CN202111047109.7 discloses a multi-view multi-target association method based on a deep neural network. By performing target features on multiple target detection frames at the same time under multiple perspectives, and then inputting their corresponding Euclidean distances into the deep neural network to determine whether the corresponding loss model converges, this method has good performance in fixed detection target scenes, but cannot obtain target feature information in a timely manner when multiple drones and targets are relatively dynamic, and it is easy to cause target information loss, affecting task execution. CN201610420557.X discloses a recognition-assisted multi-target tracking method based on a deep neural network. This method uses the correlation of target information between frames to obtain the matching relationship of the target, and uses a deep learning network to restore the trajectories of multiple targets to improve the target correlation calculation, but this method mainly relies on the continuous dynamic information of the target and cannot be applied to the detection and group target association calculation in the target relative dynamic scene. CN103677734A discloses an image matching method for multi-target object tracking. The image matching method for object tracking combines labeling technology, wavelet image transform, and image matching technology to improve the calculation speed while ensuring the calculation accuracy. It is a simple and effective multi-target object tracking method. However, this method mainly searches for targets through a template matching algorithm. When the target has a large scale change under different viewing angles, it is difficult to accurately match the target. Summary of the invention

[0004] The present invention discloses a method for matching multiple targets under cross-viewing angles. The algorithm is mainly aimed at the problem that when multiple aircrafts collaboratively perceive group targets, the detected group targets have scale and shape change differences, target occlusion and other problems in different detection fields, resulting in the inability to directly achieve group target matching under cross-viewing angles through geometric features.

[0005] In order to solve the problem of matching multiple targets under cross-viewing angles, the present invention proposes a target matching method under cross-viewing angles that only relies on the sensor attitude angle and the target sight angle. First, the acquired sight angle of the target body is converted to the sight angle under the reference system according to the sensor attitude angle, and then the direction vector of the target relative to the detection sensor under different detection angles is calculated based on the direction cosine of the target under the reference system to obtain the direction vector of the target under different viewing angles. The characteristic that the direction vector of the same target under different detection angles is in the same plane is utilized, that is, by judging whether the direction vector of the target under different viewing angles is in the same plane and has a unique intersection, the consistency of the target is determined. A consistent matching strategy under multiple detection angles is designed to reduce the influence of the detection sensor attitude angle measurement error and the target angle measurement error on the matching accuracy.

[0006] The above-mentioned multi-target matching method under a cross-viewing angle includes the following steps:

[0007] Step 1: Use multiple cameras to collect images of multiple targets at different angles to obtain images of multiple targets in different fields of view;

[0008] Step 2: Use an intelligent target recognition algorithm to detect target images taken at different angles, then calculate the sight angles of multiple targets in the image in the camera coordinate system, and temporarily add label attributes to each detected target;

[0009] Step 3: Calculate the sight angle of the target in the reference coordinate system based on the attitude angle of the camera in the reference coordinate system and the conversion relationship between the camera coordinate system and the reference coordinate system;

[0010] Step 4: Calculate the direction cosine based on the sight angle of the target in the reference coordinate system, and then calculate the direction vector of the target at different detection angles. At this time, the direction vector of the target at different angles can be obtained;

[0011] Step 5: Using the characteristics that the direction vectors of the same target under different detection perspectives are in the same plane and have a unique intersection, a multi-target matching model is constructed, and a multi-target matching strategy is designed to reduce the impact of external measurement errors on the matching results. It is determined whether the target sight angles under different detection perspectives are coplanar and intersecting. If the judgment result is coplanar and intersecting, it is classified as a consistent target and a label attribute is added to record it.

[0012] Preferably, in step one, multiple cameras are designed to capture images of multiple targets at different angles, so that the multiple targets have certain shape and scale changes under different detection viewing angles.

[0013] Preferably, in step 2, the position information of the target in the image is obtained by adopting an intelligent target recognition algorithm, and then based on the conversion relationship between the image coordinate system and the camera coordinate system, the line of sight angles of multiple targets detected in the image in the camera coordinate system are calculated, and a label attribute is temporarily added to each detected target.

[0014] Preferably, in step three, since the sight angles of different detection fields of view of the target obtained in step two are not in the same coordinate system, it is necessary to calculate the sight angle of the target in the reference coordinate system based on the camera's attitude angle and the conversion relationship between the camera coordinate system and the reference coordinate system.

[0015] Preferably, in step 4, the direction cosines are calculated based on the sight angle of the target in the reference coordinate system, and then the direction vector of the target at different detection viewing angles is calculated, and the direction vector of the target at different viewing angles can be obtained.

[0016] Preferably, in step five, by analyzing the sight angle characteristics of different targets under multiple detection perspectives, that is, the direction vectors formed by the sight lines of the same target under different detection perspectives are in the same plane and have a unique intersection, a multi-target matching model is constructed, and at the same time, in order to reduce the impact of external measurement errors on the matching results, a multi-target matching strategy is designed to determine whether the sight angles of the targets under different detection perspectives are coplanar and intersecting. If the judgment result is that they are coplanar and intersecting, they are consistent targets.

[0017] Experimental verification shows that the present invention can effectively realize the consistent recognition of multiple targets under multiple detection perspectives, and provide technical support for the tracking of multiple targets under cross-perspectives. Compared with the prior art, the technical benefits of the present invention are:

[0018] (1) The present invention constructs a multi-target matching model for multi-view detection by analyzing the coplanar and intersecting sight lines of the target under different detection perspectives.

[0019] (2) The present invention designs a multi-target matching strategy, which can effectively reduce the impact of external measurement errors on matching results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 The present invention is an algorithm flow chart of a multi-target matching method under a cross-perspective.

[0021] Figure 2 0.3° noise and different relative distances to the target￥ 12 The change of the value, where (a) the target relative distance change, (b) ￥ 12 Changes in value.

[0022] Specific technical solutions

[0023] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0024] The present invention discloses a multi-target matching method under a cross-viewing angle, and the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods:

[0025] Step 1: Use multiple cameras to collect images of multiple targets at different angles to obtain images of multiple targets in different fields of view;

[0026] Step 2: Use an intelligent target recognition algorithm to detect target images taken at different angles, then calculate the sight angles of multiple targets in the image in the camera coordinate system, and temporarily add label attributes to each detected target;

[0027] Step 3: Calculate the sight angle of the target in the reference coordinate system based on the attitude angle of the camera in the reference coordinate system and the conversion relationship between the camera coordinate system and the reference coordinate system;

[0028] Step 4: Calculate the direction cosine based on the sight angle of the target in the reference coordinate system, and then calculate the direction vector of the target at different detection angles. At this time, the direction vector of the target at different angles can be obtained;

[0029] Step 5: Using the characteristics that the direction vectors of the same target under different detection perspectives are in the same plane and have a unique intersection, a multi-target matching model is constructed, and a multi-target matching strategy is designed to reduce the impact of external measurement errors on the matching results. It is determined whether the target sight angles under different detection perspectives are coplanar and intersecting. If the judgment result is coplanar and intersecting, it is classified as a consistent target and a label attribute is added to record it.

[0030] As described in step 1, multiple simulated targets (such as cartons, tables and chairs, pedestrians, etc.) are placed at the determined target position, and multiple cameras are set to detect the target at a certain attitude angle, and the cameras should have a cross field of view, and then target image acquisition is performed in this state;

[0031] As described in step 2, the intelligent target recognition method is first used to detect the target in the image captured by the camera, and the position of the target in the image (x, y) is recorded. Then, according to the internal parameters of the camera, the focal length f, pixel size (dux, duy), resolution W×H, field of view (θ x ,θ y) and other information to calculate the sight angle of the target in the camera coordinate system. The sight angle calculation formula is shown in (1), where θ v ,θ h The vertical and horizontal sight angles are respectively positive in the upper right side of the image. Finally, temporary label attributes are added to the targets detected at different detection angles.

[0032]

[0033] As described in step 3, the target sight angle in the camera coordinate system is obtained in step 2. To achieve collaborative positioning, the target information under different viewing angles needs to be unified into the reference coordinate system. Therefore, it is necessary to calculate the sight angle of the target in the reference coordinate system based on the camera attitude angle and the conversion relationship between the camera coordinate system and the reference coordinate system. Assume that the attitude angles (pitch angle, yaw angle, roll angle) of the camera in the reference coordinate system are θ p ,θ y ,θ r , the coordinate transformation matrix from the A reference system to the camera coordinate system can be expressed as:

[0034]

[0035] The vertical and horizontal sight angles θ of the target in the reference coordinates are v c ,θ h c It can be expressed as:

[0036]

[0037] in,

[0038]

[0039] As described in step 4, after obtaining the sight angle of the target in the reference coordinate system, the direction vector of the target at different detection angles is calculated. At this time, the direction vector of the target at different angles can be obtained. Assuming that the direction vectors at two different detection angles are T1 and T2, they are expressed as:

[0040]

[0041] As described in step 5, according to the characteristics that the direction vectors of the same target under different detection angles are in the same plane and have a unique intersection, first assume that the direction vectors of the two sight angles are coplanar. Then, the direction vector can be obtained based on the principle that the two vectors are coplanar, which can be expressed as:

[0042]

[0043] Then, assuming that the position of camera 1 is the origin, the position information (X, Y, Z) of camera 2 relative to the origin is measured, and the direction vectors of camera 1 and camera 2 are m 12 = [XYZ], according to the coplanarity of the direction vectors of the two sight angles, we know that vector m 12 and vectors T1 and T2 are coplanar, so according to the vector coplanarity theorem, we can get vector m 12 and the normal vector Vertical.

[0044] Due to the existence of external measurement errors such as target sight angle and attitude angle, like Figure 2 As the relative position of the target and the detection platform changes and 0.3° noise is added to the target's elevation and azimuth sight angles, the 12 Therefore, this patent designs a multi-target matching strategy to improve the accuracy of judging whether the target sight angles under different detection angles are coplanar and intersecting, and reduce the impact of external measurement errors on the matching results. Assuming that detection platform 1 and detection platform 2 detect multiple targets, solve the cross-matching set Set the judgment threshold σ, and then calculate The set of values ​​smaller than σ, that is, if Then it is determined as the number of elements in the set, otherwise it is eliminated. Then the minimum value in the set is found, and the minimum value corresponds to the final consistency target matching result.

Claims

1. A multi-target matching method under cross-viewing angles, characterized in that: The steps include: Step 1: Use multiple cameras to collect images of multiple targets at different angles to obtain images of multiple targets in different fields of view; Step 2: Use an intelligent target recognition algorithm to detect target images taken at different angles, then calculate the sight angles of multiple targets in the image in the camera coordinate system, and temporarily add label attributes to each detected target; Step 3: Calculate the sight angle of the target in the reference coordinate system based on the attitude angle of the camera in the reference coordinate system and the conversion relationship between the camera coordinate system and the reference coordinate system; Step 4: Calculate the direction cosine based on the sight angle of the target in the reference coordinate system, and then calculate the direction vector of the target at different detection viewing angles. At this time, the direction vector of the target at different viewing angles can be obtained; Step 5: Using the characteristics that the direction vectors of the same target under different detection perspectives are in the same plane and have a unique intersection, a multi-target matching model is constructed, and a multi-target matching strategy is designed to reduce the impact of external measurement errors on the matching results. It is determined whether the target sight angles under different detection perspectives are coplanar and intersecting. If the judgment result is coplanar and intersecting, it is classified as a consistent target and a label attribute is added to record it.

2. The method for matching multiple targets under cross-viewing angles according to claim 1, characterized in that: In step one, multiple simulated targets are placed at the determined target positions, and multiple cameras are set to detect the targets at certain attitude angles. The cameras should have a cross-field of view, and then target images are collected in this state.

3. The method for matching multiple targets under cross-viewing angles according to claim 2, characterized in that: The simulated targets include cartons, tables and chairs, and pedestrians.

4. The method for matching multiple targets under cross-viewing angles according to claim 2, characterized in that: In step 2, the intelligent target recognition method is first used to detect the target in the image captured by the camera, and the position of the target in the image (x, y) is recorded. Then, according to the internal parameters of the camera, the focal length f, pixel size (dux, duy), resolution W×H, field of view (θ x ,θ y ) information to calculate the sight angle of the target in the camera coordinate system; The calculation formula of sight angle is shown in (1), where θ v ,θ h The vertical and horizontal sight angles are respectively positive in the upper right side of the image. Finally, temporary label attributes are added to the targets detected at different detection angles.

5. The method for matching multiple targets under cross-viewing angles according to claim 4, characterized in that: In step 3, the target sight angle in the camera coordinate system is obtained in step 2. To achieve collaborative positioning, the target information under different viewing angles needs to be unified into the reference coordinate system. Therefore, it is necessary to calculate the sight angle of the target in the reference coordinate system according to the camera attitude angle and the conversion relationship between the camera coordinate system and the reference coordinate system. Assume that the attitude angle of the camera in the reference coordinate system, that is, the pitch angle, yaw angle, and roll angle are θ respectively. p ,θ y ,θ r , the coordinate transformation matrix from the A reference system to the camera coordinate system can be expressed as: The vertical and horizontal sight angles θ of the target in the reference coordinates are v c ,θ h c It can be expressed as: in, 6. The method for matching multiple targets under cross-viewing angles according to claim 5, characterized in that: In step 4, assuming that the direction vectors at two different detection angles are T1 and T2, they are expressed as:

7. The method for matching multiple targets under cross-viewing angles according to claim 5, characterized in that: In step 5, we first assume that the direction vectors of the two sight angles are coplanar. Then, based on the principle that the two vectors are coplanar, we can get their direction vectors, which can be expressed as: Then, assuming that the position of camera 1 is the origin, the position information (X, Y, Z) of camera 2 relative to the origin is measured, and the direction vectors of camera 1 and camera 2 are m 12 = [XYZ], according to the coplanarity of the direction vectors of the two sight angles, we know that vector m 12 and vectors T1 and T2 are coplanar, so according to the vector coplanarity theorem, we can get vector m 12 and the normal vector Vertical, that is Due to the existence of external measurement errors such as target sight angle and attitude angle, Assume that detection platform 1 and detection platform 2 detect multiple targets, solve the cross-matching set Set the judgment threshold σ, and then calculate The set of values ​​smaller than σ, that is, if The number of elements in the set is determined to be the element number of the set, otherwise it is eliminated data; then the minimum value in the set is found, and the minimum value corresponds to the final consistency target matching result.

Images