Multi-sensor based target matching method, device and equipment, and storage medium

Abstract

The application provides a multi-sensor-based target matching method, device, equipment and storage medium. The method comprises: obtaining first perception data of each first target perceived by a first sensor and second perception data of each second target perceived by a second sensor; setting first and second search areas for each first target and second target according to measured coordinates of each of the first targets perceived by the first sensor and measured coordinates of each of the second targets perceived by the second sensor; for each first target, obtaining the coincidence degree of the first search area corresponding to the first target and the second search area of each second target, and obtaining the first perception data and the second perception data corresponding to each target satisfying a preset condition according to the coincidence degree, and obtaining the second target matched with the first target according to the first perception data and the second perception data. The method of the application solves the problem that the prior art lacks in achieving high-precision and high-efficiency target matching.

Description

TECHNICAL FIELD

[0001] The present application relates to unmanned technology, and in particular to a target matching method and device based on multiple sensors, equipment and storage medium. BACKGROUND

[0002] In recent years, unmanned technology has developed rapidly. The increasing demand for reducing accident rates, energy saving and emission reduction, and cost reduction has attracted much attention to unmanned vehicles.

[0003] An unmanned vehicle, also known as an autonomous vehicle, senses the surrounding environment by loading multiple sensors, and the vehicle controller processes and analyzes the sensing information to ultimately achieve automatic route planning and autonomous driving. The vehicle is affected by weather, road conditions and other factors. In order to fully utilize the advantages of each sensor and improve the reliability of autonomous driving, the current mainstream method is to perform multi-sensor fusion matching. Commonly used sensors for sensing the environment include cameras, laser radars, and millimeter wave radars. However, the output of each sensor is independent, and in order to accurately and effectively use each data, target matching must be performed.

[0004] The prior art lacks high-precision and high-efficiency target matching. SUMMARY

[0005] The present application provides a target matching method, device, equipment and storage medium based on multiple sensors to solve the problem that the prior art lacks high-precision and high-efficiency target matching.

[0006] In one aspect, the present application provides a target matching method based on multiple sensors, comprising:

[0007] Obtaining first sensing data of each first target sensed by a first sensor and second sensing data of each second target sensed by a second sensor, the first sensing data and the second sensing data each including the relative distance, relative speed and orientation of the target;

[0008] According to the measured coordinates of the plurality of first targets sensed by the first sensor, a first search area is set for each first target, and according to the measured coordinates of the plurality of second targets sensed by the second sensor, a second search area is set for each second target;

[0009] For each first target, the degree of overlap of the first search area corresponding to the first target and the second search area of each second target is obtained, and the first sensing data and the second sensing data corresponding to the first target and the second target satisfying the preset condition are obtained;

[0010] According to the first perception data and the second perception data corresponding to the first target and at least one second target respectively, the second target matched with the first target is obtained when the coincidence degree meets a preset condition.

[0011] Optionally, the first search area for each first target is set according to the measured coordinate of each first target sensed by the first sensor, including:

[0012] The first search area is set according to the measurement error of the x-axis and the y-axis of the first sensor, the first preset length, and the measured coordinate of each first target, wherein the first search area is a rectangle, the measured coordinate is the center of the rectangle, and the first preset length is used to determine the length and width of the rectangle.

[0013] The second search area for each second target is set according to the measured coordinate of each second target sensed by the second sensor, including:

[0014] The second search area is set according to the measurement error of the x-axis and the y-axis of the second sensor, the second preset length, and the measured coordinate of each second target, wherein the second search area is a rectangle, the measured coordinate is the center of the rectangle, and the second preset length is used to determine the length and width of the rectangle.

[0015] Optionally, the coincidence degree of the first search area corresponding to the first target and the second search area of each second target is obtained for each first target, including:

[0016] The area of the overlapping region of the first search area and the second search area is determined according to the center, length and width of the first search area, and the center, length and width of the second search area.

[0017] The coincidence degree is determined according to the area of the overlapping region, the first area of the first search area, the second area of the second search area, and the first search area and the second search area.

[0018] Optionally, the method further comprises:

[0019] The coincidence degree is compared with an overlap threshold value, and if the coincidence degree is greater than the overlap threshold value, it is determined that the coincidence degree meets a preset condition.

[0020] Optionally, the second target matched with the first target is obtained according to the first perception data and the second perception data corresponding to the first target and at least one second target respectively when the coincidence degree meets a preset condition, including:

[0021] According to the first perception data and the second perception data corresponding to the first target and each of the at least one second target respectively, the similarity of the first target and each of the at least one second target is obtained when the coincidence degree meets a preset condition.

[0022] According to the similarity of the first target and each of the at least one second target, the second target matched with the first target is obtained by using a local nearest neighbor association algorithm.

[0023] Optionally, the similarity of the first target and each of the at least one second target is obtained according to the first perception data and the second perception data corresponding to the first target and each of the at least one second target respectively when the coincidence degree meets a preset condition, and the similarity of the first target and each of the at least one second target comprises:

[0024] For the first target and any second target, the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance and the orientation Euclidean distance of the two targets are obtained.

[0025] According to the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance, the orientation Euclidean distance, the coincidence degree and the weight corresponding to each of them, the similarity of the first target and the second target is obtained.

[0026] Optionally, the first perception data of each first target sensed by the first sensor and the second perception data of each second target sensed by the second sensor are obtained

[0027] The first raw data sensed by the first sensor is obtained, and the first raw data is converted into a vehicle coordinate system to obtain the first perception data.

[0028] The second raw data sensed by the second sensor is obtained, and the second raw data is converted into a vehicle coordinate system to obtain the second perception data.

[0029] On the other hand, the application provides a target matching device based on multiple sensors, comprising:

[0030] The acquisition module is configured to acquire the first perception data of each first target sensed by the first sensor and the second perception data of each second target sensed by the second sensor, wherein the first perception data and the second perception data each comprise the relative distance, the relative speed and the orientation of the target.

[0031] The setting module is configured to set a first search area for each of the first targets according to the measured coordinates of the multiple first targets sensed by the first sensor, and set a second search area for each of the second targets according to the measured coordinates of the multiple second targets sensed by the second sensor.

[0032] The computing module is configured to, for each first target, acquire a coincidence degree of a first search region corresponding to the first target and a second search region of each second target, and acquire first perception data and second perception data corresponding to the first target and the second target respectively, which satisfy a preset condition;

[0033] The matching module is configured to acquire a second target matched with the first target according to the first perception data and the second perception data corresponding to the first target and the at least one second target respectively, which satisfy the preset condition.

[0034] Optionally, the setting module is specifically configured to:

[0035] The first search region is a rectangle, the measured coordinate is the center of the rectangle, and the first preset length is used to determine the length and width of the rectangle.

[0036] The second search region is set for each second target according to the measured coordinate of each second target sensed by the second sensor, and the setting includes:

[0037] The second search region is a rectangle, the measured coordinate is the center of the rectangle, and the second preset length is used to determine the length and width of the rectangle.

[0038] Optionally, the computing module is specifically configured to:

[0039] The area of the overlapping region of the first search region and the second search region is determined according to the center, length and width of the first search region and the center, length and width of the second search region.

[0040] The coincidence degree of the first search region and the second search region is determined according to the area of the overlapping region, the first area of the first search region, the second area of the second search region.

[0041] Optionally, the computing module is specifically configured to:

[0042] The coincidence degree is compared with an overlap threshold value, and if the coincidence degree is greater than the overlap threshold value, it is determined that the coincidence degree satisfies the preset condition.

[0043] Optionally, the matching module is specifically configured to:

[0044] According to the first perception data and the second perception data corresponding to the first target and each second target respectively, the similarity of the first target and each second target is obtained according to the coincidence degree satisfying the preset condition;

[0045] According to the similarity of the first target and each second target, the second target matched with the first target is obtained by using a local nearest neighbor correlation algorithm.

[0046] Optionally, the matching module is specifically configured to:

[0047] For the first target and any second target, the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance and the orientation Euclidean distance of the two targets are obtained.

[0048] According to the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance, the orientation Euclidean distance, the coincidence degree, and the respective corresponding weights, the similarity of the first target and the second target is obtained.

[0049] Optionally, the obtaining module is specifically configured to:

[0050] The first raw data sensed by the first sensor is obtained, and the first raw data is converted to the vehicle coordinate system to obtain the first perception data.

[0051] The second raw data sensed by the second sensor is obtained, and the second raw data is converted to the vehicle coordinate system to obtain the second perception data.

[0052] The processor and the memory;

[0053] The memory stores computer execution instructions;

[0054] The processor executes the computer execution instructions stored in the memory, so that the electronic device executes the method of any one of the first aspect.

[0055] The fourth aspect of the present application provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the processor executes the computer execution instructions, the computer execution instructions are used to realize the determination method of the driver program of the hardware peripheral of any one of the first aspect.

[0056] The embodiment provides a multi-sensor-based target matching method, device and equipment and a storage medium, which comprises the following steps: acquiring first sensing data of each first target sensed by a first sensor and second sensing data of each second target sensed by a second sensor; setting a first search area for each first target according to measured coordinates of the plurality of first targets sensed by the first sensor, and setting a second search area for each second target according to measured coordinates of the plurality of second targets sensed by the second sensor; for each first target, acquiring the coincidence degree of the first search area corresponding to the first target and the second search area of each second target, and acquiring the first sensing data and the second sensing data corresponding to the first target and the second target respectively when the coincidence degree meets a preset condition; and acquiring the second target matched with the first target according to the first sensing data and the second sensing data corresponding to the first target and at least one second target respectively when the coincidence degree meets the preset condition. The method sets a search area for a target according to sensing data sensed by different sensors, acquires a search area coincidence degree, and then acquires a matching target, thereby solving the problem that a multi-sensor system has low target matching efficiency due to different data dimensions and ranges and observation noise. BRIEF DESCRIPTION OF DRAWINGS

[0057] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0058] Figure 1 A specific application scenario diagram of the multi-sensor-based target matching method provided by the present application is shown in the following figure:

[0059] Figure 2 A multi-sensor-based target matching method provided by the embodiment of the present application is shown in the following figure: Figure 1 ;

[0060] Figure 3a A multi-sensor-based target matching method provided by the embodiment of the present application is shown in the following figure: Figure 2 ;

[0061] Figure 3b A search area diagram provided by the embodiment of the present application is shown in the following figure:

[0062] Figure 4 A multi-sensor-based target matching method provided by the embodiment of the present application is shown in the following figure:

[0063] Figure 5 A multi-sensor-based target matching device provided by the embodiment of the present application is shown in the following figure:

[0064] Figure 6 A hardware structure diagram of a multi-sensor-based target matching device provided by the embodiment of the present application is shown in the following figure.

[0065] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0066] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0067] Figure 1 A diagram illustrating a specific application scenario of the multi-sensor-based target matching method provided in this application. (Example:) Figure 1 As shown, the application scenario includes: target 101, target 102, and radars A103 and B104 installed on vehicle 105. For example, when there are two close targets 101 and 102 in front of vehicle 105, and the two radars installed on the vehicle are not of the same type (radar A103 is a millimeter-wave radar, and radar B104 is a lidar), the two radars respectively perceive targets 101 and 102. The obtained perception data needs to be matched before further processing can proceed. This involves processing the data of the two targets perceived by radar A103 and the data of the two targets perceived by radar B104 to determine the millimeter-wave radar data and lidar data of target 101, and the millimeter-wave radar data and lidar data of target 102. Existing technologies lack methods for rapid target matching.

[0068] This application provides a target matching method based on multiple sensors. This method sets a search area for the target based on the perception data of the target perceived by different sensors, obtains the overlap degree of the search area, and then matches different targets acquired by multiple sensors based on the perception data of targets whose overlap degree meets preset conditions. This solves the problem of low target matching efficiency caused by observation noise and the different data dimensions and ranges caused by multiple sensors in multi-sensor systems.

[0069] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0070] Figure 2 A multi-sensor-based target matching method provided for an embodiment of the present application Figure 1 As shown in Figure 2 The method of the present embodiment includes:

[0071] S201, acquiring first perception data of each first target perceived by a first sensor and second perception data of each second target perceived by a second sensor, the first perception data and the second perception data each including a relative distance, a relative speed and an orientation of the target;

[0072] In the present embodiment, the first sensor is any one of a plurality of sensors installed on a vehicle body, and the second sensor is any one of a plurality of sensors installed on the vehicle body. The first target is a target in the vehicle's surrounding environment within the perception range of the first sensor, and the number of first targets can be one or more. The second target is a target in the vehicle's surrounding environment within the perception range of the second sensor, and the number of second targets can be one or more. When both the first target and the second target are multiple, the second target matched with the first target, i.e., the first target and the second target correspond to the same target, needs to be obtained.

[0073] The first perception data is data collected by the first sensor for the first target, and the second perception data is data collected by the second sensor for the second target. The first perception data and the second perception data each include a relative distance, a relative speed and an orientation of the target.

[0074] The relative distance of the target refers to the distance of the target relative to the origin of the vehicle coordinate system, the relative speed refers to the speed of the target relative to the origin of the vehicle coordinate system, and the orientation refers to the angle of the target relative to the origin of the vehicle coordinate system.

[0075] Automatic driving target detection relies on multiple sensors, and cameras, millimeter wave radars and laser radars are the most important and most widely used three types of sensors. In the present embodiment, the first sensor and the second sensor can be any one of a camera, a millimeter wave radar and a laser radar.

[0076] S202, setting a first search area for each first target according to the measured coordinates of the plurality of first targets perceived by the first sensor, and setting a second search area for each second target according to the measured coordinates of the plurality of second targets perceived by the second sensor;

[0077] In this embodiment, the measured coordinate of the first target refers to the specific coordinate of the first target relative to the vehicle coordinate system, and the first search region refers to a region of a fixed area centered on the measured coordinate of the first target and pre-set. The measured coordinate of the second target refers to the specific coordinate of the second target relative to the vehicle coordinate system, and the second search region refers to a region of a fixed area centered on the measured coordinate of the second target and pre-set.

[0078] The search region is set because there is noise in the sensor, and therefore the perception data of the perceived target is accurate within a certain range, and therefore the set search region ensures that the perceived target is within the search region, thereby establishing a basis for the next step of target overlap calculation.

[0079] S203, for each first target, obtaining the overlap of the first search region corresponding to the first target and the second search region of each second target, and obtaining the first perception data and the second perception data corresponding to the first target and the second target respectively which meet the preset condition;

[0080] In this embodiment, for the multiple targets perceived by the first sensor, each target is taken as a first target, and the first target is taken as a reference, and the multiple targets perceived by the second sensor are each taken as a second target, and the overlap of the first search region corresponding to each first target and the second search region corresponding to each second target is obtained in turn. The overlap refers to the degree of overlap of the two search regions.

[0081] In order to eliminate unmatched targets in advance, a preset condition, i.e., a threshold of the overlap, is set to screen out the first target and the second target which meet the preset condition. At this time, the first target and the second target have a certain possibility of being the same target, and therefore the first perception data and the second perception data corresponding to the first target and the second target respectively are obtained for further processing.

[0082] S204, according to the first perception data and the second perception data corresponding to the first target and at least one second target which meet the preset condition, obtaining the second target matched with the first target.

[0083] In this embodiment, after the previous steps, the range of target matching has been narrowed, and on this basis, a data association method is adopted to obtain the matched target. For example, the nearest neighbor method, joint probability data association, multiple hypothesis tracking, and similarity algorithm, etc. The implementation of the data association method is not particularly limited in this embodiment, as long as the second target matched with the first target can be obtained.

[0084] The embodiment provides a multi-sensor-based target matching method, which comprises the following steps: acquiring first perception data of each first target perceived by a first sensor and second perception data of each second target perceived by a second sensor; setting a first search area for each first target according to measured coordinates of the plurality of first targets perceived by the first sensor; setting a second search area for each second target according to measured coordinates of the plurality of second targets perceived by the second sensor; for each first target, acquiring the overlap degree of the first search area corresponding to the first target and the second search area of each second target, and acquiring the first perception data and the second perception data corresponding to the first target and the second target respectively when the overlap degree meets a preset condition; and acquiring the second target matched with the first target according to the first perception data and the second perception data corresponding to the first target and at least one second target respectively when the overlap degree meets the preset condition. The method sets a search area for a target according to perception data perceived by different sensors, acquires the search area overlap degree, and then acquires a matched target, so that the problem of low target matching efficiency of a multi-sensor system caused by different data dimensions and ranges and observation noise is solved.

[0085] Figure 3a The multi-sensor-based target matching method provided in the embodiment Figure 2 , Figure 3b The search area schematic diagram provided in the embodiment is shown in Figure 2 The process of acquiring the overlap degree is described in detail in the embodiment. As shown in Figure 3a The method comprises the following steps:

[0086] S301, acquiring first raw data perceived by a first sensor, converting the first raw data to a vehicle coordinate system to obtain first perception data;

[0087] In the embodiment, the first raw data refers to data about a first target acquired by the first sensor, and the data is in a coordinate system with the first sensor as the origin. The vehicle coordinate system refers to a coordinate system with the center of the rear axle of the vehicle as the origin of the vehicle coordinate system, a front direction as the x-axis and a left side as the y-axis. The first perception data is data obtained by converting the raw data of the first target.

[0088] S302, acquiring second raw data perceived by a second sensor, converting the second raw data to the vehicle coordinate system to obtain second perception data;

[0089] In the embodiment, the second raw data refers to data about a second target acquired by the second sensor, and the data is in a coordinate system with the second sensor as the origin. The second perception data is data obtained by converting the raw data of the second target.

[0090] S303, setting a first search region according to the measurement errors of the x-axis and y-axis of the first sensor, the first preset length, and the measured coordinates of each first target, wherein the first search region is a rectangle, the measured coordinates are the center of the rectangle, and the first preset length is used to determine the length and width of the rectangle;

[0091] In this embodiment, the first sensor is a laser radar, and the measured coordinates of the first target are the coordinates of the target in the vehicle coordinate system. Assuming that the measured coordinates of the first target are (Rx, Ry), the x-axis measurement error is δx, the y-axis measurement error is δy, and the first preset length is used to determine the length and width of the rectangle, which is set to 2 meters here, the first search region is a rectangle with a length of 2+a*|δy| and a width of 2c+b*|δx| and with (Rx, Ry) as the center, where a and b are set to natural numbers, and c is greater than or equal to 0 and less than or equal to 1.

[0092] S304, setting a second search region according to the measurement errors of the x-axis and y-axis of the second sensor, the second preset length, and the measured coordinates of each second target, wherein the second search region is a rectangle, the measured coordinates are the center of the rectangle, and the second preset length is used to determine the length and width of the rectangle.

[0093] In this embodiment, the second sensor is a millimeter wave radar, and the measured coordinates of the second target are the coordinates of the target in the vehicle coordinate system. Assuming that the measured coordinates of the second target are (Lx, Ly), the x-axis measurement error is εx, the y-axis measurement error is εy, and the first preset length is used to determine the length and width of the rectangle, which is set to 2 meters here, the first search region is a rectangle with a length of 2+a*|εy| and a width of 2c+b*|εx| and with (Lx, Ly) as the center, where a and b are set to natural numbers, and c is greater than or equal to 0 and less than or equal to 1.

[0094] S305, determining the area of the overlapping region of the first search region and the second search region according to the center, length and width of the first search region and the center, length and width of the second search region.

[0095] According to the center, length and width of the first search region and the center, length and width of the second search region, the area of the overlapping region of the two can be determined in the same coordinate system. For example, as shown in Figure 3b According to the measured coordinates (Rx, Ry) of the first target, the measured coordinates (Lx, Ly) of the second target, the length of the first search region is 2+a*|δy| and the width is 2c+b*|δx|, the length of the second search region is 2+a*|εy| and the width is 2c+b*|εx|, the fixed point coordinates and length of m and n can be calculated respectively, and then the area SQ of the overlapping region is obtained.

[0096] S306, according to the area of the overlapping region, the first area of the first search region, the second area of the second search region, the coincidence degree of the first search region and the second search region;

[0097] In this embodiment, the area SR of the first search region is 2*|δx|(2+2*|δy|), and the area SL of the second search region is 2*|εx|(2+2*|εy|).

[0098] In this embodiment, the coincidence degree IOU is the ratio of the area SQ of the overlapping region to the smaller one of the area SR of the first search region and the area SL of the second search region. It can be calculated according to the following formula.

[0099] IOU = SQ / min(SL, SQ)

[0100] S307, comparing the coincidence degree with the overlap threshold value, if the coincidence degree is greater than the overlap threshold value, it is determined that the coincidence degree meets the preset condition.

[0101] In this embodiment, the overlap threshold value is set to 0.5, that is, when the coincidence degree IOU is greater than or equal to 0.5, the first target and the second target have a matching possibility, forming a matchable target group; when the coincidence degree IOU is less than 0.5, the first target and the second target have a matching possibility of 0, and the second target is filtered out.

[0102] The technical scheme of the present application will be described in detail below with reference to a specific embodiment.

[0103] Suppose that the first target of the first sensor has 7, which are R1, R2, R3, R4, R5, R6 and R7, and the second target of the second sensor has 5, which are L1, L2, L3, L4 and L5. After comparison with the coincidence threshold value, the matching group is as shown in Table 1, wherein "-" represents that the coincidence degree is lower than the threshold value, and there is no matching possibility; "√" represents that the coincidence degree is higher than the threshold value, and there is a matching possibility. According to Table 1, the matchable targets are: (1) L1 and R1, R4; (2) L2 and R2; (3) L3 and R2, R6; (4) L4 and R3; (5) L5 and R5, R6. Here, other unmatched target groups are filtered out, and the matching efficiency is improved.

[0104] Table 1

[0105]

[0106] This embodiment provides a target matching method based on multiple sensors. The method acquires first raw data perceived by a first sensor and converts it to a vehicle coordinate system to obtain first perception data; acquires second raw data perceived by a second sensor and converts it to a vehicle coordinate system to obtain second perception data; sets a first search area based on the measurement errors of the first sensor's x-axis and y-axis dimensions, a first preset length, and the measured coordinates of each first target; sets a second search area based on the measurement errors of the second sensor's x-axis and y-axis dimensions, a second preset length, and the measured coordinates of each second target; determines the area of ​​the overlapping region between the first and second search areas based on the center, length, and width of the first and second search areas; calculates the overlap degree between the overlapping area, the first area of ​​the first search area, the second area of ​​the second search area, and the overlap ratio of the first and second search areas; compares the overlap degree with an overlap threshold; if the overlap degree is greater than the overlap threshold, it is determined that the overlap degree meets a preset condition. This method uses the same coordinate system of the original data, sets up a search area that takes into account the error of different sensors, calculates the overlap degree, compares it with the threshold, and eliminates a batch of mismatched targets in advance, which narrows the range for subsequent target matching and improves efficiency and accuracy.

[0107] Figure 4 Flowchart 3 illustrates the multi-sensor-based target matching method provided in this application embodiment. Based on the above embodiments, a detailed process is given of obtaining the similarity between the first target and each second target by using the first and second sensing data corresponding to a first target and at least one second target that satisfy preset overlap conditions, thereby determining the matching target.

[0108] In this embodiment, the similarity calculation methods include: Euclidean distance, cosine similarity, Pearson correlation coefficient, modified cosine similarity, Hamming distance, and Manhattan distance. This embodiment does not impose any particular limitation on the specific similarity algorithm; for ease of explanation, Euclidean distance will be used as an example for detailed description.

[0109] like Figure 4 As shown, the method in this embodiment includes:

[0110] S401. For the first target and any second target, obtain the Euclidean distance between the two targets in the x-direction, the Euclidean distance between the two targets in the y-direction, the Euclidean distance between the two targets in the relative velocity direction, and the Euclidean distance in the orientation direction.

[0111] In this embodiment, the measured coordinates of the first target are (Rx, Ry), and the measured coordinates of the second target are (Lx, Ly). Therefore, the Euclidean distance d1 between the two targets in the x-direction is...

[0112]

[0113] wherein Sx is the standard deviation of the x direction of the relative distance component of the first target and the second target.

[0114] The relative distance Euclidean distance d2 of the y direction of the two targets is

[0115]

[0116] wherein Sy is the standard deviation of the y direction of the relative distance component of the first target and the second target.

[0117] The relative velocity Euclidean distance d3 of the two targets is

[0118]

[0119] wherein Sx2 is the standard deviation of the x direction of the relative velocity component of the first target and the second target, and Sy2 is the standard deviation of the y direction of the relative velocity component of the first target and the second target.

[0120] The orientation Euclidean distance d4 of the two targets is

[0121]

[0122] wherein S3 is the standard deviation of the orientation component of the first target and the second target.

[0123] S402, according to the relative distance Euclidean distance of the x direction, the relative distance Euclidean distance of the y direction, the relative velocity Euclidean distance, the orientation Euclidean distance, the coincidence degree, and the corresponding weight of each, the similarity of the first target and the second target is obtained;

[0124] In the embodiment, the corresponding weights Q1, Q2, Q3, Q4, Q5 of the relative distance Euclidean distance d1 of the x direction, the relative distance Euclidean distance d2 of the y direction, the relative velocity Euclidean distance d3, the orientation Euclidean distance d4, and the coincidence degree IOU are respectively given, and then the similarity X of the first target and the second target is

[0125] X=d1*Q1+d2*Q2+d3*Q3+d4*Q4+d5*Q5

[0126] S403, according to the similarity of the first target and each second target, the second target matched with the first target is obtained by using the local nearest neighbor correlation algorithm.

[0127] K-Nearest Neighbor (KNN) is a non-parametric statistical method for classification and regression. In KNN classification, the input contains the k closest training samples in the feature space, and the output is a classification group. The classification of an object is determined by the "majority vote" of its neighbors, and the most common classification among the k nearest neighbors (k is a positive integer, usually small) determines the class assigned to the object. For example, there is a first target R1, three second targets L1, L2 and L3, if the class of R1 is to be obtained, that is, to belong to the three second targets as three categories. If the similarity X11 between R1 and L1 is 0.2, the similarity X12 between R1 and L2 is 0.3, and the similarity between R1 and L3 is 0.4, when 2 nearest neighbors are selected, that is, k is 2, the nearest neighbors of R1 are L1 and L2, and the value of L1 is smaller, then R1 and L1 are in the same class, that is, R1 and L1 match in this embodiment.

[0128] The embodiment provides a target matching method based on multiple sensors. The method comprises the following steps: acquiring the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance and the direction Euclidean distance of the two targets for the first target and any second target; acquiring the similarity between the first target and the second target according to the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance, the direction Euclidean distance and the corresponding weight; and obtaining the second target matched with the first target through the K-Nearest Neighbor algorithm according to the similarity between the first target and each second target. The method calculates the Euclidean distance between the targets, combines the coincidence degree and gives different weights, adopts the K-Nearest Neighbor algorithm to obtain the matched target, comprehensively considers various factors affecting target matching, simplifies the calculation process, improves the target matching efficiency, and increases the accuracy of the target matching result.

[0129] Figure 5 A structure diagram of a target matching device based on multiple sensors is provided for the embodiment of the application. The device of the embodiment can be in the form of software and / or hardware. As shown in the figure, the target matching device 500 based on multiple sensors provided by the embodiment of the application comprises an acquisition module 501, a setting module 502, a calculation module 503 and a matching module 504, Figure 5

[0130] The acquisition module 501 is used to acquire the first perception data of each first target perceived by the first sensor and the second perception data of each second target perceived by the second sensor, and the first perception data and the second perception data both comprise the relative distance, the relative speed and the direction of the target.

[0131] ​The setting module 502 is configured to set a first search region for each first target according to the respective measured coordinates of the plurality of first targets sensed by the first sensor, and set a second search region for each second target according to the respective measured coordinates of the plurality of second targets sensed by the second sensor;

[0132] The calculation module 503 is configured to, for each first target, acquire the coincidence degree of the first search region corresponding to the first target and the second search region of each second target, and acquire the respective first sensing data and second sensing data of the first target and the second target corresponding to the coincidence degree satisfying the preset condition;

[0133] The matching module 504 is configured to acquire the second target matched with the first target according to the respective first sensing data and second sensing data of the first target and at least one second target corresponding to the coincidence degree satisfying the preset condition.

[0134] In a possible implementation, the setting module is specifically configured to:

[0135] The first search region is set according to the measurement errors of the x-axis and y-axis of the first sensor, the first preset length, and the respective measured coordinates of each first target, wherein the first search region is a rectangle, the measured coordinates are the center of the rectangle, and the first preset length is used to determine the length and width of the rectangle.

[0136] The second search region is set for each second target according to the respective measured coordinates of the plurality of second targets sensed by the second sensor, including:

[0137] The second search region is set according to the measurement errors of the x-axis and y-axis of the second sensor, the second preset length, and the respective measured coordinates of each second target, wherein the second search region is a rectangle, the measured coordinates are the center of the rectangle, and the second preset length is used to determine the length and width of the rectangle.

[0138] In a possible implementation, the calculation module is specifically configured to:

[0139] The area of the overlapping region of the first search region and the second search region is determined according to the center, length and width of the first search region, and the center, length and width of the second search region.

[0140] The coincidence degree is determined according to the area of the overlapping region, the first area of the first search region, the second area of the second search region, and the first search region and the second search region.

[0141] In a possible implementation, the calculation module is specifically configured to:

[0142] The coincidence degree is compared with the overlap threshold value, and if the coincidence degree is greater than the overlap threshold value, it is determined that the coincidence degree satisfies the preset condition.

[0143] In a possible implementation, the matching module is specifically configured to:

[0144] According to the first perception data and the second perception data corresponding to the first target and each second target respectively, the similarity of the first target and each second target is obtained according to the coincidence degree satisfying a preset condition;

[0145] According to the similarity of the first target and each second target, the second target matched with the first target is obtained by using a local nearest neighbor correlation algorithm.

[0146] In a possible implementation, the matching module is specifically configured to:

[0147] For the first target and any second target, the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance, and the orientation Euclidean distance of the two targets are obtained.

[0148] According to the relative distance Euclidean distance in the x direction, the relative distance Euclidean distance in the y direction, the relative speed Euclidean distance, the orientation Euclidean distance, and the coincidence degree, and the weights corresponding thereto, the similarity of the first target and the second target is obtained.

[0149] In a possible implementation, the obtaining module is specifically configured to:

[0150] The first raw data sensed by the first sensor is obtained, and the first raw data is converted to the vehicle coordinate system to obtain the first perception data.

[0151] The second raw data sensed by the second sensor is obtained, and the second raw data is converted to the vehicle coordinate system to obtain the second perception data.

[0152] Figure 6 A hardware structure diagram of the target matching device based on multiple sensors provided by the embodiments of the present application is provided. As shown in the figure, Figure 6 The target matching device based on multiple sensors 600 includes:

[0153] a processor 601 and a memory 602;

[0154] The memory stores computer execution instructions;

[0155] The processor executes the computer execution instructions stored in the memory 602, so that the electronic device executes the target matching method based on multiple sensors as described above.

[0156] It should be appreciated that the processor 601 described above can be a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), or the like. The general-purpose processor can be a microprocessor or the processor can be any conventional processor. The steps of the methods disclosed in connection with the present application can be directly embodied in the hardware processor or a combination of hardware processor and software modules in the processor. The memory 602 can include a random access memory (RAM) and can further include a non-volatile memory (NVM), e.g., at least one disk memory, and can be a USB flash disk, a mobile hard disk, a read-only memory, a magnetic disk or an optical disk, or the like.

[0157] The embodiments of the present application also provide a computer readable storage medium, and the computer readable storage medium stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement a target matching method based on multiple sensors.

[0158] Other embodiments of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the present application cover any and all variations of the application that come within the scope of the general inventive concepts described herein. The specification and examples are illustrative only and not restrictive of the present application. The true scope and spirit of the application is indicated by the appended claims.

[0159] It should be understood that the application is not limited to the precise construction that has been described above and shown in the accompanying drawings, and that various modifications and changes can be made by those skilled in the art without departing from the scope of the application. The scope of the application is limited only by the appended claims.

Claims

1. A multi-sensor based target matching method, characterized by, The method comprises: acquiring first perception data of each first target perceived by a first sensor and second perception data of each second target perceived by a second sensor, the first perception data and the second perception data each comprising a relative distance, a relative speed and an orientation of a target; setting a first search region for each first target according to a measurement error of an x-axis and a y-axis of the first sensor, a first preset length and a measured coordinate of each first target, wherein the first search region is a rectangle, the measured coordinate is a center of the rectangle, and the first preset length is used to determine a length and a width of the rectangle; setting a second search region for each second target according to a measurement error of an x-axis and a y-axis of the second sensor, a second preset length and a measured coordinate of each second target, wherein the second search region is a rectangle, the measured coordinate is a center of the rectangle, and the second preset length is used to determine a length and a width of the rectangle; determining an area of an overlapping region of the first search region and the second search region for each first target according to a center, a length and a width of the first search region and a center, a length and a width of the second search region; acquiring a coincidence degree of the first search region corresponding to the first target and the second search region of each second target according to the area of the overlapping region, a first area of the first search region and a second area of the second search region, and acquiring first perception data and second perception data corresponding to a first target and a second target each satisfying a preset condition, wherein the coincidence degree is a ratio of the area of the overlapping region to a smaller one of the first area and the second area; acquiring a second target matching the first target according to the first perception data and the second perception data corresponding to the first target and the at least one second target each satisfying the preset condition.

2. The method of claim 1, wherein, The method further comprises: comparing the coincidence degree with an overlap threshold value, and determining that the coincidence degree satisfies the preset condition if the coincidence degree is greater than the overlap threshold value.

3. The method of claim 1, wherein, The acquiring of the second target matching the first target according to the first perception data and the second perception data corresponding to the first target and the at least one second target each satisfying the preset condition comprises: acquiring a similarity of the first target and each second target according to the first perception data and the second perception data corresponding to the first target and the at least one second target each satisfying the preset condition; obtaining the second target matching the first target through a local nearest neighbor correlation algorithm according to the similarity of the first target and each second target.

4. The method of claim 3, wherein, The acquiring of the similarity of the first target and each second target according to the first perception data and the second perception data corresponding to the first target and the at least one second target each satisfying the preset condition comprises: acquiring a relative distance Euclidean distance in an x direction, a relative distance Euclidean distance in a y direction, a relative speed Euclidean distance and an orientation Euclidean distance of the two targets for the first target and any second target. The similarity between the first target and the second target is obtained based on the relative distance Euclidean distance in the x-direction, the relative distance Euclidean distance in the y-direction, the relative velocity Euclidean distance, the orientation Euclidean distance, the degree of overlap, and their respective weights.

5. The method of claim 1, wherein, The first sensing data of each first target sensed by the first sensor and the second sensing data of each second target sensed by the second sensor are obtained. Acquire the first raw data sensed by the first sensor, and transform the first raw data into the vehicle coordinate system to obtain the first sensed data; The second raw data sensed by the second sensor is acquired, and the second raw data is transformed into the vehicle coordinate system to obtain the second sensed data.

6. A multi-sensor based object matching device for implementing the multi-sensor based object matching method of any one of claims 1-5, characterized by include: The acquisition module is used to acquire first perception data of each first target perceived by the first sensor and second perception data of each second target perceived by the second sensor. Both the first perception data and the second perception data include the relative distance, relative speed and orientation of the target. The setting module is used to set a first search area for each first target based on the measured coordinates of each of the multiple first targets sensed by the first sensor, and to set a second search area for each of the multiple second targets sensed by the second sensor. The calculation module is used to obtain the overlap degree between the first search region corresponding to the first target and the second search region of each second target for each first target, and to obtain the first perception data and second perception data corresponding to the first target and the second target respectively whose overlap degree meets the preset conditions; The matching module is used to obtain a second target that matches the first target based on the first perception data and the second perception data corresponding to the first target and at least one second target, respectively, which meet the preset conditions for overlap.

7. An electronic device comprising: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method of any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement a multi-sensor-based target matching method as described in any one of claims 1 to 5.

Images