Target-aware observation point cloud alignment method, device, and storage medium

Abstract

This application discloses a target-aware observation point cloud alignment method, device, and storage medium, relating to the field of target tracking technology. It aims to at least improve the accuracy of acquiring observation point clouds of moving targets, thereby improving the accuracy of target processing using these observation point clouds. The method includes: performing at least one round of trajectory processing operations on the moving target. The trajectory processing operations include: obtaining prior trajectory information of the moving target; performing point cloud alignment operations on each sensor based on the prior trajectory information and the observation point clouds corresponding to each sensor to obtain aligned point clouds corresponding to each sensor; the point cloud alignment operations include: performing trajectory filtering based on the prior trajectory information and the observation point clouds corresponding to the sensors to obtain posterior trajectory information associated with the sensors; estimating target trajectory information at least through the posterior trajectory information; and correcting the observation point clouds based on the trajectory offset information of the posterior trajectory information and the target trajectory information to obtain the corresponding aligned point clouds.

Description

Technical Field

[0001] This application relates to the field of target tracking, and in particular to a target-aware observation point cloud alignment method, device, and storage medium. Background Technology

[0002] In a fusion perception system for moving targets, multiple sensors collect observation point clouds of the same moving target. The system operates on a target detection cycle, with time differences in the detection times of different sensors for the same moving target within each cycle. Each sensor independently tracks the target using its own observation point cloud, obtaining its own target trajectory information. The fusion perception system then directly fuses this target trajectory information to output the final fusion perception result. However, each sensor often only uses its own local observation point cloud during the entire target tracking process. Subsequent target-related processing relies solely on the filtered local information from each sensor's trajectory, leading to insufficient accuracy in the final target processing result. For example, fusing and detecting target bounding boxes using only the filtered local information from each sensor's trajectory only utilizes a local point cloud of the target, potentially causing discrepancies between the predicted and actual target volume. This significantly impacts the accuracy of the target bounding box prediction, especially for larger targets such as cars / trucks, where the predicted bounding box may differ considerably from the actual bounding box. Summary of the Invention

[0003] This application provides a target-aware observation point cloud alignment method, device, and storage medium to at least improve the accuracy of the observation point cloud of the sensor for the moving target, thereby improving the accuracy of target processing of the moving target using the observation point cloud collected by multiple sensors.

[0004] Firstly, this application provides a target-aware observation point cloud alignment method, which includes:

[0005] Perform at least one round of trajectory processing operations on the moving target, the trajectory processing operations including:

[0006] Obtain prior trajectory information of a moving target, wherein the prior trajectory information includes at least the prior position information and trajectory speed information of the moving target;

[0007] Based on the prior track information and the observation point cloud corresponding to each sensor in at least one sensor, a point cloud alignment operation is performed on each sensor to obtain an aligned point cloud corresponding to each sensor. The point cloud alignment operation includes:

[0008] Based on the prior track information and the observation point cloud corresponding to the sensor, track filtering is performed to obtain the posterior track information associated with the sensor;

[0009] At least using the posterior position and velocity information in the posterior track information, the target track information corresponding to the target time can be estimated; and

[0010] Based on the track offset information of the posterior track information and the target track information, the observation point cloud corresponding to the sensor is corrected to obtain the alignment point cloud corresponding to the sensor.

[0011] In the method provided in this application embodiment, after obtaining posterior trajectory information and target trajectory information of the target at the time to be predicted based on the prior trajectory information of the moving target, the observation point cloud collected by the sensor is corrected based on the trajectory offset information between the target posterior trajectory information and the posterior trajectory information. That is, the observation point cloud is corrected by using the deviation between the predicted trajectory information and the real trajectory information, thereby obtaining an aligned point cloud. The obtained aligned point cloud is closer to the real shape of the moving target than the observation point cloud, and the obtained aligned point cloud contains more realistic and richer target features than the observation point cloud, thereby improving the accuracy of target processing such as target classification, target box detection, and part segmentation using the aligned point cloud corresponding to one or more sensors.

[0012] In one possible implementation, the observation point cloud corresponding to the sensor is corrected according to the following formula (1) to obtain the aligned point cloud:

[0013] PD t_sys =PD t1 +(PH t_sys -PH t1 )Formula (1)

[0014] In formula (1), PD t_sys For the position of the aligned point cloud, PD t1 The location of the observed point cloud, (PH) t_sys -PH t1 ) represents the track offset information, PH t1 For the aforementioned posterior track information, PH t_sys This refers to the target trajectory information.

[0015] In one possible implementation, the trajectory speed information includes the moving speed of the moving target, or the trajectory speed information includes the moving speed and acceleration of the moving target, and the observation point cloud corresponding to the sensor is collected for the moving target at the observation time corresponding to the sensor;

[0016] The step of estimating the target trajectory information corresponding to the target time using at least the posterior position information and trajectory velocity information in the posterior trajectory information includes:

[0017] The position offset information is determined based on the observation time difference and the trajectory velocity information in the posterior trajectory information, wherein the observation time difference is the difference between the observation time corresponding to the sensor and the target time;

[0018] The target position information is determined based on the posterior position information in the posterior track information and the position offset information;

[0019] The target position information and the trajectory speed information are updated into the target trajectory information.

[0020] In one possible implementation, the prior track information also includes the angular deflection information of the moving target;

[0021] The step of determining the position offset information based on the observation time difference and the track velocity information in the posterior track information includes: determining the position offset information based on the observation time difference and the track velocity information in the posterior track information and the angle deflection information;

[0022] Determining the target trajectory information based on the target position information and the trajectory speed information includes: determining the target trajectory information based on the target position information, the trajectory speed information, and the angle deflection information.

[0023] In one possible implementation, the prior track information further includes at least one of the following prior auxiliary information of the moving target: prior energy reflection, prior radial velocity, prior radial distance, prior horizontal angle, and prior pitch angle. The prior radial velocity is the sensor radial component of the target velocity, the prior radial distance represents the distance of the moving target relative to the corresponding sensor, the prior horizontal angle represents the horizontal angle of the moving target relative to the corresponding sensor, and the prior pitch angle represents the pitch angle of the moving target relative to the corresponding sensor.

[0024] When correcting the observation point cloud corresponding to the sensor based on the track offset information of the posterior track information and the target track information, the method further includes: estimating the posterior auxiliary information corresponding to each prior auxiliary information, and updating the estimated posterior auxiliary information into the target track information.

[0025] In one possible implementation, estimating the posterior auxiliary information corresponding to each prior auxiliary information and updating the estimated posterior auxiliary information to the target track information includes:

[0026] When the prior energy reflection information includes the prior energy reflection information, the prior energy reflection information is determined as the posterior energy reflection information, and the posterior energy reflection information is updated in the target trajectory information.

[0027] When the prior trajectory information includes a prior pitch angle, the prior pitch angle is determined as a subsequent pitch angle, and the subsequent pitch angle is updated in the target trajectory information;

[0028] When the prior auxiliary information includes a prior horizontal angle, the posterior horizontal angle is determined based on the target position information in the target trajectory information, and the posterior horizontal angle is updated in the target trajectory information.

[0029] When the prior auxiliary information includes the prior radial velocity, the posterior radial velocity is determined based on the posterior horizontal angle and the trajectory velocity information in the target trajectory information, and the posterior radial velocity is updated in the target trajectory information.

[0030] When the prior auxiliary information includes a prior radial distance, the posterior radial distance is determined based on the target position information in the target track information, and the posterior radial distance is updated in the target track information.

[0031] In one possible implementation, the track speed information includes the moving speed of the moving target;

[0032] The posterior horizontal angle is determined according to the target position information in the target trajectory information according to the following formula (2); and / or

[0033] The posterior radial velocity is determined according to the following formula (3), based on the posterior horizontal angle and the trajectory velocity information in the target trajectory information; and / or,

[0034] The posterior radial distance is determined according to the target position information in the target trajectory information according to the following formula (4); wherein:

[0035] azimuth2=atan2(posy2,posx2) formula (2)

[0036] Formula (3) = d_vel2 = cos(azimuth2) × velx2 + sin(azimuth2) × vely2

[0037] range2=sqrt(posx2×posx2+posy2×posy2) formula (4)

[0038] In formulas (2) to (4), azimuth2 is the posterior horizontal angle, range2 is the posterior radial distance, and d_vel2 is the posterior radial velocity; posx2 and posy2 are the X-axis and Y-axis coordinates of the target position information in the two-dimensional coordinate system, respectively, and velx2 and vely2 are the velocity components of the target trajectory information in the X-axis and Y-axis of the two-dimensional coordinate system, respectively.

[0039] In one possible implementation, after performing point cloud alignment operations on each sensor based on the prior track information and the observation point clouds corresponding to each sensor in at least one sensor to obtain the aligned point cloud for each sensor, the method further includes:

[0040] The moving target is processed based on the alignment point cloud corresponding to each sensor. The target processing includes at least one of target classification, target bounding box detection, and target part segmentation.

[0041] In one possible implementation, the prior track information of the (i+1)th round in the at least one round of track processing operation is obtained based on the updated track information obtained in the ith round, and the prior track information of the first round is obtained based on preset track information, where i is an integer greater than 0;

[0042] When performing point cloud alignment operations on each sensor based on the prior track information and the observation point cloud corresponding to each sensor in at least one sensor to obtain the aligned point cloud corresponding to each sensor, the method further includes:

[0043] The target trajectory information corresponding to each sensor is fused to obtain the updated trajectory information.

[0044] Secondly, this application provides a target-aware observation point cloud alignment device, which includes a priori information acquisition unit and a point cloud alignment unit. The point cloud alignment unit includes a trajectory filtering unit, a trajectory estimation unit, and an offset alignment unit; wherein:

[0045] The prior information acquisition unit is used to obtain prior trajectory information of the moving target in each round of trajectory processing operation on the moving target. The prior trajectory information includes at least the prior position information and trajectory speed information of the moving target.

[0046] The point cloud alignment unit is used to perform point cloud alignment operation on each sensor based on the prior track information and the observation point cloud corresponding to each sensor in each round of track processing operation on the moving target, so as to obtain the aligned point cloud corresponding to each sensor.

[0047] In the point cloud alignment operation: the track filtering unit performs track filtering based on the prior track information and the observation point cloud corresponding to the sensor to obtain the posterior track information associated with the sensor; the track estimation unit estimates the target track information corresponding to the target time based at least on the posterior position information and track velocity information in the posterior track information; and the offset alignment unit corrects the observation point cloud corresponding to the sensor based on the track offset information of the posterior track information and the target track information to obtain the alignment point cloud corresponding to the sensor.

[0048] Thirdly, embodiments of this application also provide a target-aware observation point cloud alignment device, which includes a processor and a memory. The memory is used to store programs executable by the processor, and the processor is used to read the programs in the memory and execute any of the methods described in the first aspect of this application.

[0049] Fourthly, embodiments of this application also provide a computer storage medium having a computer program stored thereon, which, when executed by a processor, is used to implement the steps of the method described in the first aspect above.

[0050] These or other aspects of this application will become more apparent in the following description of embodiments. Attached Figure Description

[0051] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0052] Figure 1 This is a schematic diagram illustrating the detection principle for sensing moving targets, provided in an embodiment of this application.

[0053] Figure 2 This is a schematic diagram of the perception principle of a target perception system provided in an embodiment of this application;

[0054] Figure 3 A flowchart illustrating a target-aware observation point cloud alignment method provided in this application embodiment;

[0055] Figure 4 A flowchart illustrating the estimation of target trajectory information using posterior trajectory information is provided as an embodiment of this application.

[0056] Figure 5 A schematic diagram illustrating the principle of determining the posterior radial distance and the posterior horizontal angle, provided for an embodiment of this application;

[0057] Figure 6 A structural diagram of a target-aware observation point cloud alignment device provided in an embodiment of this application;

[0058] Figure 7 This is a structural diagram of a target-aware observation point cloud alignment device provided in an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0060] In the embodiments of this application, the term "and / or" describes the relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following associated objects have an "or" relationship.

[0061] The application scenarios described in this application are for the purpose of more clearly illustrating the technical solutions of this application, and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that with the emergence of new application scenarios, the technical solutions provided in this application are also applicable to similar technical problems. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0062] First, let's explain the key terms in the embodiments of this application:

[0063] Mobile target: refers to a movable device or object. In the embodiments of this application, the mobile target may include, but is not limited to, devices without life characteristics such as motor vehicles (such as cars, trucks, etc.), non-motor vehicles (such as bicycles), robots, etc. In the following description of the embodiments of this application, a vehicle is used as an example of a mobile target.

[0064] Sensors: Devices / appliances that can collect point clouds of moving targets, including but not limited to lidar and / or millimeter-wave radar.

[0065] Please see Figure 1 This is a schematic diagram illustrating the detection principle for sensing moving targets. In the process of sensing moving targets in this embodiment, a target detection period T is set that conforms to the fusion target perception system. In each target detection period T, a trajectory processing operation is performed on the moving target to obtain the updated trajectory information of the moving target in this target detection period T. Furthermore, in each target detection period T, N sensors can collect observation point clouds of the moving target, and then the target trajectory information obtained from each sensor is fused to obtain the updated trajectory information in the target detection period T. The aforementioned N can be a positive integer greater than or equal to 1. Please refer to [link to previous document]. Figure 2This paper provides a schematic diagram of the perception principle of a target perception system. The diagram uses a vehicle as the moving target. In each target detection cycle, sensors 1, 2, and 3 collect observation point clouds of the vehicle. Because there is a time difference in the collection and synchronization of the observation point clouds by different sensors, the times when sensors 1, 2, and 3 simultaneously observe the point cloud are t1, t2, and t3, respectively. The target time within a target detection cycle T is denoted as t_sys. Please refer to [link to relevant documentation]. Figure 2 Thus, during a target detection cycle T, the target perception system can acquire the updated track information obtained from the track processing operation performed in the previous target detection cycle T, as prior track information, and then perform the following operations on sensor 1, sensor 2, and sensor 3 respectively:

[0066] 1) For sensor 1: Based on the prior track information and the observation point cloud synchronized by sensor 1 at time t1, track filtering is performed to obtain the posterior track information associated with sensor 1 (hereinafter referred to as posterior track t1); and the target track information corresponding to t_sys (hereinafter referred to as track t1_sys) is estimated at least through the posterior position information and track velocity information in the posterior track t1; and the observation point cloud corresponding to sensor 1 is corrected based on the track offset information of posterior track t1 and track t1_sys to obtain the alignment point cloud 1 corresponding to sensor 1, and the alignment point cloud 1 is updated to the alignment point cloud set;

[0067] 2) For sensor 2: Based on the prior track information and the observation point cloud synchronized by sensor 2 at time t2, track filtering is performed to obtain the posterior track information associated with sensor 2 (hereinafter referred to as posterior track t2); and the target track information corresponding to t_sys (hereinafter referred to as track t2_sys) is estimated at least through the posterior position information and track velocity information in the posterior track t2; and the observation point cloud corresponding to sensor 2 is corrected based on the track offset information of posterior track t2 and track t2_sys to obtain the alignment point cloud 2 corresponding to sensor 2, and the alignment point cloud 2 is updated to the alignment point cloud set;

[0068] 3) For sensor 3: Based on the prior track information and the observation point cloud synchronized by sensor 3 at time t3, track filtering is performed to obtain the posterior track information associated with sensor 3 (hereinafter referred to as posterior track t3); and the target track information corresponding to t_sys (hereinafter referred to as track t3_sys) is estimated at least through the posterior position information and track velocity information in the posterior track t3; and the observation point cloud corresponding to sensor 3 is corrected based on the track offset information of posterior track t3 and track t3_sys to obtain the alignment point cloud 3 corresponding to sensor 3, and the alignment point cloud 3 is updated to the alignment point cloud set;

[0069] Furthermore, the tracks t1_sys, t2_sys, and t3_sys obtained from the track processing operation in the current target detection period T can be fused to obtain the updated track information for the current round of track processing operation (i.e., the current target detection period T). Thus, the updated track information obtained from the track processing operation in the current target detection period T can be used as the prior track information for the next round of track processing operation in the next target detection period T+1.

[0070] Furthermore, after performing the aforementioned processing on sensors 1, 2, and 3 respectively, the moving target can be further processed by the alignment point cloud (containing the global alignment point cloud) obtained from the current trajectory processing operation, including target classification, target bounding box detection, or target segmentation. This target processing can then be used as prior trajectory information for the next trajectory processing operation. For example, if the moving target is a vehicle, 3D bounding box detection can be performed on the vehicle based on the alignment point cloud obtained from this trajectory processing operation, thereby predicting the vehicle volume / 3D bounding box based on the alignment point cloud corresponding to the global observation point cloud. The target trajectory information and 3D bounding box information corrected by this method can be used as prior trajectory information for the next target detection cycle (the next t1&t2&...&t_sys) of the target perception system. Because this method uses the latest trajectory information to correct the observation point clouds of different sensors, it can obtain more original point cloud information compared to existing methods and better eliminates systematic errors, thereby improving the overall tracking performance throughout the entire target tracking lifecycle.

[0071] The following provides a detailed description of the target-aware observation point cloud alignment method provided in the embodiments of this application. Please refer to [link to relevant documentation]. Figure 3 The method specifically includes the following steps:

[0072] Step S300: Perform at least one round of track processing operation on the moving target. One round of track processing operation includes steps S310 and S320:

[0073] Step S310: Obtain the prior track information of the moving target. The prior track information includes at least the prior position information and track speed information of the moving target.

[0074] As one embodiment, the track information in this application embodiment may include, but is not limited to, one or more combinations of information such as the position information of the moving target, track speed information, angular deflection information (such as, but not limited to, the orientation or deflection angle of the moving target), energy reflection, radial velocity, radial distance, horizontal angle, and pitch angle. The energy reflection, radial velocity (in this application embodiment, the radial velocity is the sensor's radial component of the target velocity, and Doppler velocity may be used as a specific example of radial velocity), radial distance, horizontal angle, and pitch angle may be observation parameters of a reflection point on the moving target. Therefore, the prior track information may also include the angular deflection information of the moving target (such as, but not limited to, the orientation or deflection angle of the moving target). The prior track information may also include at least one prior auxiliary information among prior energy reflection, prior radial velocity, prior radial distance, prior horizontal angle, and prior pitch angle. Other contents of the prior track information will be described below.

[0075] Step S320: Based on prior track information and the observation point cloud corresponding to each sensor in at least one sensor, perform point cloud alignment operation on each sensor to obtain the aligned point cloud corresponding to each sensor. The point cloud alignment operation for a single sensor includes the following steps S3201 to S3203:

[0076] Step S3201: Based on the prior track information and the observation point cloud corresponding to the sensor, track filtering is performed to obtain the posterior track information associated with the sensor.

[0077] As one embodiment, in step S3201, point cloud filtering, target association, and track filtering can be performed sequentially based on prior track information and the observation point cloud corresponding to the sensor. Point cloud filtering can be performed, but is not limited to, using point cloud clustering methods, which can be implemented by one or more data clustering methods such as K-means, DBSCAN, and hierarchical clustering. Target association can be implemented, but is not limited to, using one or more methods such as nearest neighbor, global nearest neighbor, and JPDA. Track filtering can be implemented, but is not limited to, using one or more methods such as Kalman filtering, extended Kalman filtering, unscented Kalman filtering, and particle filtering.

[0078] Step S3202: Estimate the target trajectory information corresponding to the target time using at least the posterior position information and trajectory velocity information in the posterior trajectory information.

[0079] Step S3203: Based on the track offset information of the posterior track information and the target track information, the observation point cloud corresponding to the sensor is corrected to obtain the alignment point cloud corresponding to the sensor.

[0080] As one embodiment, after obtaining the alignment point cloud corresponding to each sensor in step S320, step S340 may also be included;

[0081] Step S340: Update the alignment point cloud corresponding to each sensor to the alignment point cloud set, and perform target processing on the moving target based on the alignment point cloud set. The aforementioned target processing includes at least one of target classification, target bounding box detection, and target part segmentation.

[0082] If the moving target is a car, vehicle classification, vehicle bounding box detection, and vehicle part segmentation can be performed on the car in step S340.

[0083] As one embodiment, in step S340, a deep learning network can be trained, and then the alignment cloud can be input into the trained deep learning network, and the processing result of the aforementioned target processing can be output through the trained deep learning network.

[0084] As one embodiment, in step S310, the prior track information of the (i+1)th round in at least one round of track processing is obtained based on the updated track information obtained in the i-th round, that is, the prior track information of the 3rd round is obtained based on the updated track information obtained in the 2nd round, the prior track information of the 2nd round is obtained based on the updated track information obtained in the 1st round, and so on; wherein the prior track information of the 1st round can be obtained based on preset track information (such as the initial position information and initial velocity information of the moving target, and other relevant information, etc.), and i is an integer greater than 0; furthermore, in step S320, during the process of obtaining the alignment point cloud corresponding to each sensor, the target track information corresponding to each sensor can also be fused to obtain the aforementioned updated track information (as can be referred to) Figure 2 (Example in the example); In this embodiment of the application, the method of fusing multiple target track information is not limited in many ways. Those skilled in the art can set it according to actual needs. For example, the mean or mode of the same element in multiple target track information can be used as the element value of the updated track information. The aforementioned elements can include, but are not limited to, one or more track information such as the position information of the moving target (such as prior position information, target position information, etc.), track speed information (such as moving speed, acceleration, etc.), angle deflection information, energy reflection (such as prior energy reflection), radial velocity (such as prior radial velocity), radial distance (such as prior radial distance), horizontal angle (such as prior horizontal angle), pitch angle (such as prior pitch angle), etc.

[0085] The following provides further explanation of steps S3201 to S3203:

[0086] As one embodiment, in step S3203, the observation point cloud corresponding to each sensor can be corrected according to, but is not limited to, the following formula (1) to obtain the alignment point cloud:

[0087] PD t_sys =PD t1 +(PH t_sys -PH t1 )Formula (1)

[0088] In formula (1), PD t_sys For the position of the alignment point cloud corresponding to a sensor, PD t1 The location of the point cloud observed by the sensor, (PH) t_sys -PH t1 ) represents track offset information, PH t1 For retrospective track information, PH t_sys For target trajectory information.

[0089] As one embodiment, the aforementioned trajectory velocity information may include the moving target's speed, or the aforementioned trajectory velocity information may include both the moving target's speed and acceleration, and the observation point cloud corresponding to the sensor is collected for the moving target at the observation time corresponding to that sensor; please refer to Figure 4 Therefore, in step S3202, but not limited to the following steps S410 to S440, for a single sensor, the target trajectory information corresponding to the target time can be estimated using posterior trajectory information:

[0090] Step S410: Determine the observation time difference between the observation time corresponding to the sensor and the target time.

[0091] Wherein, the observation time is the moment when the sensor synchronizes and / or acquires the observation point cloud, the target time is the periodic time corresponding to the current trajectory processing operation / target detection period T, and the observation time difference is the difference between the observation time corresponding to the sensor and the target time corresponding to the current target detection period T. In the embodiments of this application, the observation time difference between the observation time and the target time of each sensor in one or more sensors set for the mobile device can be obtained in advance. Since the observation times of different sensors may be partially different or completely different, the observation time difference between the observation time and the target time of each sensor may also be different. Those skilled in the art can set it according to actual needs.

[0092] Step S420: Determine the position offset information based on the observation time difference and the track speed information in the posterior track information.

[0093] Step S430: Determine the target position information based on the posterior position information and position offset information in the posterior track information.

[0094] Step S440: Update the target position information and trajectory speed information to the target trajectory information corresponding to the sensor.

[0095] As one embodiment, the prior position information of the moving target in the aforementioned prior track information can be the two-dimensional position of the moving target in two-dimensional space or the three-dimensional position in three-dimensional space; for example, the prior position information can be represented as two-dimensional coordinates (posx, posy) in a two-dimensional coordinate system, where posy and posy are the horizontal coordinates of the X-axis and the vertical coordinates of the Y-axis of the two-dimensional coordinate system, respectively; the prior position information can also be represented as two-dimensional coordinates (posx, posy, posz) in a three-dimensional coordinate system, where posy, posy, and posy are the coordinates of the X-axis, Y-axis, and Z-axis of the three-dimensional coordinate system, respectively.

[0096] The aforementioned two-dimensional coordinate system may, but is not limited to, be established with the center of the rear axle of the vehicle as the origin, the direction from the center of the rear axle towards the center of the front of the vehicle as the positive x-direction, perpendicular to the x-axis, and the direction extending laterally to the left of the rear axle center (counterclockwise 90 degrees) as the positive y-axis; the aforementioned three-dimensional coordinate system may, but is not limited to, be established with the center of the rear axle of the vehicle as the origin, the direction from the center of the rear axle towards the center of the front of the vehicle as the positive x-direction, perpendicular to the x-axis, the direction extending laterally to the left of the rear axle center (counterclockwise 90 degrees) as the positive y-axis, and the positive z-axis direction as the upward direction.

[0097] As one embodiment, when the track speed information includes the moving speed of the moving target, the moving speed can be represented, but is not limited to, in vector form or [speed scalar value, speed direction information]; for example, the speed information can be expressed as a two-dimensional vector or a three-dimensional vector, and the two-dimensional vector can be, but is not limited to, the speed vector of the moving speed originating from the center of mass or rotation center of the target and along the x and y axes.

[0098] For ease of understanding, a specific example of target trajectory information estimation in steps S410 to S440 is given here. In this example, the prior trajectory information = {(posx1, posy1), (velx1, vely1)}, where (posx1, posy1) represents the two-dimensional coordinates of the prior position information of the moving target. In this example, posx1 and posy1 are 0.3 and 0.4, respectively. (velx1, vely1) represents the two-dimensional coordinates of the moving target's velocity. In this example, velx1 and vely1 are 0.2 m / s and 0 m / s, respectively. The observation time difference is t0, which is 0.1 s. Therefore, the position offset information can be determined as follows: (velx1*t0, vely2*t0) is (0.2*0.1, 0*0.1). Further, the sum of the posterior position information and the position offset information is taken as the target position information, then the target position information (posx2, posy2) is (posx1+velx1*t0, posy1+vely2*t0), and the calculated target position information is (0.3+0.2*0.1, 0.4+0*0.1), which is (0.32, 0.4). Further, {(posx2, posy2), (velx1, vely1)} is updated to the target track information {(0.32, 0.4), (0.2, 0)}.

[0099] As one embodiment, when the track speed information includes both the moving speed and acceleration of the moving target, the form of the acceleration can be, but is not limited to, the form of the moving speed described above, and will not be repeated here; and when the track speed information includes both the moving speed and acceleration of the moving target, in step S420, the position offset information can be determined by referring to the similar method described above, based on the observation time difference and the moving speed combined with the acceleration, and will not be repeated here.

[0100] As one embodiment, the prior track information also includes the angle deflection information of the moving target, which may be, but is not limited to, yaw angle, etc.; then in the aforementioned steps S420 and S430, the position offset information can be determined based on the observation time difference corresponding to the sensor, the track speed information and the angle deflection information in the posterior track information; and in step S440, the target track information is determined based on the target position information, track speed information and angle deflection information.

[0101] As one embodiment, the prior trajectory information also includes at least one of the following prior auxiliary information: prior energy reflection power1, prior radial velocity d_vel1, prior radial distance range1, prior horizontal angle azimuth1, and prior pitch angle elevation1 of the moving target. The prior radial distance range1 represents the distance of the moving target relative to the corresponding sensor, the prior horizontal angle azimuth1 represents the horizontal angle of the moving target relative to the corresponding sensor, and the prior pitch angle elevation1 represents the pitch angle of the moving target relative to the corresponding sensor. Please continue to see... Figure 3 Furthermore, the process of step S3203 may also include step S3204.

[0102] Step S3204: Estimate the posterior auxiliary information corresponding to each prior auxiliary information, and update the estimated posterior auxiliary information into the target track information.

[0103] Furthermore, in step S3204, the estimation methods for the posterior auxiliary information corresponding to the above five types of prior auxiliary information estimation refer to the following methods: posterior estimation methods A1 to A5.

[0104] Posterior estimation method A1: for prior energy reflection

[0105] The prior energy reflection information is determined as the posterior energy reflection information, and the posterior energy information is reflected and updated in the target track information, that is, power2 = power1, and power1 is the prior energy reflection information.

[0106] Posterior estimation method A2: for prior pitch angle

[0107] The prior pitch angle is determined as the posterior pitch angle, and the posterior pitch angle is updated in the target trajectory information, that is, elevation2 = elevation1, where elevation1 is the prior pitch angle and elevation2 is the posterior pitch angle.

[0108] Posterior estimation method A3: for prior horizontal angles

[0109] The posterior horizontal angle is determined based on the target position information in the target track information, and then updated to the target track information.

[0110] As one example, please refer to Figure 5 When the target position information is represented as two-dimensional coordinates (posx2, posy2), the posterior horizontal angle can be determined according to the target position information (posx2, posy2) in the target track information, but not limited to the following formula (2);

[0111] azimuth2=atan2(posy2,posx2) formula (2)

[0112] In formula (2), azimuth2 is the posterior horizontal angle, and posx2 and posy2 are the coordinates of the target position information on the X-axis and Y-axis in the two-dimensional coordinate system, respectively.

[0113] Posterior estimation method A3: for prior radial velocity

[0114] The posterior radial velocity is determined based on the posterior horizontal angle and the track velocity information in the target track information, and the posterior radial velocity is updated in the target track information.

[0115] As an example, the velocity information in the posterior track information is represented as a two-dimensional vector (velx2, vely2), where velx2 and vely2 represent the components of the moving velocity in the posterior track information in the X-axis and Y-axis directions of the two-dimensional coordinate system, respectively. Then, the posterior radial velocity can be estimated further according to, but not limited to, the following formula (3).

[0116] Formula (3) = d_vel2 = cos(azimuth2) × velx2 + sin(azimuth2) × vely2

[0117] In formula (3), azimuth2 is the posterior horizontal angle, d_vel2 is the posterior radial velocity; velx2 and vely2 are the velocity components of the target trajectory information in the X-axis and Y-axis of the two-dimensional coordinate system, respectively.

[0118] Posterior estimation method A5: for prior radial distance

[0119] The posterior radial distance is determined based on the target position information in the target track information, and the posterior radial distance is updated in the target track information.

[0120] As one example, please refer to Figure 5 When the target location information is represented as two-dimensional coordinates (posx2, posy2), the posterior radial distance can be determined according to, but is not limited to, the following formula (4);

[0121] range2=sqrt(posx2×posx2+posy2×posy2) formula (4)

[0122] In formula (4), range2 is the posterior radial distance, and posx2 and posy2 are the X-axis and Y-axis coordinates of the target position information in the two-dimensional coordinate system, respectively.

[0123] Please refer to Figure 6Based on the same inventive concept, this application also provides a target-aware observation point cloud alignment device 600, characterized in that the device includes a priori information acquisition unit 610 and a point cloud alignment unit 620, wherein the aforementioned point cloud alignment unit 620 includes a track filtering unit 6201, a track estimation unit 6202, and an offset alignment unit 6203; wherein:

[0124] The aforementioned prior information acquisition unit 610 is used to obtain prior trajectory information of the moving target in each round of trajectory processing operation on the moving target. The aforementioned prior trajectory information includes at least the prior position information and trajectory speed information of the moving target.

[0125] The aforementioned point cloud alignment unit 620 is used to perform point cloud alignment operations on each sensor based on the aforementioned prior track information and the observation point cloud corresponding to each sensor in each round of track processing operation on the moving target, so as to obtain the aligned point cloud corresponding to each sensor.

[0126] In the aforementioned point cloud alignment operation: the aforementioned track filtering unit 6201 performs track filtering based on the aforementioned prior track information and the observation point cloud corresponding to the aforementioned sensor to obtain the posterior track information associated with the aforementioned sensor; the aforementioned track estimation unit 6202 is used to estimate the target track information corresponding to the target time based at least on the posterior position information and track velocity information in the aforementioned posterior track information; and the aforementioned offset alignment unit 6203 is used to correct the observation point cloud corresponding to the aforementioned sensor based on the track offset information of the aforementioned posterior track information and the aforementioned target track information to obtain the alignment point cloud corresponding to the aforementioned sensor.

[0127] Since the observation point cloud alignment device 600 is the same as the one used in the embodiment of this application... Figure 2 and Figure 3 The illustrated method is an execution device, and the principle of the physical device in solving the problem is similar to that of the method. Therefore, the implementation of the observation point cloud alignment device 600 can be referred to the implementation of the method, and the repeated parts will not be described again.

[0128] Please refer to Figure 7 Based on the same inventive concept, this application also provides a target-aware observation point cloud alignment device 700, including a memory 710 and a processor 720. The memory 710 stores computer instructions, and the processor 720 executes the computer instructions to achieve the aforementioned... Figure 3 Any of the methods shown in the illustration; since the device is the physical device of the observation point cloud alignment device 600 in the embodiments of this application, that is, the physical device that performs the method provided in this application, and the principle of the physical device in solving the problem is similar to that of the method, the implementation of the physical device can refer to the implementation of the method, and the repeated parts will not be described again.

[0129] Based on the same inventive concept, this disclosure provides a computer storage medium, which includes computer program code. When the computer program code is run on a computer, it causes the computer to perform the actions described above. Figure 3 The following diagram illustrates any one of the methods. Since the principle by which the computer storage medium solves the problem is similar to the aforementioned methods, the implementation of the computer storage medium can be found in the implementation of the methods; repeated details will not be elaborated further.

[0130] In specific implementation, computer storage media can include: Universal Serial Bus Flash Drive (USB), portable hard drive, Read-Only Memory (ROM), Random Access Memory (RAM), magnetic disk or optical disk, and other storage media that can store program code.

[0131] Based on the same inventive concept, this disclosure also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to perform the aforementioned actions. Figure 3 This illustrates any one of the methods. Because the problem-solving principles of the aforementioned computer program products are similar to those described above... Figure 3 The illustrations show that any of the methods are similar, therefore the implementation of the above computer program product can be found in the implementation of the method, and repeated parts will not be described again.

[0132] Computer program products may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0133] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0134] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A target-aware-based method for aligning observation point clouds, characterized in that, include: Perform at least one round of trajectory processing operations on the moving target, the trajectory processing operations including: Obtain prior trajectory information of a moving target, wherein the prior trajectory information includes at least the prior position information and trajectory speed information of the moving target; Based on the prior track information and the observation point cloud corresponding to each sensor in at least one sensor, a point cloud alignment operation is performed on each sensor to obtain an aligned point cloud corresponding to each sensor. The point cloud alignment operation includes: Based on the prior track information and the observation point cloud corresponding to the sensor, track filtering is performed to obtain the posterior track information associated with the sensor; At least using the posterior position and velocity information in the posterior track information, the target track information corresponding to the target time can be estimated; and Based on the track offset information of the posterior track information and the target track information, the observation point cloud corresponding to the sensor is corrected to obtain the alignment point cloud corresponding to the sensor.

2. The method as described in claim 1, characterized in that, The observation point cloud corresponding to the sensor is corrected according to the following formula (1) to obtain the aligned point cloud: Formula (1) In the formula (1), The position of the aligned point cloud, The location of the observed point cloud, This refers to the track offset information. This refers to the subsequent trajectory information. This refers to the target trajectory information.

3. The method as described in claim 1 or 2, characterized in that, The trajectory speed information includes the moving speed of the moving target, or the trajectory speed information includes the moving speed and acceleration of the moving target, and the observation point cloud corresponding to the sensor is collected for the moving target at the observation time corresponding to the sensor; The step of estimating the target trajectory information corresponding to the target time using at least the posterior position information and trajectory velocity information in the posterior trajectory information includes: The position offset information is determined based on the observation time difference and the trajectory velocity information in the posterior trajectory information, wherein the observation time difference is the difference between the observation time corresponding to the sensor and the target time; The target position information is determined based on the posterior position information in the posterior track information and the position offset information; The target position information and the trajectory speed information are updated into the target trajectory information.

4. The method as described in claim 3, characterized in that, The prior trajectory information also includes the angle deflection information of the moving target; The step of determining the position offset information based on the observation time difference and the track velocity information in the posterior track information includes: determining the position offset information based on the observation time difference and the track velocity information in the posterior track information and the angle deflection information; Determining the target trajectory information based on the target position information and the trajectory speed information includes: determining the target trajectory information based on the target position information, the trajectory speed information, and the angle deflection information.

5. The method as described in claim 1, characterized in that, The prior trajectory information also includes at least one of the prior auxiliary information of the moving target, including prior energy reflection, prior radial velocity, prior radial distance, prior horizontal angle, and prior pitch angle. The prior radial velocity is the sensor radial component of the target velocity. The prior radial distance represents the distance of the moving target relative to the corresponding sensor. The prior horizontal angle represents the horizontal angle of the moving target relative to the corresponding sensor. The prior pitch angle represents the pitch angle of the moving target relative to the corresponding sensor. When correcting the observation point cloud corresponding to the sensor based on the track offset information of the posterior track information and the target track information, the method further includes: estimating the posterior auxiliary information corresponding to each prior auxiliary information, and updating the estimated posterior auxiliary information into the target track information.

6. The method as described in claim 5, characterized in that, The process of estimating the posterior auxiliary information corresponding to each prior auxiliary information and updating the estimated posterior auxiliary information to the target trajectory information includes: When the prior energy reflection information includes the prior energy reflection information, the prior energy reflection information is determined as the posterior energy reflection information, and the posterior energy reflection information is updated in the target trajectory information. When the prior trajectory information includes a prior pitch angle, the prior pitch angle is determined as a subsequent pitch angle, and the subsequent pitch angle is updated in the target trajectory information; When the prior auxiliary information includes a prior horizontal angle, the posterior horizontal angle is determined based on the target position information in the target trajectory information, and the posterior horizontal angle is updated in the target trajectory information. When the prior auxiliary information includes the prior radial velocity, the posterior radial velocity is determined based on the posterior horizontal angle and the trajectory velocity information in the target trajectory information, and the posterior radial velocity is updated in the target trajectory information. When the prior auxiliary information includes a prior radial distance, the posterior radial distance is determined based on the target position information in the target track information, and the posterior radial distance is updated in the target track information.

7. The method of claim 6, characterized in that, The trajectory speed information includes the moving speed of the moving target; The posterior horizontal angle is determined according to the target position information in the target trajectory information according to the following formula (2); and / or According to the following formula (3), the posterior radial velocity is determined based on the posterior horizontal angle and the trajectory velocity information in the target trajectory information; and / or, The posterior radial distance is determined according to the target position information in the target trajectory information using the following formula (4); where: Formula (2) Formula (3) Formula (4) In formulas (2) to (4), Let be the posterior horizontal angle. The posterior radial distance is... The posterior radial velocity; and These are the X-axis and Y-axis coordinates of the target location information in a two-dimensional coordinate system, respectively. and These are the velocity components of the target trajectory information along the X and Y axes in the two-dimensional coordinate system.

8. The method as described in claim 1, characterized in that, After performing point cloud alignment operations on each sensor based on the prior track information and the observation point clouds corresponding to each sensor in at least one sensor to obtain the aligned point cloud corresponding to each sensor, the process further includes: The moving target is processed based on the alignment point cloud corresponding to each sensor. The target processing includes at least one of target classification, target bounding box detection, and target part segmentation.

9. The method as described in claim 1, characterized in that, In the at least one round of track processing operation, the prior track information of the (i+1)th round is obtained based on the updated track information obtained in the ith round, and the prior track information of the first round is obtained based on the preset track information, where i is an integer greater than 0; When performing point cloud alignment operations on each sensor based on the prior track information and the observation point cloud corresponding to each sensor in at least one sensor to obtain the aligned point cloud corresponding to each sensor, the method further includes: The target trajectory information corresponding to each sensor is fused to obtain the updated trajectory information.

10. A target-sensing-based observation point cloud alignment device, characterized in that, The device includes a priori information acquisition unit and a point cloud alignment unit, wherein the point cloud alignment unit includes a trajectory filtering unit, a trajectory estimation unit, and an offset alignment unit; wherein: The prior information acquisition unit is used to obtain prior trajectory information of the moving target in each round of trajectory processing operation on the moving target. The prior trajectory information includes at least the prior position information and trajectory speed information of the moving target. The point cloud alignment unit is used to perform point cloud alignment operation on each sensor based on the prior track information and the observation point cloud corresponding to each sensor in each round of track processing operation on the moving target, so as to obtain the aligned point cloud corresponding to each sensor. In the point cloud alignment operation: the track filtering unit performs track filtering based on the prior track information and the observation point cloud corresponding to the sensor to obtain the posterior track information associated with the sensor; the track estimation unit estimates the target track information corresponding to the target time based at least on the posterior position information and track velocity information in the posterior track information; and the offset alignment unit corrects the observation point cloud corresponding to the sensor based on the track offset information of the posterior track information and the target track information to obtain the alignment point cloud corresponding to the sensor.

11. A anechoic chamber radar calibration device based on lidar point clouds, characterized in that, It includes a memory and a processor, the memory being used to store computer instructions, and the processor being used to execute the computer instructions to implement the method as claimed in any one of claims 1-9.

12. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which includes program instructions that, when executed by the computer, cause the computer to perform the method as described in any one of claims 1-9.

Images