Cross-laser projection method and device for 3D target, electronic equipment and storage medium

Abstract

The present application relates to the field of automatic driving perception, in particular to a 3D target cross-laser projection method and device, electronic equipment and storage medium. The present application obtains associated point cloud data of target point cloud data from a point cloud data sequence, and extracts a 3D target and corresponding position data based thereon. Further based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is labeled to the target point cloud data, so as to label the emission ray of the point cloud and the position data of the 3D target, so that in the case that the point clouds of the two are not synchronized in time, the 3D target is labeled to the point cloud data of the first laser radar. Thus, the problem that the models of the mass production car radar and the collection car radar are different, resulting in that the point clouds generated by the two are not synchronized in time, the output point cloud of the collection car radar cannot be aligned with the output point cloud of the mass production car radar in time, and cannot be directly used as the input of the algorithm is solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving perception, and in particular to a method, device, electronic device and storage medium for cross-laser projection of a 3D target. Background Art

[0002] In the field of autonomous driving, perception is crucial. BEV perception, in particular, has been a hot development area in recent years, particularly in 3D detection algorithms. These algorithms convert single- or multimodal information, such as images, into BEV features for perception and output 3D detection information of objects. Implementing 3D detection algorithms requires 3D ground truth data for training. The current industry practice involves installing a laser scanner (lidar) on the roof of the vehicle to capture a full 360-degree field of view, then using the point cloud for 3D ground truth annotation. However, the different models of radars used in production vehicles and those in the acquisition vehicle result in temporal asynchrony between the point clouds they generate. The ground truth output of the acquisition vehicle's radar cannot be aligned with the point cloud output by the production vehicle's radar, making it unsuitable for direct algorithm input. While separate 3D annotation of the point cloud from the production vehicle's radar can address this issue, it incurs additional costs. Summary of the Invention

[0003] In view of this, the embodiments of the present invention provide a cross-laser projection method, device, electronic device and storage medium for 3D targets to solve the problem that the models of mass-produced vehicle radar and the collection vehicle radar are different, resulting in the point clouds generated by the two being temporally asynchronous, and the true value of the point cloud output by the collection vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as input to the algorithm.

[0004] In a first aspect, an embodiment of the present invention provides a method for cross-laser projection of a 3D target, the method comprising:

[0005] Obtain target point cloud data obtained by a first laser radar deployed on the vehicle to detect the current environment, and obtain a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar used to scan the field of view of the vehicle's current direction of travel, and the second laser radar is a laser radar used to scan the entire field of view around the vehicle;

[0006] Acquire associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D object based on the associated point cloud data, and acquire position data of the 3D object in the associated point cloud data;

[0007] Based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

[0008] Furthermore, obtaining the associated point cloud data of the target point cloud data from the point cloud data sequence includes:

[0009] Obtaining the timestamp corresponding to the target point cloud data;

[0010] Acquire a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from the point cloud data sequence;

[0011] Acquire first point cloud data of a frame closest to the timestamp from the first point cloud dataset, and acquire second point cloud data of a frame closest to the timestamp from the second point cloud dataset;

[0012] The first point cloud data and the second point cloud data are used as the associated point cloud data.

[0013] Furthermore, extracting a 3D target from the associated point cloud data and obtaining position data of the 3D target in the associated point cloud data includes:

[0014] Acquire an associated same 3D object based on the first point cloud data and the second point cloud data;

[0015] Acquire a first center point of the 3D object in the first point cloud data and a second center point of the 3D object in the second point cloud data;

[0016] The first center point and the second center point are used as position data of the 3D object in the associated point cloud data.

[0017] Furthermore, the step of labeling the 3D target to the target point cloud data based on the emitted ray corresponding to each point cloud in the target point cloud data and the position data includes:

[0018] Acquire, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray that intersects the line connecting the first center point and the second center point;

[0019] Determine an intersection point between the target emitted ray and the connecting line and a time deviation corresponding to the intersection point;

[0020] The intersection point with the smallest time deviation is marked as the center point of the 3D target in the target point cloud data.

[0021] Furthermore, obtaining, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with the line connecting the first center point and the second center point, includes:

[0022] Converting the point cloud contained in the target point cloud data to the vehicle coordinate system of the vehicle to obtain a transformation matrix of the vehicle between the start time and the end time of scanning the target point cloud data, wherein the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle;

[0023] Using the transformation matrix, the first center point, the second center point, and all point clouds included in the target point cloud data are transformed into the vehicle coordinate system corresponding to the start time of the scan;

[0024] Acquire an emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the start time of the scan;

[0025] It is detected whether the emission ray intersects with the line between the first center point and the second center point, and the intersecting emission ray is used as the target emission ray.

[0026] Furthermore, determining the intersection point between the target emitted ray and the connecting line and the time deviation corresponding to the intersection point includes:

[0027] Converting the first center point and the second center point to the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point;

[0028] Calculate a first emission time of the ray emitted from the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculate a second emission time of the ray emitted from the second center point based on the second coordinates and the acquisition parameters of the second laser radar;

[0029] Calculate the time interpolation of the intersection point on the line and the emission time corresponding to the intersection point using the first emission time and the second emission time;

[0030] A time offset is calculated based on the time interpolation and the emission instant of the intersection point.

[0031] Furthermore, the calculating of the first emission time of the ray emitted from the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and the calculating of the second emission time of the ray emitted from the second center point based on the second coordinates and the acquisition parameters of the second laser radar, include:

[0032] Extracting an initial scanning moment and a scanning period from acquisition parameters of the second laser radar;

[0033] Determining a first angle between the 3D object and the rear of the vehicle using the first coordinates, and determining a second angle between the 3D object and the rear of the vehicle using the second coordinates;

[0034] The first emission time is calculated using the first angle, the initial scanning time, and the scanning period, and the second emission time is calculated using the second angle, the initial scanning time, and the scanning period.

[0035] Furthermore, the calculating the first emission time by using the first angle, the initial scanning time, and the scanning period, and the calculating the second emission time by using the second angle, the initial scanning time, and the scanning period, include:

[0036] Calculating a first ratio between the first angle and a preset value, calculating a first product between the first ratio and the scanning period, and summing the first product with the initial scanning time to obtain the first emission time;

[0037] A second ratio between the second angle and a preset value is calculated, a second product between the second ratio and the scanning period is calculated, and the second emission time is obtained by summing the second product and the initial scanning time.

[0038] Furthermore, the calculating of the time interpolation of the intersection point on the line using the first emission time and the second emission time includes:

[0039] Obtaining a first distance between the first center point and the intersection point, and calculating a time difference between the second emission moment and the first emission moment;

[0040] determining a first proportional relationship between the target time and the time difference based on a ratio between the first distance and the length of the connecting line, and calculating the target time based on the proportional relationship and the time difference;

[0041] The time interpolation is calculated using the target time and the first emission time.

[0042] Furthermore, marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes:

[0043] Obtaining the heading angle corresponding to the intersection point with the smallest time deviation;

[0044] Obtaining the coordinates and size parameters of the intersection point with the smallest time deviation;

[0045] The center point of the 3D object is marked in the target point cloud data using the coordinates, the size parameters, and the heading angle.

[0046] Furthermore, obtaining the heading angle corresponding to the intersection point with the smallest time deviation includes:

[0047] Obtaining a first distance between the first center point and the intersection point, and obtaining a first heading corresponding to the first center point and a second heading corresponding to the second center point;

[0048] calculating a heading difference between the second heading and the first heading;

[0049] determining a proportional relationship between the target heading and the heading difference based on a ratio between a first distance and a length of the connecting line, and calculating the target heading based on the proportional relationship and the heading difference, wherein the first distance is the distance between the first center point and the intersection point;

[0050] The heading angle is calculated using the target heading and the first heading.

[0051] In a second aspect, an embodiment of the present invention provides a 3D target cross-laser projection device, characterized in that the device includes:

[0052] an acquisition module, configured to acquire target point cloud data obtained by a first laser radar deployed on a vehicle to detect a current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning a field of view of the vehicle's current direction of travel, and the second laser radar is a laser radar for scanning a full field of view around the vehicle;

[0053] an extraction module, configured to obtain associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D object based on the associated point cloud data, and obtain position data of the 3D object in the associated point cloud data;

[0054] A labeling module is used to label the 3D target to the target point cloud data based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

[0055] In a third aspect, an embodiment of the present invention provides a computer device comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, computer instructions being stored in the memory, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0057] This application obtains the associated point cloud data of the target point cloud data from the point cloud data sequence, and extracts the 3D target and its corresponding position data based on this. Further, based on the emission ray corresponding to each point cloud in the target point cloud data and the acquired position data, the 3D target is annotated to the target point cloud data, so that the emission ray of the point cloud and the position data of the 3D target are used for annotation, so that when the point clouds of the two are not synchronized in time, the 3D target that is compatible with the point cloud data of the first laser radar can be determined based on the point cloud data output by the second laser radar. This solves the problem that the point clouds generated by the production vehicle radar and the acquisition vehicle radar are not synchronized in time due to the different models of the production vehicle radar, and the point cloud output by the acquisition vehicle radar cannot be aligned in time with the point cloud output by the production vehicle radar, and cannot be directly used as the input of the algorithm.

[0058] This application obtains the associated point cloud data of the target point cloud data from the point cloud data sequence, and extracts the 3D target and its corresponding position data based on this. Further, based on the emission ray and position data corresponding to each point cloud in the target point cloud data, the 3D target is annotated to the target point cloud data, thereby using the emission ray of the point cloud and the position data of the 3D target for annotation, so that when the point clouds of the two are not synchronized in time, the 3D target is annotated to the point cloud data of the first lidar. This solves the problem that the production vehicle radar and the collection vehicle radar are of different models, resulting in the point clouds generated by the two being out of sync in time, and the point cloud output by the collection vehicle radar cannot be aligned in time with the point cloud output by the production vehicle radar, and cannot be directly used as input to the algorithm.

[0059] This application converts the point cloud contained in the target point cloud data into a self-vehicle coordinate system with the center point of the vehicle's rear axle as the origin, and obtains the corresponding transformation matrix between the start and end times of the scan. This allows point cloud data from different times and sources to be unified into a standard coordinate system, facilitating subsequent overall analysis and processing and eliminating the confusion that may be caused by coordinate system differences. Secondly, the first center point, the second center point, and all point clouds are transformed into the self-vehicle coordinate system corresponding to the start time of the scan using the transformation matrix, further ensuring that the data is in the same time and space reference, laying the foundation for accurate comparison and association of various data elements. Furthermore, obtaining the emitted ray corresponding to each point cloud in this unified self-vehicle coordinate system can clearly and standardizedly describe the relative positional relationship between the point cloud and the vehicle, providing convenient conditions for subsequent operations such as intersection detection. Finally, the target emitted ray is determined by detecting whether the emitted ray intersects with the line connecting the first and second center points. This allows the emitted ray related to a specific target to be accurately screened from the numerous emitted rays, helping to improve the accuracy and effectiveness of the entire system's target perception and processing.

[0060] This application converts the first center point and the second center point into the vehicle coordinate system and obtains the corresponding first coordinates and second coordinates, which enables data from different sources to be analyzed in a unified coordinate system, laying the foundation for subsequent accurate calculations and associations, and effectively avoiding confusion and errors caused by inconsistent coordinate systems. The emission time of the emission rays corresponding to the first and second center points is calculated based on the above coordinates and the acquisition parameters of the second laser radar, specifically by extracting the initial scanning time and scanning period, and combining the determined first and second angles. This calculation method can accurately restore the emission time information of the emission rays corresponding to each center point based on the position of the target in the vehicle coordinate system and the scanning characteristics of the radar, so that the time context of the data generation can be more clearly grasped, which helps to deeply understand the status of the data at different times.

[0061] This application further uses the first emission moment and the second emission moment to calculate the time interpolation of the intersection point on the connecting line and its emission moment, and calculates the time deviation based on this. In the process of calculating the time interpolation, the distance between the first center point and the intersection point is first obtained and the time difference is calculated. Then, the proportional relationship between the target time and the time difference is determined by the distance ratio to calculate the target time, and finally the time interpolation is obtained. This method can accurately quantify the position characteristics of the intersection point in the time dimension, and the time deviation calculation can clearly compare the time differences between different points. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0063] Figure 1 is a schematic diagram of a process of cross-laser projection of a 3D object according to some embodiments of the present invention;

[0064] Figure 2 is a schematic diagram of a line connecting an emission ray and a center point according to some embodiments of the present invention;

[0065] Figure 3 is a schematic diagram of heading projection according to some embodiments of the present invention;

[0066] Figure 4 is a structural block diagram of a 3D target cross-laser projection device according to an embodiment of the present invention;

[0067] Figure 5 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0069] According to an embodiment of the present invention, a method, apparatus, electronic device and storage medium for cross-laser projection of a 3D target are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0070] In this embodiment, a cross-laser projection method for a 3D target is provided. Figure 1 is a flow chart of a cross-laser projection method for a 3D target according to an embodiment of the present invention, such as Figure 1 As shown, the process includes the following steps:

[0071] Step S101: obtain target point cloud data obtained by a first laser radar deployed on the vehicle to detect the current environment, and obtain a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar used to scan the field of view of the vehicle's current driving direction, and the second laser radar is a laser radar used to scan the entire field of view around the vehicle.

[0072] In the embodiment of the present application, the first laser radar performs a field of view scan in the current direction of travel of the vehicle, which can provide accurate environmental information in front of the vehicle and ensure the accuracy of the driving route judgment, while the second laser radar is responsible for performing a 360° full-field field scan around the vehicle, which can fully grasp the surrounding situation and eliminate visual blind spots. The two work together to build a comprehensive and detailed vehicle environment perception system. The second laser radar is mainly used to mark the true value of the bev 3D target. Specifically, the second laser radar scans the surrounding environment, and then projects the pre-marked target true value onto the point cloud data of the first laser radar. The point cloud collected by the first laser radar is called Q, and the point cloud collected by the second laser radar is called point cloud P.

[0073] The point cloud information covers the emission time t of the emitted ray and the spatial coordinates x, y, z of the corresponding reflection point. And the 3D true value annotation has been completed based on the point cloud P. The annotation information includes the center point position, length, width, height and heading angle of the object. Specifically, in the operation of obtaining relevant data, first, a certain frame Q is clearly known, and its point cloud scanning start and end times are t1 and t2 respectively. Then, the two frames of point cloud P before and after Q are taken, namely point cloud P1 and point cloud P2. Since the scanning cycle of point cloud P is longer than that of Q, and the upper and lower frames of point cloud P are continuous in time, the scanning time of the two frames of point cloud data detected by the second laser radar, point cloud P1 and point cloud P2, can cover the time range corresponding to the target point cloud data detected by the first laser radar Q in the time period from t1 to t2, thereby obtaining the target point cloud data currently detected by the first laser radar, and the two adjacent frames of point cloud data detected by the second laser radar.

[0074] Step S102 : acquiring associated point cloud data of the target point cloud data from the point cloud data sequence, extracting a 3D target based on the associated point cloud data, and acquiring position data of the 3D target in the associated point cloud data.

[0075] In an embodiment of the present application, obtaining associated point cloud data of target point cloud data from a point cloud data sequence includes: obtaining a timestamp corresponding to the target point cloud data; obtaining a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from the point cloud data sequence; obtaining first point cloud data of the frame closest to the timestamp from the first point cloud data set; and obtaining second point cloud data of the frame closest to the timestamp from the second point cloud data set. The first point cloud data and the second point cloud data are used as associated point cloud data.

[0076] Specifically, the first step is to obtain the timestamp corresponding to the target point cloud data. This timestamp is the key identifier for accurately locating the target point cloud data at a specific moment on the timeline. Next, the point cloud data sequence detected by the second lidar is obtained. This sequence contains multiple frames of continuous point cloud data. These point cloud data are the results of the second lidar scan at different times, showing a continuous state arranged in chronological order.

[0077] After obtaining this complete point cloud data sequence, further filtering is performed based on the timestamp of the previously acquired target point cloud data. From this multi-frame continuous point cloud data sequence, all point cloud data before the timestamp are found and classified as the first point cloud data set; at the same time, all point cloud data after the timestamp are found and formed into the second point cloud data set.

[0078] Finally, within the first point cloud dataset, the time intervals between each point cloud data point and the timestamp are compared to select the frame closest to the timestamp and identify it as the first point cloud data point. Similarly, within the second point cloud dataset, the time intervals between each point cloud data point and the timestamp are compared to identify the frame closest to the timestamp and identify it as the second point cloud data point. Through this series of operations, it is possible to accurately obtain the two frames of specific point cloud data closely associated with the target point cloud data point from the large amount of point cloud data detected by the second lidar based on the timestamp of the target point cloud data point.

[0079] In an embodiment of the present application, extracting a 3D object from associated point cloud data and obtaining position data of the 3D object in the associated point cloud data includes: obtaining the associated 3D object based on first point cloud data and second point cloud data; obtaining a first center point of the 3D object in the first point cloud data and a second center point of the 3D object in the second point cloud data; and using the first center point and the second center point as the position data of the 3D object in the associated point cloud data.

[0080] Specifically, first make sure that the first point cloud data (point cloud P1) and the second point cloud data (point cloud P2) have been acquired. Next, obtain the associated identical 3D target from these two parts of point cloud data. The association here means that the point cloud data in different frames correspond to the same actual target object. After determining this associated identical 3D target, further obtain the relevant information of the target in the first point cloud data, that is, find the center point position of the target in the first point cloud data, and define it as the first center point. Similarly, for the second point cloud data, the center point position of the same 3D target in it must also be obtained, and this center point is called the second center point. Finally, the first center point and the second center point are used as the position data of the 3D target in the associated point cloud data.

[0081] It should be noted that the scanning start and end times of the point cloud data Q of the first laser radar are t1 and t2. The two frames of point cloud data P1 and P2 before and after the point cloud data Q are taken from the point cloud data sequence detected by the second laser radar. Since the scanning cycle of the second laser radar is longer than that of the first laser radar, this means that within the same time interval, the second laser radar takes more time to complete a scan. Moreover, the upper and lower frames of the point cloud data detected by the second laser radar are continuous in time. Assume that the scanning cycle of the second laser radar is T and the scanning cycle of the first laser radar is t (T>t). If the first laser radar completes the scanning of a frame of data Q within the time interval t1-t2, then since the scanning cycle of the second laser radar is long and continuous, its two frames of data P1 and P2 will cover this relatively short t1-t2 interval in time.

[0082] Step S103, based on the emission ray and position data corresponding to each point cloud in the target point cloud data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

[0083] In an embodiment of the present application, based on the emitted ray and position data corresponding to each point cloud in the target point cloud data, annotating the 3D target to the target point cloud data includes the following steps A1-A3:

[0084] Step A1: acquiring, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with the line connecting the first center point and the second center point.

[0085] Specifically, obtaining the target emission ray that intersects the line between the first center point and the second center point from the emission ray corresponding to each point cloud in the target point cloud data includes the following steps A101-A104:

[0086] Step A101: Convert the point cloud contained in the target point cloud data to the ego vehicle coordinate system to obtain the transformation matrix of the ego vehicle between the start time and the end time of the scan of the target point cloud data, where the origin of the ego vehicle coordinate system is the center point of the ego vehicle's rear axle.

[0087] Specifically, a vehicle coordinate system is set, with its origin being the center point of the rear axle of the vehicle, as a unified reference frame to describe the position of the point cloud. Next, with the help of odometry, the position and posture information of the vehicle at any time can be obtained. Here we focus on the position and posture information of the vehicle at times t1 and t2. The posture information covers the position and posture of the vehicle in three-dimensional space. Based on the obtained posture information of the vehicle at times t1 and t2, the three-dimensional space coordinate transformation process from time t2 to time t1 is calculated. This process involves the possible position movement (corresponding to the translation operation) and posture change (corresponding to the rotation operation) of the vehicle between these two moments, so the transformation matrix containing the rotation matrix and the translation matrix can be obtained by calculation.

[0088] Specifically, the rotation matrix is ​​determined based on the difference in the vehicle's orientation at times t1 and t2 using relevant mathematical formulas (such as the rotation matrix calculation formula based on Euler angles). The translation matrix is ​​calculated based on the difference in the vehicle's position coordinates at these two times. Finally, the resulting rotation and translation matrices are combined to successfully obtain the transformation matrix of the ego vehicle between the start and end times of the target point cloud data scan, laying the foundation for the subsequent accurate conversion of the point cloud contained in the target point cloud data to the ego vehicle coordinate system.

[0089] In step A102 , the first center point, the second center point, and all point clouds included in the target point cloud data are transformed into the vehicle coordinate system corresponding to the start time of the scan using a transformation matrix.

[0090] Specifically, first, we obtain the transformation matrix from the target point cloud data's scan end time to the scan start time. This transformation matrix includes a rotation matrix and a translation matrix, which describes the changes in the vehicle's position and posture between these two times. For the first center point, assuming its coordinate in the original coordinate system is K1, we need to transform it to the vehicle coordinate system corresponding to the scan start time. This is achieved through matrix multiplication based on the principle of three-dimensional space coordinate transformation. That is, we first multiply the original coordinate vector with the rotation matrix in the transformation matrix to obtain the rotated coordinate vector. Then, we add the vector corresponding to the translation matrix. This gives us the new coordinate K2 of the first center point in the vehicle coordinate system corresponding to the scan start time.

[0091] The same approach applies to the second center point. Assuming the coordinates of the second center point in the original coordinate system are L1, follow the above steps to first multiply it by the rotation matrix to obtain the rotated coordinate vector. Then, add the vector corresponding to the translation matrix to obtain the new coordinates of the second center point in the vehicle coordinate system at the start of the scan, L2.

[0092] For all point clouds contained in the target point cloud data, for each of them, for example, a point cloud with a coordinate of Pi in the original coordinate system (where i represents the sequence number of the point cloud), the same transformation method is also used. First, the coordinate vector of the point cloud is multiplied by the rotation matrix to obtain the rotated coordinate vector, and then the vector corresponding to the translation matrix is ​​added. Then, each point cloud is accurately transformed to the vehicle coordinate system corresponding to the start time of the scan, and the corresponding new coordinates Pi' are obtained. Through this operation, the process of uniformly transforming the first center point, the second center point, and all point clouds contained in the target point cloud data to the vehicle coordinate system corresponding to the start time of the scan using the transformation matrix is ​​completed.

[0093] Step A103 : obtaining the emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the start time of the scan.

[0094] Specifically, at this point, the point clouds in the entire target point cloud data are already in the vehicle coordinate system with the center point of the vehicle's rear axle as the coordinate origin. For each point cloud in the target point cloud data, the origin of the vehicle coordinate system (that is, the center point of the vehicle's rear axle) is used as the starting point of the emission ray, and then the specific coordinate position of the point cloud in the vehicle coordinate system corresponding to the start time of the scan is used as the end point of the emission ray. In this way, the emission ray is determined from the starting point to the end point. In this way, this operation is performed for each point cloud in the target point cloud data, thereby obtaining the emission ray corresponding to each point cloud in the target point cloud data.

[0095] It should be noted that the point cloud contained in the target point cloud data is first converted to the vehicle's own-vehicle coordinate system. By obtaining the vehicle's transformation matrix between the start and end times of the target point cloud data scan, the changes in the vehicle's motion state during this scanning phase can be clearly determined. This transformation matrix is ​​then used to transform the first center point, the second center point, and all point clouds to the own-vehicle coordinate system corresponding to the start time of the scan. This step is equivalent to using a unified, relatively initial vehicle state as a reference, eliminating differences in coordinates and other aspects caused by vehicle motion. Then, the emission ray corresponding to the point cloud is obtained in the own-vehicle coordinate system corresponding to this unified starting time, and its intersection with the line connecting the two center points is tested to determine the target emission ray. The entire process is based on the consideration of the vehicle's own motion, adjusting and processing the relevant data. This allows the subsequent determination of the center point of the 3D target in the target point cloud data to eliminate the influence of vehicle motion, thereby compensating for vehicle motion and ensuring the accuracy and rationality of data processing and annotation.

[0096] Step A104 , detecting whether the emission ray intersects with the line connecting the first center point and the second center point, and taking the intersecting emission ray as the target emission ray.

[0097] Specifically, based on the coordinates of the first and second center points in the ego-vehicle coordinate system at the start of the scan, the equation of the line connecting the two center points is determined using a two-point equation or other method. This line represents the line connecting the first and second center points. Then, for each previously acquired ray determined by each point cloud in the target point cloud data (the ray starts at the center point of the ego-vehicle rear axle and ends at the coordinates of the point cloud in the ego-vehicle coordinate system), the equation for the ray is combined with the equation for the connecting line to form a system of equations. This system of equations is solved to determine whether it has a solution. If the system has a solution, it means that the ray intersects the line connecting the first and second center points in space, and the ray is determined as the target ray. If the system has no solution, it means that the ray does not intersect the line and does not meet the criteria for being a target ray. This detection and judgment is repeated for all ray types one by one, identifying those that intersect the line connecting the first and second center points and defining them as target rays.

[0098] This application converts the point cloud contained in the target point cloud data into a self-vehicle coordinate system with the center point of the vehicle's rear axle as the origin, and obtains the corresponding transformation matrix between the start and end times of the scan. This allows point cloud data from different times and sources to be unified into a standard coordinate system, facilitating subsequent overall analysis and processing and eliminating the confusion that may be caused by coordinate system differences. Secondly, the first center point, the second center point, and all point clouds are transformed into the self-vehicle coordinate system corresponding to the start time of the scan using the transformation matrix, further ensuring that the data is in the same time and space reference, laying the foundation for accurate comparison and association of various data elements. Furthermore, obtaining the emitted ray corresponding to each point cloud in this unified self-vehicle coordinate system can clearly and standardizedly describe the relative positional relationship between the point cloud and the vehicle, providing convenient conditions for subsequent operations such as intersection detection. Finally, the target emitted ray is determined by detecting whether the emitted ray intersects with the line connecting the first and second center points. This allows the emitted ray related to a specific target to be accurately screened from the numerous emitted rays, helping to improve the accuracy and effectiveness of the entire system's target perception and processing.

[0099] Step A2: determining the intersection point between the target emitted ray and the connecting line and the time deviation corresponding to the intersection point.

[0100] Specifically, in the embodiment of the present application, determining the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point includes the following steps A201-A204:

[0101] In step A201 , the first center point and the second center point are converted into the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point.

[0102] Step A202, calculate the first emission time of the emission ray corresponding to the first center point based on the first coordinate and the acquisition parameters of the second laser radar, and calculate the second emission time of the emission ray corresponding to the second center point based on the second coordinate and the acquisition parameters of the second laser radar.

[0103] Specifically, calculating a first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculating a second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar, includes the following steps A2021-A2023:

[0104] Step A2021, extract the initial scanning time and scanning period from the acquisition parameters of the second lidar.

[0105] Specifically, within the acquired second LiDAR acquisition parameters, search for the parameter value associated with the start of the scan. This parameter value clearly identifies the moment when the second LiDAR begins a complete scan, also known as the initial scan time. For example, if the initial scan time of point cloud P is Tp1, we need to accurately find the record corresponding to this start scan time point from the entire second LiDAR parameter set and extract it as the initial scan time required in the following steps.

[0106] The scan period is the time it takes for the radar to complete a full 360-degree scan. By analyzing relevant parameters, if the radar's scan frequency is known (e.g., scans per second), the scan period can be calculated as its reciprocal (i.e., scan period = 1 / scan frequency). If the parameters directly specify the time it takes for each 360-degree scan, as in the case where the time T required for a point cloud P to scan 360 degrees is given, then this time value can be directly extracted as the scan period.

[0107] Step A2022, using the first coordinate to determine a first angle between the 3D target and the rear end of the vehicle, and using the second coordinate to determine a second angle between the 3D target and the rear end of the vehicle.

[0108] Specifically, in the vehicle coordinate system with the center point of the rear axle of the vehicle as the coordinate origin, when determining the first angle using the first coordinate, first determine a vector pointing from the center point of the rear axle of the vehicle to the first center point based on the first coordinate corresponding to the first center point of the 3D target in the first point cloud data, and at the same time, determine the direction vector of the rear end of the vehicle. By calculating the dot product and respective module lengths of these two vectors, and then calculating the cosine value based on the formula for the relationship between the vector dot product and the angle, the first angle between the 3D target (represented by the first center point) and the rear end of the vehicle is finally obtained through the inverse trigonometric function. Similarly, when determining the second angle using the second coordinate, determine the vector pointing from the center point of the rear axle of the vehicle to the second center point based on the second coordinate corresponding to the second center point of the 3D target in the second point cloud data. Combined with the direction vector of the rear end of the vehicle, repeat the above steps of calculating the dot product, module length, cosine value and solving through the inverse trigonometric function to obtain the second angle between the 3D target (represented by the second center point) and the rear end of the vehicle.

[0109] Step A2023, calculate the first emission time using the first angle, the initial scanning time and the scanning period, and calculate the second emission time using the second angle, the initial scanning time and the scanning period.

[0110] Specifically, calculating the first emission time using the first included angle, the initial scanning time, and the scanning period, and calculating the second emission time using the second included angle, the initial scanning time, and the scanning period, include:

[0111] (1) Calculating a first ratio between a first angle and a preset value, calculating a first product between the first ratio and a scanning period, and summing the first product with an initial scanning time to obtain a first emission time.

[0112] The calculation formula for the first launch time is as follows: Where T1 is the first emission time, is the first included angle, T is the scanning period, and Tp1 is the initial scanning time.

[0113] (2) Calculating a second ratio between the second angle and the preset value, calculating a second product between the second ratio and the scanning period, and summing the second product with the initial scanning time to obtain a second emission time.

[0114] The calculation formula for the second launch time is as follows: Where T2 is the second emission time, is the first included angle, T is the scanning period, and Tp1 is the initial scanning time.

[0115] Step A203 , using the first emission time and the second emission time, calculates the time interpolation of the intersection point on the connecting line and calculates the emission time corresponding to the intersection point.

[0116] Specifically, the time interpolation of the intersection point on the connecting line is calculated using the first emission time and the second emission time, including: obtaining a first distance (O1Q) between the first center point O1 and the intersection point Q, and calculating the time difference between the second emission time T2 and the first emission time T1; determining a first proportional relationship between the target time Ti and the time difference (T2-T1) based on the ratio (O1Q / O1O2) between the first distance (O1Q) and the connecting line length (O1O2), and calculating the target time based on the proportional relationship and the time difference; and calculating the time interpolation (Tp) using the target time and the first emission time, Tp=T1+Ti.

[0117] Each intersection point has a corresponding emission ray, and the emission time Ta of this emission ray is based on the first lidar's own operating record mechanism. When the first lidar performs a scan, it detects the surrounding environment according to a specific time sequence. Each time it emits a ray to detect a target and receives the reflected signal to form point cloud data, it accurately records the specific time of the emission ray. Therefore, once the intersection point and its corresponding emission ray are determined, the specific time corresponding to the emission of the emission ray can be found from the radar device's detailed operating records. This time is Ta.

[0118] Step A204 : Calculate the time offset based on the time interpolation and the emission time of the intersection point.

[0119] Specifically, the time deviation of the intersection point Q on the point clouds P and Q is calculated. The time deviation ΔT is obtained by calculating the absolute value of the difference between the time interpolation Tp and the emission time Ta of the intersection point.

[0120] This application converts the first center point and the second center point into the vehicle coordinate system and obtains the corresponding first coordinates and second coordinates, which enables data from different sources to be analyzed in a unified coordinate system, laying the foundation for subsequent accurate calculations and associations, and effectively avoiding confusion and errors caused by inconsistent coordinate systems. The emission time of the emission rays corresponding to the first and second center points is calculated based on the above coordinates and the acquisition parameters of the second laser radar, specifically by extracting the initial scanning time and scanning period, and combining the determined first and second angles. This calculation method can accurately restore the emission time information of the emission rays corresponding to each center point based on the position of the target in the vehicle coordinate system and the scanning characteristics of the radar, so that the time context of the data generation can be more clearly grasped, which helps to deeply understand the status of the data at different times.

[0121] This application further uses the first emission moment and the second emission moment to calculate the time interpolation of the intersection point on the connecting line and its emission moment, and calculates the time deviation based on this. In the process of calculating the time interpolation, the distance between the first center point and the intersection point is first obtained and the time difference is calculated. Then, the proportional relationship between the target time and the time difference is determined by the distance ratio to calculate the target time, and finally the time interpolation is obtained. This method can accurately quantify the position characteristics of the intersection point in the time dimension, and the time deviation calculation can clearly compare the time differences between different points.

[0122] Step A203: Mark the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data.

[0123] Specifically, in the embodiment of the present application, marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes the following steps A2031-A2033:

[0124] Step A2031, obtain the heading angle corresponding to the intersection point with the smallest time deviation.

[0125] Specifically, obtaining the heading angle corresponding to the intersection point with the smallest time deviation includes: obtaining a first distance between the first center point and the intersection point, and obtaining a first heading Yaw1 corresponding to the first center point and a second heading Yaw2 corresponding to the second center point; calculating the heading difference (Yaw2-Yaw1) between the second heading Yaw2 and the first heading Yaw1; determining the proportional relationship between the target heading and the heading difference (Yaw2-Yaw1) based on the ratio (O1Q / O1O2) between the first distance (O1Q) and the length of the connecting line (O1O2), and calculating the target heading Yawi based on the proportional relationship and the heading difference, wherein the first distance is the distance between the first center point and the intersection point; and calculating the heading angle Yaw using the target heading Yawi and the first heading.

[0126] Step A2032, obtaining the coordinates and size parameters of the intersection point with the smallest time deviation.

[0127] Specifically, after obtaining the time deviation ΔT of all intersection points, all calculated ΔT values ​​are compared one by one. By comparing the sizes of these values, the smallest value is selected. The intersection point corresponding to this minimum value is recorded as the target intersection point, and it is marked as Qi. The intersection point Qi has specific coordinates, which are determined in the corresponding coordinate system (such as the self-vehicle coordinate system with the center point of the rear axle of the self-vehicle as the coordinate origin set earlier). It accurately describes the specific position of Qi in space, that is, the center point position of the target on the Q point cloud, and its coordinates can be expressed as (x, y, z). At the same time, the size parameters corresponding to Qi, the size parameters here include length, width, and height, and these size parameters are consistent with the relevant sizes corresponding to the point cloud P.

[0128] Step A2033: Use coordinates, size parameters, and heading angles to mark the center point of the 3D target in the target point cloud data.

[0129] Specifically, after obtaining the coordinates, size parameters and known heading angle of the intersection point with the smallest time deviation, the annotation operation is performed in the target point cloud data. First, the obtained coordinate information is marked as the center point position of the 3D true value target in the target point cloud data. This coordinate accurately determines the specific position of the target in three-dimensional space. Then, combined with the size parameters, these size parameters can further clarify the size range of the target in each dimension, such as length, width, height and other information, so that the shape and occupied range of the target in space are clearer. Finally, using the known heading angle, it can describe the angular relationship of the target relative to a reference direction (such as the direction of vehicle travel, etc.). By integrating the three aspects of coordinates, size parameters and heading angle, the point is completely and accurately marked as the center point of the 3D true value target in the target point cloud data, providing key information such as accurate target position and morphology for subsequent related applications such as target recognition and path planning.

[0130] This application first obtains the target point cloud data currently detected by a first laser radar and two adjacent frames of point cloud data from a second laser radar associated with it. The two frames of point cloud data are used to find the associated same 3D target and its first and second center points in the two frames of data. Then, from the emission rays corresponding to each point cloud of the target point cloud data, the target emission ray, the intersection point, and the corresponding time offset that intersect with the line connecting the two center points are determined. The intersection point with the smallest time offset is then marked as the center point of the 3D target in the target point cloud data. In this way, by establishing an association between the point cloud data of different frames and the target point cloud data of the mass-produced vehicle, finding a corresponding relationship, and marking them based on the minimum time offset, it is possible to determine a 3D target annotation that is compatible with the point cloud data of the mass-produced vehicle based on the point cloud data of the acquisition vehicle when the point clouds of the two are out of sync in time. This solves the problem that the point clouds generated by the mass-produced vehicle radar and the acquisition vehicle radar are out of sync in time due to the different models of the two radars, and the point cloud output by the acquisition vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, making them incompatible as direct input to the algorithm.

[0131] The following is a complete target labeling process:

[0132] Step 1: Name the second lidar point cloud P and the first lidar Q. Specify that the point cloud information includes the emission time t of the emitted ray and the spatial coordinates x, y, and z of the corresponding reflection point. Also, confirm that the 3D ground truth annotations have been completed based on point cloud P, including the object's center position, length, width, height, and heading angle.

[0133] Step 2: For a known point cloud Q in a certain frame, determine the start and end times of its point cloud scanning as t1 and t2 respectively. Select the two point clouds P before and after Q, and record them as point cloud P1 and point cloud P2 respectively. Given the scanning cycle characteristics of point cloud P (the scanning cycle is longer than that of Q and the upper and lower frames are continuous in time), ensure that the scanning time of point cloud P1 and point cloud P2 can cover t1 and t2.

[0134] Step 3: Find the same associated target on point cloud P1 and point cloud P2. The target is recorded as P1 on point cloud P1 and P2 on point cloud P2. At the same time, determine the center point of P1 as O1 and the center point of P2 as O2.

[0135] Step 4: Using the center point of the rear axle of the vehicle as the coordinate origin, convert the center point O1 of P1 and the center point O2 of P2 to the vehicle coordinate system.

[0136] Step 5: Based on the initial scanning time Tp1 of the point cloud P and the time T used to scan 360 degrees, when the coordinates of the center point of the object are known, the angle ψ between the point and the rear direction of the vehicle is calculated by the formula (calculation formula: the emission time of the ray corresponding to the center point Then, the emission time T1 of the ray corresponding to O1 and the emission time T2 of the ray corresponding to O2 are obtained.

[0137] In step 6, all point clouds on Q are first converted to the ego-vehicle coordinate system, using the center of the ego-vehicle's rear axle as the coordinate origin. Then, using the ego-vehicle's pose information at times t1 and t2 obtained through odometry, the 3D coordinate transformation from time t2 to time t1 (including the rotation matrix and translation matrix) is calculated. This transformation matrix is ​​then used to uniformly transform O1, O2, and all point clouds on Q to the ego-vehicle coordinate system at the initial moment of the Q scan (the origin of this coordinate system is C). This process compensates for the ego-vehicle motion of the point clouds.

[0138] Step 7: In the three-dimensional space coordinate system, connect points O1 and O2 to form a line O1O2. Since the actual time interval between O1 and O2 is relatively short, O1 and O2 can be regarded as the uniform linear motion trajectory of the target.

[0139] Step 8: Select a point A from the Q point cloud, determine its corresponding emission ray as CA, and record the emission time of the emission ray as Ta.

[0140] Step 9: Determine whether the emitted ray CA intersects with the line O1O2 in space. If so, name the intersection point Q.

[0141] Step 10, reference Figure 2 If there is an intersection point Q, calculate the time interpolation of point Q on O1O2 according to the formula O1Q / O1O2=Ti / T2-T1 (where Tp=T1+Ti).

[0142] Step 11: If there is an intersection point Q, calculate the heading difference of point Q (e.g. Figure 3 shown).

[0143] Step 12: For the intersection point Q, calculate its time deviation on the point clouds P and Q. The calculation formula is ΔT = |Tp-Ta|.

[0144] Step 13: Repeat steps 8 to 12 for all other points on Q. If the emitted ray intersects the line O1O2, calculate the corresponding ΔT.

[0145] Step 14: Compare all calculated ΔT values ​​and find the minimum value. The intersection point Qi corresponding to the minimum value is the center point of the target on the Q point cloud. The heading corresponding to Qi is the Yaw obtained by interpolation. Its length, width, and height are consistent with the point cloud P. From this, the coordinates x, y, z, length, width, height, and heading angle of the center point of the target on Q can be obtained.

[0146] Step 14: Repeat steps 2 to 14 to project all 3D objects marked on point cloud P1 and point cloud P2 onto Q, thereby completing the 3D true value marking of Q.

[0147] In this embodiment, a device for projecting a 3D object across a laser beam is also provided. The device is used to implement the above-mentioned embodiments and preferred implementations, and details that have already been described will not be repeated. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0148] This embodiment provides a 3D target cross-laser projection device, such as Figure 4 Shown, including:

[0149] An acquisition module 401 is configured to acquire target point cloud data obtained by a first laser radar deployed on the vehicle to detect the current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning the field of view of the vehicle's current direction of travel, and the second laser radar is a laser radar for scanning the entire field of view around the vehicle;

[0150] An extraction module 402 is configured to obtain associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D object based on the associated point cloud data, and obtain position data of the 3D object in the associated point cloud data;

[0151] The labeling module 403 is used to label the 3D target to the target point cloud data based on the emission ray and position data corresponding to each point cloud in the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

[0152] In an embodiment of the present application, an extraction module is used to obtain a timestamp corresponding to target point cloud data; obtain a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from a point cloud data sequence; obtain the first point cloud data of the frame closest to the timestamp from the first point cloud data set, and obtain the second point cloud data of the frame closest to the timestamp from the second point cloud data set; and use the first point cloud data and the second point cloud data as associated point cloud data.

[0153] In an embodiment of the present application, an extraction module is used to obtain the same associated 3D target based on first point cloud data and second point cloud data; obtain a first center point of the 3D target in the first point cloud data and a second center point of the 3D target in the second point cloud data; and use the first center point and the second center point as position data of the 3D target in the associated point cloud data.

[0154] In the embodiment of the present application, the marking module includes:

[0155] a processing submodule, configured to obtain, from the emitted rays corresponding to each point cloud in the target point cloud data, a target emitted ray intersecting with the line connecting the first center point and the second center point;

[0156] A determination submodule is used to determine the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point;

[0157] The screening submodule is used to mark the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data.

[0158] In the embodiment of the present application, the processing submodule includes:

[0159] A first conversion unit is configured to convert the point cloud included in the target point cloud data into an ego vehicle coordinate system, and obtain a transformation matrix of the ego vehicle between the start time and the end time of scanning the target point cloud data, wherein the origin of the ego vehicle coordinate system is the center point of the ego vehicle rear axle;

[0160] An execution unit, configured to transform the first center point, the second center point, and all point clouds included in the target point cloud data into a self-vehicle coordinate system corresponding to a scanning start time using a transformation matrix;

[0161] An acquisition unit is used to acquire the emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the start time of scanning;

[0162] The detection unit is used to detect whether the emission ray intersects with the line connecting the first center point and the second center point, and use the intersecting emission ray as the target emission ray.

[0163] In the embodiment of the present application, the submodule is determined, including:

[0164] A second conversion unit is used to convert the first center point and the second center point into the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point;

[0165] A first calculation unit is used to calculate a first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and to calculate a second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar;

[0166] A second calculation unit is used to calculate the time interpolation of the intersection point on the connecting line and calculate the emission time corresponding to the intersection point by using the first emission time and the second emission time;

[0167] The third calculation unit is configured to calculate the time deviation based on the time interpolation and the emission time of the intersection point.

[0168] In an embodiment of the present application, the first calculation unit is used to extract the initial scanning moment and the scanning period from the acquisition parameters of the second laser radar; determine the first angle between the 3D target and the rear end of the vehicle using the first coordinate, and determine the second angle between the 3D target and the rear end of the vehicle using the second coordinate; calculate the first emission moment using the first angle, the initial scanning moment and the scanning period, and calculate the second emission moment using the second angle, the initial scanning moment and the scanning period.

[0169] In an embodiment of the present application, the first calculation unit is used to calculate a first ratio between the first angle and the preset value, calculate a first product between the first ratio and the scanning period, and sum the first product with the initial scanning time to obtain a first emission time; calculate a second ratio between the second angle and the preset value, calculate a second product between the second ratio and the scanning period, and sum the second product with the initial scanning time to obtain a second emission time.

[0170] In an embodiment of the present application, the second calculation unit is used to obtain a first distance between the first center point and the intersection point, and calculate the time difference between the second launch moment and the first launch moment; determine a first proportional relationship between the target time and the time difference based on the ratio between the first distance and the length of the connecting line, and calculate the target time based on the proportional relationship and the time difference; and calculate time interpolation using the target time and the first launch moment.

[0171] In an embodiment of the present application, the screening submodule is used to obtain the heading angle corresponding to the intersection point with the smallest time deviation; obtain the coordinates and size parameters of the intersection point with the smallest time deviation; and use the coordinates, size parameters and heading angle to mark the center point of the 3D target in the target point cloud data.

[0172] In an embodiment of the present application, the screening submodule is used to obtain a first distance between a first center point and an intersection point, and obtain a first heading corresponding to the first center point and a second heading corresponding to the second center point; calculate a heading difference between the second heading and the first heading; determine a proportional relationship between a target heading and the heading difference based on a ratio between the first distance and a length of a connecting line, and calculate the target heading based on the proportional relationship and the heading difference; and calculate a heading angle using the target heading and the first heading.

[0173] See also Figure 5 , Figure 5 is a structural diagram of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components utilize different buses to communicate with each other and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in the memory or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Equally, multiple computer devices can be connected, and each device provides part of the necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0174] The processor 10 may be a central processing unit, a network processor, or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic, or any combination thereof.

[0175] The memory 20 stores instructions that can be executed by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0176] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created based on the use of a computer device for displaying a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0177] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0178] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0179] The embodiment of the present invention also provides a computer-readable storage medium. The above-mentioned method according to the embodiment of the present invention can be implemented in hardware, firmware, or implemented as a computer code that can be recorded in a storage medium, or implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state drive, etc.; further, the storage medium can also include a combination of the above-mentioned types of memory. It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor or hardware, the method shown in the above embodiment is implemented.

[0180] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for cross-laser projection of a 3D target, characterized in that: The method comprises: Obtain target point cloud data obtained by a first laser radar deployed on the vehicle to detect the current environment, and obtain a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar used to scan the field of view of the vehicle's current direction of travel, and the second laser radar is a laser radar used to scan the entire field of view around the vehicle; Acquire associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D object based on the associated point cloud data, and acquire position data of the 3D object in the associated point cloud data; Based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

2. The method according to claim 1, characterized in that The step of acquiring associated point cloud data of the target point cloud data from the point cloud data sequence includes: Obtaining the timestamp corresponding to the target point cloud data; Acquire a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from the point cloud data sequence; Acquire first point cloud data of a frame closest to the timestamp from the first point cloud dataset, and acquire second point cloud data of a frame closest to the timestamp from the second point cloud dataset; The first point cloud data and the second point cloud data are used as the associated point cloud data.

3. The method according to claim 2, characterized in that The step of extracting a 3D target from the associated point cloud data and obtaining position data of the 3D target in the associated point cloud data includes: Acquire an associated same 3D object based on the first point cloud data and the second point cloud data; Acquire a first center point of the 3D object in the first point cloud data and a second center point of the 3D object in the second point cloud data; The first center point and the second center point are used as position data of the 3D object in the associated point cloud data.

4. The method according to claim 3, characterized in that The step of labeling the 3D target to the target point cloud data based on the emitted ray corresponding to each point cloud in the target point cloud data and the position data includes: Acquire, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray that intersects the line connecting the first center point and the second center point; Determine an intersection point between the target emitted ray and the connecting line and a time deviation corresponding to the intersection point; The intersection point with the smallest time deviation is marked as the center point of the 3D target in the target point cloud data.

5. The method according to claim 4, characterized in that The acquiring, from the emitted rays corresponding to each point cloud in the target point cloud data, the target emitted ray intersecting the line connecting the first center point and the second center point, comprises: Converting the point cloud contained in the target point cloud data to the vehicle coordinate system of the vehicle to obtain a transformation matrix of the vehicle between the start time and the end time of scanning the target point cloud data, wherein the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle; Using the transformation matrix, the first center point, the second center point, and all point clouds included in the target point cloud data are transformed into the vehicle coordinate system corresponding to the start time of the scan; Acquire an emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the start time of the scan; It is detected whether the emission ray intersects with the line between the first center point and the second center point, and the intersecting emission ray is used as the target emission ray.

6. The method according to claim 4, characterized in that The determining the intersection point between the target emitted ray and the connecting line and the time deviation corresponding to the intersection point includes: Converting the first center point and the second center point to the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point; Calculate a first emission time of the ray emitted from the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculate a second emission time of the ray emitted from the second center point based on the second coordinates and the acquisition parameters of the second laser radar; Calculate the time interpolation of the intersection point on the line and the emission time corresponding to the intersection point using the first emission time and the second emission time; A time offset is calculated based on the time interpolation and the emission instant of the intersection point.

7. The method according to claim 6, characterized in that The calculating the first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and the calculating the second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar, include: Extracting an initial scanning moment and a scanning period from acquisition parameters of the second laser radar; Determining a first angle between the 3D object and the rear of the vehicle using the first coordinates, and determining a second angle between the 3D object and the rear of the vehicle using the second coordinates; The first emission time is calculated using the first angle, the initial scanning time, and the scanning period, and the second emission time is calculated using the second angle, the initial scanning time, and the scanning period.

8. The method according to claim 7, characterized in that The calculating the first emission time by using the first included angle, the initial scanning time, and the scanning period, and the calculating the second emission time by using the second included angle, the initial scanning time, and the scanning period, include: Calculating a first ratio between the first angle and a preset value, calculating a first product between the first ratio and the scanning period, and summing the first product with the initial scanning time to obtain the first emission time; A second ratio between the second angle and a preset value is calculated, a second product between the second ratio and the scanning period is calculated, and the second emission time is obtained by summing the second product and the initial scanning time.

9. The method according to claim 6, characterized in that The calculating of the time interpolation of the intersection point on the line by using the first emission time and the second emission time includes: Obtaining a first distance between the first center point and the intersection point, and calculating a time difference between the second emission moment and the first emission moment; determining a first proportional relationship between the target time and the time difference based on a ratio between the first distance and the length of the connecting line, and calculating the target time based on the proportional relationship and the time difference; The time interpolation is calculated using the target time and the first emission time.

10. The method according to claim 4, characterized in that The step of marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes: Obtaining the heading angle corresponding to the intersection point with the smallest time deviation; Obtaining the coordinates and size parameters of the intersection point with the smallest time deviation; The center point of the 3D object is marked in the target point cloud data using the coordinates, the size parameters, and the heading angle.

11. The method according to claim 10, characterized in that The obtaining of the heading angle corresponding to the intersection point with the smallest time deviation includes: Obtaining a first distance between the first center point and the intersection point, and obtaining a first heading corresponding to the first center point and a second heading corresponding to the second center point; calculating a heading difference between the second heading and the first heading; determining a proportional relationship between the target heading and the heading difference based on a ratio between a first distance and a length of the connecting line, and calculating the target heading based on the proportional relationship and the heading difference, wherein the first distance is the distance between the first center point and the intersection point; The heading angle is calculated using the target heading and the first heading.

12. A 3D target cross-laser projection device, characterized in that: The device comprises: an acquisition module, configured to acquire target point cloud data obtained by a first laser radar deployed on a vehicle to detect a current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning a field of view of the vehicle's current direction of travel, and the second laser radar is a laser radar for scanning a full field of view around the vehicle; an extraction module, configured to obtain associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D object based on the associated point cloud data, and obtain position data of the 3D object in the associated point cloud data; A labeling module is used to label the 3D target to the target point cloud data based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first lidar and reflected back after encountering an object and being received.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 11 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 11.

Images