Cross-laser projection method and device of 3D target, electronic equipment and storage medium

Abstract

The invention relates to the field of automatic driving perception, in particular to a cross-laser projection method and device for a 3D target, electronic equipment and a storage medium. According to the application, the associated point cloud data of the target point cloud data is acquired from the point cloud data sequence, and the 3D target and the corresponding position data are extracted based on the associated point cloud data. And further marking the 3D target to the target point cloud data based on the emission ray and the position data corresponding to each point cloud in the target point cloud data, so that marking is carried out by using the emission ray of the point cloud and the position data of the 3D target, and the 3D target is marked to the point cloud data of the first laser radar under the condition that the point cloud time of the point cloud and the point cloud time of the 3D target are not synchronous. Therefore, the problems that due to the fact that the models of the mass production vehicle radar and the collection vehicle radar are different, point clouds generated by the mass production vehicle radar and the collection vehicle radar are not synchronous in time, and the point clouds output by the collection vehicle radar and the point clouds output by the mass production vehicle radar cannot be aligned in time and cannot be directly used as input of an algorithm are solved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving perception, and in particular to a method, device, electronic device and storage medium for cross-laser projection of a 3D target. Background Art

[0002] In the field of autonomous driving, perception is crucial, among which BEV perception has been a hot development direction in recent years, especially 3D detection algorithms. This algorithm can convert single-modal or multi-modal information such as images into BEV features to achieve perception and output 3D detection information of objects. The premise for implementing a 3D detection algorithm is to have 3D true value data that can be used for training. The current common practice in the industry is to install a laser scanning device (lidar) on the top of the vehicle to obtain a 360-degree full field of view, and then use the point cloud for 3D true value annotation. However, the models of mass-produced vehicle radars and collection vehicle radars are different, resulting in the point clouds generated by the two being out of sync in time. The true value of the point cloud output by the collection vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as the input of the algorithm. Although 3D annotation based on the point cloud of the mass-produced vehicle radar can solve this problem, it will incur additional costs. Summary of the invention

[0003] In view of this, the embodiments of the present invention provide a cross-laser projection method, device, electronic device and storage medium for 3D targets to solve the problem that the models of mass-produced vehicle radar and collection vehicle radar are different, resulting in the point clouds generated by the two being out of sync in time, and the true value of the point cloud output by the collection vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as input to the algorithm.

[0004] In a first aspect, an embodiment of the present invention provides a method for cross-laser projection of a 3D target, the method comprising:

[0005] Acquire target point cloud data obtained by detecting the current environment by a first laser radar deployed on the vehicle, and acquire a point cloud data sequence obtained by detecting the current environment by a second laser radar deployed on the vehicle, wherein the first laser radar is a laser radar for scanning the field of view of the current driving direction of the vehicle, and the second laser radar is a laser radar for scanning the full field of view around the vehicle;

[0006] Acquire associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D target based on the associated point cloud data, and acquire position data of the 3D target in the associated point cloud data;

[0007] Based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back and received after encountering an object.

[0008] Furthermore, the step of acquiring the associated point cloud data of the target point cloud data from the point cloud data sequence includes:

[0009] Obtaining a timestamp corresponding to the target point cloud data;

[0010] Acquire a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from the point cloud data sequence;

[0011] Acquire first point cloud data of a frame closest to the timestamp from the first point cloud data set, and acquire second point cloud data of a frame closest to the timestamp from the second point cloud data set;

[0012] The first point cloud data and the second point cloud data are used as the associated point cloud data.

[0013] Furthermore, extracting a 3D target from the associated point cloud data and obtaining position data of the 3D target in the associated point cloud data includes:

[0014] Acquire the same associated 3D object based on the first point cloud data and the second point cloud data;

[0015] Acquire a first center point of the 3D object in the first point cloud data and a second center point of the 3D object in the second point cloud data;

[0016] The first center point and the second center point are used as position data of the 3D target in the associated point cloud data.

[0017] Furthermore, the step of labeling the 3D target to the target point cloud data based on the emitted ray corresponding to each point cloud in the target point cloud data and the position data includes:

[0018] Acquire, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with the line between the first center point and the second center point;

[0019] Determine the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point;

[0020] The intersection point with the smallest time deviation is marked as the center point of the 3D target in the target point cloud data.

[0021] Furthermore, the step of acquiring, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with a line connecting the first center point and the second center point comprises:

[0022] The point cloud contained in the target point cloud data is converted to the vehicle coordinate system of the vehicle to obtain a transformation matrix of the vehicle between the scanning start time and the scanning end time of the target point cloud data, wherein the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle;

[0023] Using the transformation matrix, transform the first center point, the second center point and all point clouds included in the target point cloud data to the vehicle coordinate system corresponding to the start time of the scan;

[0024] Acquire the emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the scanning start time;

[0025] It is detected whether the emission ray intersects with the line between the first center point and the second center point, and the intersecting emission ray is used as the target emission ray.

[0026] Further, the determining of the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point includes:

[0027] Convert the first center point and the second center point to the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point;

[0028] Calculate a first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculate a second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar;

[0029] Calculate the time interpolation of the intersection point on the connecting line and calculate the emission time corresponding to the intersection point by using the first emission time and the second emission time;

[0030] A time offset is calculated based on the time interpolation and the emission instant of the intersection point.

[0031] Further, the calculating of the first emission time of the emission ray corresponding to the first center point based on the first coordinate and the acquisition parameters of the second laser radar, and the calculating of the second emission time of the emission ray corresponding to the second center point based on the second coordinate and the acquisition parameters of the second laser radar, include:

[0032] Extracting an initial scanning time and a scanning period from acquisition parameters of the second laser radar;

[0033] Determine a first angle between the 3D target and the rear of the vehicle using the first coordinates, and determine a second angle between the 3D target and the rear of the vehicle using the second coordinates;

[0034] The first emission time is calculated using the first angle, the initial scanning time and the scanning period, and the second emission time is calculated using the second angle, the initial scanning time and the scanning period.

[0035] Further, the calculating the first emission time by using the first angle, the initial scanning time and the scanning period, and the calculating the second emission time by using the second angle, the initial scanning time and the scanning period, include:

[0036] Calculating a first ratio between the first angle and a preset value, calculating a first product between the first ratio and the scanning period, and summing the first product and the initial scanning time to obtain the first emission time;

[0037] A second ratio between the second angle and a preset value is calculated, a second product between the second ratio and the scanning period is calculated, and the second emission time is obtained by summing the second product and the initial scanning time.

[0038] Furthermore, the calculating the time interpolation of the intersection point on the connecting line by using the first transmitting time and the second transmitting time includes:

[0039] Acquire a first distance between the first center point and the intersection point, and calculate a time difference between the second emission time and the first emission time;

[0040] Determine a first proportional relationship between the target time and the time difference based on a ratio between the first distance and the length of the connection line, and calculate the target time based on the proportional relationship and the time difference;

[0041] The time interpolation is calculated using the target time and the first emission time.

[0042] Furthermore, marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes:

[0043] Obtaining the heading angle corresponding to the intersection point with the smallest time deviation;

[0044] Obtaining the coordinates and size parameters of the intersection point with the smallest time deviation;

[0045] The coordinates, the size parameters and the heading angle are used to mark the center point of the 3D target in the target point cloud data.

[0046] Further, the obtaining of the heading angle corresponding to the intersection point with the smallest time deviation includes:

[0047] Acquire a first distance between the first center point and the intersection point, and acquire a first heading corresponding to the first center point and a second heading corresponding to the second center point;

[0048] calculating a heading difference between the second heading and the first heading;

[0049] Determining a proportional relationship between a target heading and the heading difference based on a ratio between a first distance and a length of the connecting line, and calculating the target heading based on the proportional relationship and the heading difference, wherein the first distance is a distance between the first center point and the intersection point;

[0050] The heading angle is calculated using the target heading and the first heading.

[0051] In a second aspect, an embodiment of the present invention provides a 3D target cross-laser projection device, characterized in that the device includes:

[0052] an acquisition module, used to acquire target point cloud data obtained by a first laser radar deployed on a vehicle to detect a current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning a field of view of a current driving direction of the vehicle, and the second laser radar is a laser radar for scanning a full field of view around the vehicle;

[0053] An extraction module, used for acquiring associated point cloud data of the target point cloud data from the point cloud data sequence, extracting a 3D target based on the associated point cloud data, and acquiring position data of the 3D target in the associated point cloud data;

[0054] A labeling module is used to label the 3D target to the target point cloud data based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back after encountering an object and being received.

[0055] In a third aspect, an embodiment of the present invention provides a computer device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the method of the first aspect or any corresponding embodiment thereof by executing the computer instructions.

[0056] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to enable a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0057] This application obtains the associated point cloud data of the target point cloud data from the point cloud data sequence, and extracts the 3D target and its corresponding position data based on this. Further, based on the emission ray corresponding to each point cloud in the target point cloud data and the acquired position data, the 3D target is annotated to the target point cloud data, so that the emission ray of the point cloud and the position data of the 3D target are used for annotation, so that when the point clouds of the two are not synchronized in time, the 3D target that is compatible with the point cloud data of the first laser radar is determined based on the point cloud data output by the second laser radar. This solves the problem that the mass-produced vehicle radar and the acquisition vehicle radar are of different models, resulting in the point clouds generated by the two being out of sync in time, and the point cloud output by the acquisition vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as the input of the algorithm.

[0058] This application obtains the associated point cloud data of the target point cloud data from the point cloud data sequence, and extracts the 3D target and its corresponding position data based on this. Further, based on the emission ray and position data corresponding to each point cloud in the target point cloud data, the 3D target is annotated to the target point cloud data, so that the emission ray of the point cloud and the position data of the 3D target are used for annotation, so that when the point clouds of the two are not synchronized in time, the 3D target is annotated to the point cloud data of the first laser radar. This solves the problem that the mass-produced vehicle radar and the acquisition vehicle radar are of different models, resulting in the point clouds generated by the two being out of sync in time, and the point cloud output by the acquisition vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as the input of the algorithm.

[0059] This application converts the point cloud contained in the target point cloud data to the self-vehicle coordinate system with the center point of the rear axle of the self-vehicle as the origin, and obtains the corresponding transformation matrix between the start time and the end time of the scan, which enables the point cloud data of different times and sources to be unified into a standard coordinate system, which is convenient for subsequent overall analysis and processing, and eliminates the confusion caused by the difference in coordinate systems. Secondly, the first center point, the second center point and all point clouds are transformed into the self-vehicle coordinate system corresponding to the start time of the scan using the transformation matrix, which further ensures that the data is in the same time and space reference, laying the foundation for accurate comparison and association of various data elements. Furthermore, obtaining the emission ray corresponding to each point cloud in the unified self-vehicle coordinate system can clearly and normatively describe the relative position relationship between the point cloud and the self-vehicle, and provide convenient conditions for subsequent intersection detection and other operations. Finally, the target emission ray is determined by detecting whether the emission ray intersects with the line between the first center point and the second center point, so that the emission ray related to the specific target can be accurately screened out from the many emission rays, which helps to improve the accuracy and effectiveness of the whole system in target perception and processing.

[0060] This application converts the first center point and the second center point to the vehicle coordinate system and obtains the corresponding first coordinates and second coordinates, which enables data from different sources to be analyzed in a unified coordinate system, laying the foundation for subsequent accurate calculations and associations, and effectively avoiding confusion and errors caused by inconsistent coordinate systems. The emission time of the emission rays corresponding to the first and second center points is calculated based on the above coordinates and the acquisition parameters of the second laser radar, specifically by extracting the initial scanning time and scanning period, and combining the determined first and second angles. This calculation method can accurately restore the emission time information of the emission rays corresponding to each center point based on the position of the target in the vehicle coordinate system and the scanning characteristics of the radar, so that the time context of the data generation can be more clearly grasped, which helps to deeply understand the state of the data at different times.

[0061] This application further uses the first emission time and the second emission time to calculate the time interpolation of the intersection point on the line and its emission time, and calculates the time deviation based on this. In the process of calculating the time interpolation, first obtain the distance between the first center point and the intersection point and calculate the time difference, then determine the proportional relationship between the target time and the time difference through the distance ratio to calculate the target time, and finally obtain the time interpolation. This method can accurately quantify the position characteristics of the intersection point in the time dimension, and the time difference between different points can be clearly compared through the calculation of the time deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0063] Figure 1 is a schematic diagram of a process of cross-laser projection of a 3D object according to some embodiments of the present invention;

[0064] Figure 2 is a schematic diagram of a line connecting an emission ray and a center point according to some embodiments of the present invention;

[0065] Figure 3 is a schematic diagram of heading projection according to some embodiments of the present invention;

[0066] Figure 4 is a structural block diagram of a cross-laser projection device for a 3D object according to an embodiment of the present invention;

[0067] Figure 5 It is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0069] According to an embodiment of the present invention, a method, device, electronic device and storage medium for cross-laser projection of a 3D target are provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0070] In this embodiment, a cross-laser projection method for a 3D target is provided. Figure 1 is a flow chart of a cross-laser projection method for a 3D target according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0071] Step S101, obtain the target point cloud data obtained by the first laser radar deployed on the vehicle to detect the current environment, and obtain the point cloud data sequence obtained by the second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar used to scan the field of view of the vehicle's current driving direction, and the second laser radar is a laser radar used to scan the full field of view around the vehicle.

[0072] In an embodiment of the present application, the first laser radar performs a field of view scan for the current direction of travel of the vehicle, which can provide accurate environmental information directly in front of the vehicle and ensure the accuracy of the driving route judgment, while the second laser radar is responsible for a 360° full-field scan around the vehicle, which can fully grasp the surrounding situation and eliminate visual blind spots. The two work together to build a comprehensive and detailed vehicle environment perception system. The second laser radar is mainly used to mark the true value of the bev 3D target. Specifically, the second laser radar scans the surrounding environment, and then projects the pre-marked target true value onto the point cloud data of the first laser radar. The point cloud collected by the first laser radar is called Q, and the point cloud collected by the second laser radar is called point cloud P.

[0073] The point cloud information covers the time t when the ray is emitted and the spatial coordinates x, y, z of the corresponding reflection point. And the 3D true value annotation has been completed based on the point cloud P, and the annotation information includes the center point position, length, width, height and heading angle of the object. Specifically, in the operation of obtaining relevant data, first of all, it is clear that a certain frame Q is known, and its point cloud scanning start and end times are t1 and t2 respectively, and then the two frames of point cloud P before and after Q are taken, namely point cloud P1 and point cloud P2. Since the scanning cycle of point cloud P is longer than that of Q, and the upper and lower frames of point cloud P are continuous in time, the scanning time of the point cloud data detected by the second laser radar of point cloud P1 and point cloud P2 can cover the time range corresponding to the target point cloud data detected by the first laser radar Q in the time period from t1 to t2, thereby realizing the acquisition of the target point cloud data currently detected by the first laser radar, and the two adjacent frames of point cloud data detected by the second laser radar.

[0074] Step S102, acquiring associated point cloud data of the target point cloud data from the point cloud data sequence, extracting a 3D target based on the associated point cloud data, and acquiring position data of the 3D target in the associated point cloud data.

[0075] In an embodiment of the present application, obtaining associated point cloud data of target point cloud data from a point cloud data sequence includes: obtaining a timestamp corresponding to the target point cloud data. Obtaining a first point cloud data set before the timestamp and a second point cloud data set after the timestamp from the point cloud data sequence. Obtaining first point cloud data of the closest frame to the timestamp from the first point cloud data set, and obtaining second point cloud data of the closest frame to the timestamp from the second point cloud data set. Using the first point cloud data and the second point cloud data as associated point cloud data.

[0076] Specifically, we first need to obtain the timestamp corresponding to the target point cloud data, which is the key identifier for accurately locating the specific moment of the target point cloud data on the time axis. Then, we need to obtain the point cloud data sequence detected by the second laser radar. This sequence contains multiple frames of continuous point cloud data. These point cloud data are the results of the second laser radar scanning at different times, presenting a continuous state arranged in chronological order.

[0077] After obtaining this complete point cloud data sequence, further screening operations are performed based on the timestamp of the target point cloud data previously obtained. From this multi-frame continuous point cloud data sequence, all point cloud data before the timestamp are found and classified as the first point cloud data set; at the same time, all point cloud data after the timestamp are found to form the second point cloud data set.

[0078] Finally, in the first point cloud data set that has been divided, by comparing the time interval between each point cloud data and the timestamp, the frame of point cloud data closest to the timestamp is screened out and determined as the first point cloud data; similarly, in the second point cloud data set, in this way, by comparing the time interval between each point cloud data and the timestamp, the frame of point cloud data closest to the timestamp is found and determined as the second point cloud data. Through such a series of operations, it is possible to accurately obtain the two frames of specific point cloud data closely related to the target point cloud data from the large amount of point cloud data detected by the second laser radar according to the timestamp of the target point cloud data.

[0079] In an embodiment of the present application, extracting a 3D target from associated point cloud data and obtaining position data of the 3D target in the associated point cloud data includes: obtaining the same associated 3D target based on first point cloud data and second point cloud data. Obtaining a first center point of the 3D target in the first point cloud data and a second center point of the 3D target in the second point cloud data. Using the first center point and the second center point as the position data of the 3D target in the associated point cloud data.

[0080] Specifically, first make sure that the first point cloud data (point cloud P1) and the second point cloud data (point cloud P2) have been acquired. Next, obtain the same associated 3D target from these two parts of point cloud data. The association here means that the point cloud data in different frames corresponds to the same actual target object. After determining this associated same 3D target, further obtain the relevant information of the target in the first point cloud data, that is, find the center point position of the target in the first point cloud data, and define it as the first center point. Similarly, for the second point cloud data, the center point position of the same 3D target is also obtained, and this center point is called the second center point. Finally, the first center point and the second center point are used as the position data of the 3D target in the associated point cloud data.

[0081] It should be noted that the scanning start and end times of the point cloud data Q of the first laser radar are t1 and t2. The two frames of point cloud data P1 and P2 before and after the point cloud data Q are taken from the point cloud data sequence detected by the second laser radar. Since the scanning cycle of the second laser radar is longer than that of the first laser radar, this means that in the same time interval, the second laser radar takes more time to complete a scan. Moreover, the upper and lower frames of the point cloud data detected by the second laser radar are continuous in time. Assume that the scanning cycle of the second laser radar is T, and the scanning cycle of the first laser radar is t (T>t). If the first laser radar completes the scanning of a frame of data Q in the time interval t1-t2, then since the scanning cycle of the second laser radar is long and continuous, its two frames of data P1 and P2 will cover this relatively short t1-t2 interval in time.

[0082] Step S103, based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back after encountering an object and being received.

[0083] In an embodiment of the present application, based on the emission ray and position data corresponding to each point cloud in the target point cloud data, annotating the 3D target to the target point cloud data includes the following steps A1-A3:

[0084] Step A1, obtaining the target emission ray intersecting with the line between the first center point and the second center point from the emission ray corresponding to each point cloud in the target point cloud data.

[0085] Specifically, obtaining the target emission ray intersecting the line between the first center point and the second center point from the emission ray corresponding to each point cloud in the target point cloud data includes the following steps A101-A104:

[0086] Step A101, convert the point cloud contained in the target point cloud data to the vehicle coordinate system, and obtain the transformation matrix of the vehicle between the scanning start time and the scanning end time of the target point cloud data, wherein the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle.

[0087] Specifically, the vehicle coordinate system is set, and its origin is the center point of the rear axle of the vehicle, which is used as a unified reference frame to describe the position of the point cloud. Next, with the help of odometry, the position and posture information of the vehicle at any time can be obtained. Here we focus on the position and posture information of the vehicle at times t1 and t2. The posture information covers the position and posture of the vehicle in three-dimensional space. Based on the acquired posture information of the vehicle at times t1 and t2, the three-dimensional space coordinate transformation process from time t2 to time t1 is calculated. This process involves the possible position movement (corresponding to the translation operation) and posture change (corresponding to the rotation operation) of the vehicle between these two times, so the transformation matrix containing the rotation matrix and the translation matrix can be obtained by calculation.

[0088] Specifically, the rotation matrix is ​​determined based on the difference in the orientation of the vehicle at time t1 and t2 using relevant mathematical formulas (such as the rotation matrix calculation formula based on Euler angles, etc.); the translation matrix is ​​calculated based on the difference in the position coordinates of the vehicle at these two times. Finally, the obtained rotation matrix and translation matrix are combined to successfully obtain the transformation matrix of the vehicle between the scanning start time and the scanning end time of the target point cloud data, thus laying the foundation for the subsequent accurate conversion of the point cloud contained in the target point cloud data to the vehicle coordinate system.

[0089] Step A102, using a transformation matrix to transform the first center point, the second center point and all point clouds included in the target point cloud data into the vehicle coordinate system corresponding to the start time of scanning.

[0090] Specifically, first, the transformation matrix from the scanning end time to the scanning start time of the target point cloud data is obtained. This transformation matrix contains the rotation matrix and the translation matrix, which describes the position and posture changes of the ego vehicle between these two times. For the first center point, assuming that its coordinate in the original coordinate system is K1, it is necessary to transform it to the ego vehicle coordinate system corresponding to the scanning start time. According to the principle of three-dimensional space coordinate transformation, it is achieved through matrix multiplication. That is, the original coordinate vector is multiplied by the rotation matrix in the transformation matrix to obtain the rotated coordinate vector, and then the vector corresponding to the translation matrix is ​​added. In this way, the new coordinate K2 of the first center point in the ego vehicle coordinate system corresponding to the scanning start time is obtained.

[0091] The same method applies to the second center point. Assuming that the coordinate of the second center point in the original coordinate system is L1, according to the above steps, first multiply it with the rotation matrix to obtain the rotated coordinate vector, and then add the vector corresponding to the translation matrix to obtain the new coordinate L2 of the second center point in the vehicle coordinate system corresponding to the start time of scanning.

[0092] For all the point clouds contained in the target point cloud data, for each of them, for example, a point cloud whose coordinates in the original coordinate system are Pi (here i represents the serial number of the point cloud), the same transformation method is also used. First, the coordinate vector of the point cloud is multiplied by the rotation matrix to obtain the rotated coordinate vector, and then the vector corresponding to the translation matrix is ​​added, and then each point cloud is accurately transformed to the self-vehicle coordinate system corresponding to the start time of the scan, and the corresponding new coordinates Pi' are obtained. Through such an operation, the process of using the transformation matrix to uniformly transform the first center point, the second center point and all the point clouds contained in the target point cloud data to the self-vehicle coordinate system corresponding to the start time of the scan is completed.

[0093] Step A103, obtaining the emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the scanning start time.

[0094] Specifically, at this time, the point clouds in the entire target point cloud data are already in the vehicle coordinate system with the center point of the rear axle of the vehicle as the coordinate origin. For each point cloud in the target point cloud data, the origin of the coordinate system of the vehicle (that is, the center point of the rear axle of the vehicle) is used as the starting point of the emission ray, and then the specific coordinate position of the point cloud in the vehicle coordinate system corresponding to the start time of the scan is used as the end point of the emission ray, so that the emission ray is determined from the starting point to the end point. According to this method, this operation is performed for each point cloud in the target point cloud data, so as to obtain the emission ray corresponding to each point cloud in the target point cloud data.

[0095] It should be noted that the point cloud contained in the target point cloud data is first converted to the vehicle's own vehicle coordinate system. By obtaining the transformation matrix between the start time and the end time of the target point cloud data scanning, the vehicle's motion state changes in this scanning phase can be clearly understood. Then, the first center point, the second center point, and all point clouds are transformed to the own vehicle coordinate system corresponding to the start time of the scan using the transformation matrix. This step is equivalent to using a unified, relatively initial vehicle state as a reference, eliminating differences in coordinates and other aspects caused by vehicle motion. Then, in the own vehicle coordinate system corresponding to this unified start time, the emission ray corresponding to the point cloud is obtained, and it is detected whether it intersects with the line connecting the two center points to determine the target emission ray. The entire process is to adjust and process the relevant data based on the consideration of the vehicle's own motion, so that when the center point of the 3D target in the target point cloud data is subsequently determined, the influence of vehicle motion can be excluded, thereby achieving compensation for vehicle motion and ensuring the accuracy and rationality of data processing and annotation.

[0096] Step A104, detecting whether the emission ray intersects with the line between the first center point and the second center point, and taking the intersecting emission ray as the target emission ray.

[0097] Specifically, according to the coordinates of the first center point and the second center point in the vehicle coordinate system corresponding to the start time of scanning, the equation of the straight line connecting the two center points is determined by the two-point formula and other methods. This straight line represents the line between the first center point and the second center point. Then, for each previously acquired emission ray determined by each point cloud in the target point cloud data (the emission ray is determined by taking the center point of the rear axle of the vehicle as the starting point and the coordinates of the point cloud in the vehicle coordinate system as the end point), the emission ray equation is combined with the above-mentioned line equation to form a system of equations. By solving this system of equations, it is determined whether there is a solution. If the system of equations has a solution, it means that the emission ray intersects with the line between the first center point and the second center point in space, and this emission ray is determined as the target emission ray; if the system of equations has no solution, it means that the emission ray does not intersect with the line, and it does not meet the conditions of being the target emission ray. In this way, all emission rays are detected and judged one by one, so as to find out those emission rays that intersect with the line between the first center point and the second center point, and use them as target emission rays.

[0098] This application converts the point cloud contained in the target point cloud data to the self-vehicle coordinate system with the center point of the rear axle of the self-vehicle as the origin, and obtains the corresponding transformation matrix between the start time and the end time of the scan, which enables the point cloud data of different times and sources to be unified into a standard coordinate system, which is convenient for subsequent overall analysis and processing, and eliminates the confusion caused by the difference in coordinate systems. Secondly, the first center point, the second center point and all point clouds are transformed into the self-vehicle coordinate system corresponding to the start time of the scan using the transformation matrix, which further ensures that the data is in the same time and space reference, laying the foundation for accurate comparison and association of various data elements. Furthermore, obtaining the emission ray corresponding to each point cloud in the unified self-vehicle coordinate system can clearly and normatively describe the relative position relationship between the point cloud and the self-vehicle, and provide convenient conditions for subsequent intersection detection and other operations. Finally, the target emission ray is determined by detecting whether the emission ray intersects with the line between the first center point and the second center point, so that the emission ray related to the specific target can be accurately screened out from the many emission rays, which helps to improve the accuracy and effectiveness of the whole system in target perception and processing.

[0099] Step A2, determining the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point.

[0100] Specifically, in the embodiment of the present application, determining the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point includes the following steps A201-A204:

[0101] Step A201, converting the first center point and the second center point into the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point.

[0102] Step A202, calculate the first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculate the second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar.

[0103] Specifically, calculating the first emission time of the emission ray corresponding to the first center point based on the first coordinate and the acquisition parameters of the second laser radar, and calculating the second emission time of the emission ray corresponding to the second center point based on the second coordinate and the acquisition parameters of the second laser radar, includes the following steps A2021-A2023:

[0104] Step A2021, extract the initial scanning time and scanning period from the acquisition parameters of the second laser radar.

[0105] Specifically, in the acquired second laser radar acquisition parameters, find the parameter value related to the start of the scan. This parameter value clearly identifies the moment when the second laser radar starts a complete scanning action, that is, the initial scanning moment. For example, when the initial scanning moment of point cloud P is Tp1, we need to accurately find the corresponding record of this starting scanning time point from the entire parameter set of the second laser radar, and extract it as the initial scanning moment required in the step.

[0106] The scanning cycle refers to the time required for the radar to complete a complete 360-degree scan. By analyzing the relevant parameters, if the scanning frequency of the radar is known (for example, how many times per second), the scanning cycle can be calculated by its reciprocal (i.e., scanning cycle = 1 / scanning frequency). If the time taken for each 360-degree scan is directly given in the parameters, just as the time taken for the point cloud P to scan 360 degrees is T in the given content, then this time value can be directly extracted as the scanning cycle.

[0107] Step A2022, using the first coordinate to determine a first angle between the 3D target and the rear of the vehicle, and using the second coordinate to determine a second angle between the 3D target and the rear of the vehicle.

[0108] Specifically, in the vehicle coordinate system with the center point of the rear axle of the vehicle as the origin of the coordinates, for determining the first angle using the first coordinate, first determine a vector pointing from the center point of the rear axle of the vehicle to the first center point based on the first coordinate corresponding to the first center point of the 3D target in the first point cloud data, and at the same time clarify the direction vector of the rear end of the vehicle. By calculating the dot product and respective modulus of these two vectors, and then calculating the cosine value based on the formula of the relationship between the vector dot product and the angle, finally the first angle between the 3D target (represented by the first center point) and the rear end of the vehicle is obtained through the inverse trigonometric function. Similarly, when determining the second angle using the second coordinate, determine the vector pointing from the center point of the rear axle of the vehicle to the second center point based on the second coordinate corresponding to the second center point of the 3D target in the second point cloud data, and combine the direction vector of the rear end of the vehicle, repeat the above steps of calculating the dot product, modulus, cosine value and solving through the inverse trigonometric function, so as to obtain the second angle between the 3D target (represented by the second center point) and the rear end of the vehicle.

[0109] Step A2023, calculate the first emission time using the first angle, initial scanning time and scanning period, and calculate the second emission time using the second angle, initial scanning time and scanning period.

[0110] Specifically, calculating the first emission time by using the first angle, the initial scanning time and the scanning period, and calculating the second emission time by using the second angle, the initial scanning time and the scanning period, include:

[0111] (1) Calculating a first ratio between a first angle and a preset value, calculating a first product between the first ratio and a scanning period, and summing the first product with an initial scanning time to obtain a first emission time.

[0112] The calculation formula for the first launch time is as follows: Where T1 is the first emission time, is the first angle, T is the scanning period, and Tp1 is the initial scanning time.

[0113] (2) Calculate a second ratio between the second angle and the preset value, calculate a second product between the second ratio and the scanning period, and sum the second product with the initial scanning time to obtain a second emission time.

[0114] The calculation formula for the second launch time is as follows: Where T2 is the second emission time, is the first angle, T is the scanning period, and Tp1 is the initial scanning time.

[0115] Step A203, using the first emission time and the second emission time, calculate the time interpolation of the intersection point on the connecting line and calculate the emission time corresponding to the intersection point.

[0116] Specifically, the time interpolation of the intersection point on the connecting line is calculated using the first transmitting time and the second transmitting time, including: obtaining a first distance (O1Q) between the first center point O1 and the intersection point Q, and calculating the time difference between the second transmitting time T2 and the first transmitting time T1; determining a first proportional relationship between the target time Ti and the time difference (T2-T1) based on the ratio (O1Q / O1O2) between the first distance (O1Q) and the connecting line length (O1O2), and calculating the target time based on the proportional relationship and the time difference; calculating the time interpolation (Tp) using the target time and the first transmitting time, Tp=T1+Ti.

[0117] For the intersection point, there will be a corresponding emission ray, and the emission time Ta of the emission ray is based on the working record mechanism of the first laser radar itself. When the first laser radar performs scanning work, it will detect the surrounding environment according to a certain time sequence. Every time a emission ray is emitted to detect the target and receive the reflected signal to form point cloud data, the specific time when the emission ray is emitted will be accurately recorded. Therefore, after determining the intersection point and its corresponding emission ray, the specific time corresponding to the emission of the emission ray can be found from these detailed working records of the radar equipment. This time is Ta.

[0118] Step A204, calculating the time deviation based on the time interpolation and the emission time of the intersection point.

[0119] Specifically, the time deviation of the intersection point Q on the point clouds P and Q is calculated. The time deviation ΔT is obtained by calculating the absolute value of the difference between the time interpolation Tp and the emission time Ta of the intersection point.

[0120] This application converts the first center point and the second center point to the vehicle coordinate system and obtains the corresponding first coordinates and second coordinates, which enables data from different sources to be analyzed in a unified coordinate system, laying the foundation for subsequent accurate calculations and associations, and effectively avoiding confusion and errors caused by inconsistent coordinate systems. The emission time of the emission rays corresponding to the first and second center points is calculated based on the above coordinates and the acquisition parameters of the second laser radar, specifically by extracting the initial scanning time and scanning period, and combining the determined first and second angles. This calculation method can accurately restore the emission time information of the emission rays corresponding to each center point based on the position of the target in the vehicle coordinate system and the scanning characteristics of the radar, so that the time context of the data generation can be more clearly grasped, which helps to deeply understand the state of the data at different times.

[0121] This application further uses the first emission time and the second emission time to calculate the time interpolation of the intersection point on the line and its emission time, and calculates the time deviation based on this. In the process of calculating the time interpolation, first obtain the distance between the first center point and the intersection point and calculate the time difference, then determine the proportional relationship between the target time and the time difference through the distance ratio to calculate the target time, and finally obtain the time interpolation. This method can accurately quantify the position characteristics of the intersection point in the time dimension, and the time difference between different points can be clearly compared through the calculation of the time deviation.

[0122] Step A203: mark the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data.

[0123] Specifically, in the embodiment of the present application, marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes the following steps A2031-A2033:

[0124] Step A2031, obtain the heading angle corresponding to the intersection point with the smallest time deviation.

[0125] Specifically, obtaining the heading angle corresponding to the intersection point with the smallest time deviation includes: obtaining a first distance between the first center point and the intersection point, and obtaining a first heading Yaw1 corresponding to the first center point and a second heading Yaw2 corresponding to the second center point; calculating the heading difference (Yaw2-Yaw1) between the second heading Yaw2 and the first heading Yaw1; determining the proportional relationship between the target heading and the heading difference (Yaw2-Yaw1) based on the ratio (O1Q / O1O2) between the first distance (O1Q) and the length of the connecting line (O1O2), and calculating the target heading Yawi based on the proportional relationship and the heading difference, wherein the first distance is the distance between the first center point and the intersection point; and calculating the heading angle Yaw using the target heading Yawi and the first heading.

[0126] Step A2032, obtaining the coordinates and size parameters of the intersection point with the smallest time deviation.

[0127] Specifically, after obtaining the time deviation ΔT of all intersection points, all calculated ΔT values ​​are compared one by one. By comparing the sizes of these values, the smallest value is selected. The intersection point corresponding to this minimum value is recorded as the target intersection point, and it is marked as Qi. The intersection point Qi has specific coordinates, which are determined in the corresponding coordinate system (such as the self-vehicle coordinate system with the center point of the rear axle of the self-vehicle as the coordinate origin set earlier). It accurately describes the specific position of Qi in space, that is, the center point position of the target on the Q point cloud, and its coordinates can be expressed as (x, y, z). At the same time, the size parameters corresponding to Qi, the size parameters here include length, width, and height, and these size parameters are consistent with the relevant sizes corresponding to the point cloud P.

[0128] Step A2033, using coordinates, size parameters and heading angles to mark the center point of the 3D target in the target point cloud data.

[0129] Specifically, after obtaining the coordinates, size parameters and known heading angle of the intersection point with the smallest time deviation, the annotation operation is performed in the target point cloud data. First, the acquired coordinate information is marked as the center point position of the 3D true value target in the target point cloud data. This coordinate accurately determines the specific position of the target in three-dimensional space. Then, combined with the size parameters, these size parameters can further clarify the size range of the target in each dimension, such as length, width, height and other information, so that the shape and occupied range of the target in space are clearer. Finally, using the known heading angle, it can describe the angular relationship of the target relative to a reference direction (such as the direction of vehicle travel, etc.). By integrating the three aspects of coordinates, size parameters and heading angle information, the point is completely and accurately marked as the center point of the 3D true value target in the target point cloud data, providing accurate target position and morphology and other key information for subsequent related applications such as target recognition and path planning.

[0130] The present application first obtains the target point cloud data currently detected by the first laser radar and the two adjacent frames of point cloud data of the second laser radar associated with it, and uses the two frames of point cloud data to find the same associated 3D target and its first and second center points in the two frames of data, and then determines the target emission ray and the intersection point and the corresponding time deviation that intersect with the line connecting the two center points from the emission rays corresponding to each point cloud of the target point cloud data, and then marks the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data. In this way, by establishing an association between different frames of point cloud data and the mass-produced vehicle target point cloud data, finding a corresponding relationship and marking with the minimum time deviation as the standard, it is possible to determine the 3D target annotation that can be adapted to the mass-produced vehicle point cloud data based on the point cloud data of the acquisition vehicle when the point clouds of the two are not synchronized in time. This solves the problem that the point clouds generated by the mass-produced vehicle radar and the acquisition vehicle radar are different in model, resulting in the point clouds generated by the two being not synchronized in time, and the point cloud output by the acquisition vehicle radar cannot be aligned in time with the point cloud output by the mass-produced vehicle radar, and cannot be directly used as the input of the algorithm.

[0131] The following is a complete target labeling process:

[0132] Step 1: Name the second laser radar point cloud P and the first laser radar Q, and specify that the point cloud information includes the emission time t of the emission ray and the spatial coordinates x, y, z of the corresponding reflection point. At the same time, it is determined that the 3D true value annotation has been completed based on the point cloud P, and the annotation information covers the center point position, length, width, height and heading angle of the object.

[0133] Step 2: For a certain frame of known point cloud Q, determine the start and end times of its point cloud scanning as t1 and t2 respectively. Select the two frames of point cloud P before and after Q, and record them as point cloud P1 and point cloud P2 respectively. In view of the scanning cycle characteristics of point cloud P (the scanning cycle is longer than Q and the upper and lower frames are continuous in time), ensure that the scanning time of point cloud P1 and point cloud P2 can cover t1 and t2.

[0134] Step 3, find the same associated target on point cloud P1 and point cloud P2, the target is recorded as P1 on point cloud P1, and recorded as P2 on point cloud P2, and at the same time determine the center point of P1 as O1 and the center point of P2 as O2.

[0135] Step 4: Taking the center point of the rear axle of the vehicle as the coordinate origin, convert the center point O1 of P1 and the center point O2 of P2 to the vehicle coordinate system.

[0136] Step 5: Based on the initial scanning time Tp1 of the point cloud P and the time T used to scan 360 degrees, when the coordinates of the center point of the object are known, the angle ψ between the point and the rear direction of the vehicle is calculated by the formula (calculation formula: the emission time of the emission ray corresponding to the center point Then, the emission time T1 of the ray corresponding to O1 and the emission time T2 of the ray corresponding to O2 are obtained.

[0137] Step 6: Also use the center point of the rear axle of the ego vehicle as the coordinate origin, and convert all point clouds on Q to the ego vehicle coordinate system. Then, use the ego vehicle's posture information at times t1 and t2 obtained through odometry to calculate the three-dimensional space coordinate transformation process from time t2 to time t1 (including rotation matrix and translation matrix), and then use this transformation matrix to uniformly transform O1, O2 and all point clouds on Q to the ego vehicle coordinate system at the initial moment of Q scanning (the coordinate origin of this coordinate system is C). This process realizes the ego vehicle motion compensation for the point cloud.

[0138] Step 7, in the three-dimensional space coordinate system, connect the two points O1 and O2 to form a line O1O2. Since the actual time interval between O1 and O2 is short, O1 and O2 can be regarded as the uniform linear motion trajectory of the target.

[0139] Step 8, select a point A from the Q point cloud, determine its corresponding emission ray as CA, and record the emission time of the emission ray as Ta.

[0140] Step 9, determine whether the emitted ray CA intersects with the line O1O2 in space. If so, name the intersection point Q.

[0141] Step 10, Reference Figure 2 If there is an intersection point Q, the time interpolation of point Q on O1O2 is calculated according to the formula O1Q / O1O2=Ti / T2-T1 (where Tp=T1+Ti).

[0142] Step 11: If there is an intersection point Q, calculate the heading difference of point Q (such as Figure 3 shown).

[0143] Step 12: For the intersection point Q, calculate its time deviation on the point clouds P and Q, and the calculation formula is ΔT=|Tp-Ta|.

[0144] Step 13, repeat steps 8 to 12, and perform the same operation for all other points on Q. If the emitted ray intersects the line O1O2, calculate the corresponding ΔT.

[0145] Step 14, compare all the calculated ΔT values ​​and find the minimum value. The intersection point Qi corresponding to the minimum value is the center point of the target on the Q point cloud. The heading corresponding to Qi is the Yaw obtained by interpolation, and its length, width, and height are consistent with the point cloud P. From this, the center point coordinates x, y, z, length, width, height, and heading angle of the target on Q can be obtained.

[0146] Step 14, repeat steps 2 to 14, and project all 3D objects marked on point cloud P1 and point cloud P2 onto Q, thereby completing the 3D true value marking of Q.

[0147] In this embodiment, a 3D target cross-laser projection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0148] This embodiment provides a 3D target cross-laser projection device, such as Figure 4 As shown, including:

[0149] An acquisition module 401 is used to acquire target point cloud data obtained by a first laser radar deployed on the vehicle to detect the current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning the field of view of the current driving direction of the vehicle, and the second laser radar is a laser radar for scanning the full field of view around the vehicle;

[0150] An extraction module 402 is used to obtain associated point cloud data of the target point cloud data from the point cloud data sequence, extract the 3D target based on the associated point cloud data, and obtain position data of the 3D target in the associated point cloud data;

[0151] The labeling module 403 is used to label the 3D target to the target point cloud data based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back after encountering an object and being received.

[0152] In an embodiment of the present application, an extraction module is used to obtain a timestamp corresponding to target point cloud data; obtain a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from a point cloud data sequence; obtain the first point cloud data of the frame closest to the timestamp from the first point cloud data set, and obtain the second point cloud data of the frame closest to the timestamp from the second point cloud data set; and use the first point cloud data and the second point cloud data as associated point cloud data.

[0153] In an embodiment of the present application, an extraction module is used to obtain the same associated 3D target based on first point cloud data and second point cloud data; obtain a first center point of the 3D target in the first point cloud data and a second center point of the 3D target in the second point cloud data; and use the first center point and the second center point as position data of the 3D target in the associated point cloud data.

[0154] In the embodiment of the present application, the marking module includes:

[0155] A processing submodule, used for obtaining, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with the line between the first center point and the second center point;

[0156] A determination submodule is used to determine the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point;

[0157] The screening submodule is used to mark the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data.

[0158] In the embodiment of the present application, the processing submodule includes:

[0159] A first conversion unit is used to convert the point cloud included in the target point cloud data into the ego vehicle coordinate system, and obtain a transformation matrix of the ego vehicle between the scanning start time and the scanning end time of the target point cloud data, wherein the origin of the ego vehicle coordinate system is the center point of the rear axle of the ego vehicle;

[0160] An execution unit, used to transform the first center point, the second center point and all point clouds included in the target point cloud data into a self-vehicle coordinate system corresponding to the start time of scanning by using a transformation matrix;

[0161] An acquisition unit, used for acquiring an emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the start time of scanning;

[0162] The detection unit is used to detect whether the emission ray intersects with the line between the first center point and the second center point, and use the intersecting emission ray as the target emission ray.

[0163] In the embodiment of the present application, the submodule is determined, including:

[0164] A second conversion unit, used to convert the first center point and the second center point into the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point;

[0165] A first calculation unit is used to calculate a first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and to calculate a second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar;

[0166] A second calculation unit, used to calculate the time interpolation of the intersection point on the connecting line and calculate the emission time corresponding to the intersection point by using the first emission time and the second emission time;

[0167] The third calculation unit is used to calculate the time deviation based on the time interpolation and the emission time of the intersection point.

[0168] In an embodiment of the present application, a first calculation unit is used to extract an initial scanning moment and a scanning period from acquisition parameters of a second laser radar; determine a first angle between a 3D target and the rear of a vehicle using the first coordinates, and determine a second angle between the 3D target and the rear of the vehicle using the second coordinates; calculate a first emission moment using the first angle, the initial scanning moment and the scanning period, and calculate a second emission moment using the second angle, the initial scanning moment and the scanning period.

[0169] In an embodiment of the present application, a first calculation unit is used to calculate a first ratio between a first angle and a preset value, calculate a first product between the first ratio and a scanning period, and sum the first product with an initial scanning time to obtain a first emission time; calculate a second ratio between a second angle and a preset value, calculate a second product between the second ratio and a scanning period, and sum the second product with an initial scanning time to obtain a second emission time.

[0170] In an embodiment of the present application, a second calculation unit is used to obtain a first distance between the first center point and the intersection point, and calculate the time difference between the second launch moment and the first launch moment; determine a first proportional relationship between the target time and the time difference based on the ratio between the first distance and the length of the connecting line, and calculate the target time based on the proportional relationship and the time difference; and calculate time interpolation using the target time and the first launch moment.

[0171] In an embodiment of the present application, the screening submodule is used to obtain the heading angle corresponding to the intersection point with the smallest time deviation; obtain the coordinates and size parameters of the intersection point with the smallest time deviation; and use the coordinates, size parameters and heading angle to mark the center point of the 3D target in the target point cloud data.

[0172] In an embodiment of the present application, the screening submodule is used to obtain a first distance between a first center point and an intersection point, and obtain a first heading corresponding to the first center point and a second heading corresponding to the second center point; calculate a heading difference between the second heading and the first heading; determine a proportional relationship between a target heading and the heading difference based on a ratio between the first distance and a length of a connecting line, and calculate the target heading based on the proportional relationship and the heading difference; and calculate a heading angle using the target heading and the first heading.

[0173] See also Figure 5 , Figure 5 is a schematic diagram of the structure of a computer device provided by an optional embodiment of the present invention, such as Figure 5 As shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process the instructions executed in the computer device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as, a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system).

[0174] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0175] The memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiment.

[0176] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created by the use of a computer device based on the presentation of a small program landing page, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0177] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0178] The computer device further comprises a communication interface 30 for the computer device to communicate with other devices or a communication network.

[0179] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0180] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A cross-laser projection method for a 3D target, characterized in that: The method comprises: Acquire target point cloud data obtained by detecting the current environment by a first laser radar deployed on the vehicle, and acquire a point cloud data sequence obtained by detecting the current environment by a second laser radar deployed on the vehicle, wherein the first laser radar is a laser radar for scanning the field of view of the current driving direction of the vehicle, and the second laser radar is a laser radar for scanning the full field of view around the vehicle; Acquire associated point cloud data of the target point cloud data from the point cloud data sequence, extract a 3D target based on the associated point cloud data, and acquire position data of the 3D target in the associated point cloud data; Based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, the 3D target is annotated to the target point cloud data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back and received after encountering an object.

2. The method according to claim 1, characterized in that The step of acquiring the associated point cloud data of the target point cloud data from the point cloud data sequence comprises: Obtaining a timestamp corresponding to the target point cloud data; Acquire a first point cloud data set located before the timestamp and a second point cloud data set located after the timestamp from the point cloud data sequence; Acquire first point cloud data of a frame closest to the timestamp from the first point cloud data set, and acquire second point cloud data of a frame closest to the timestamp from the second point cloud data set; The first point cloud data and the second point cloud data are used as the associated point cloud data.

3. The method according to claim 2, characterized in that The step of extracting a 3D target from the associated point cloud data and obtaining position data of the 3D target in the associated point cloud data includes: Acquire the same associated 3D object based on the first point cloud data and the second point cloud data; Acquire a first center point of the 3D object in the first point cloud data and a second center point of the 3D object in the second point cloud data; The first center point and the second center point are used as position data of the 3D target in the associated point cloud data.

4. The method according to claim 3, characterized in that The step of labeling the 3D target to the target point cloud data based on the emitted ray corresponding to each point cloud in the target point cloud data and the position data includes: Acquire, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with the line between the first center point and the second center point; Determine the intersection point between the target emission ray and the connecting line and the time deviation corresponding to the intersection point; The intersection point with the smallest time deviation is marked as the center point of the 3D target in the target point cloud data.

5. The method according to claim 4, characterized in that The step of acquiring, from the emission rays corresponding to each point cloud in the target point cloud data, a target emission ray intersecting with a line connecting the first center point and the second center point comprises: The point cloud contained in the target point cloud data is converted to the vehicle coordinate system of the vehicle to obtain a transformation matrix of the vehicle between the scanning start time and the scanning end time of the target point cloud data, wherein the origin of the vehicle coordinate system is the center point of the rear axle of the vehicle; Using the transformation matrix, transform the first center point, the second center point and all point clouds included in the target point cloud data to the vehicle coordinate system corresponding to the start time of the scan; Acquire the emission ray corresponding to each point cloud in the target point cloud data in the vehicle coordinate system corresponding to the scanning start time; It is detected whether the emission ray intersects with the line between the first center point and the second center point, and the intersecting emission ray is used as the target emission ray.

6. The method according to claim 4, characterized in that The determining the intersection point between the target emitted ray and the connecting line and the time deviation corresponding to the intersection point includes: Convert the first center point and the second center point to the vehicle coordinate system to obtain a first coordinate corresponding to the first center point and a second coordinate corresponding to the second center point; Calculate a first emission time of the emission ray corresponding to the first center point based on the first coordinates and the acquisition parameters of the second laser radar, and calculate a second emission time of the emission ray corresponding to the second center point based on the second coordinates and the acquisition parameters of the second laser radar; Calculate the time interpolation of the intersection point on the connecting line and calculate the emission time corresponding to the intersection point by using the first emission time and the second emission time; A time offset is calculated based on the time interpolation and the emission instant of the intersection point.

7. The method according to claim 6, characterized in that The calculating of the first emission time of the emission ray corresponding to the first center point based on the first coordinate and the acquisition parameters of the second laser radar, and the calculating of the second emission time of the emission ray corresponding to the second center point based on the second coordinate and the acquisition parameters of the second laser radar, include: Extracting an initial scanning time and a scanning period from acquisition parameters of the second laser radar; Determine a first angle between the 3D target and the rear of the vehicle using the first coordinates, and determine a second angle between the 3D target and the rear of the vehicle using the second coordinates; The first emission time is calculated using the first angle, the initial scanning time and the scanning period, and the second emission time is calculated using the second angle, the initial scanning time and the scanning period.

8. The method according to claim 7, characterized in that The calculating the first emission time by using the first angle, the initial scanning time and the scanning period, and the calculating the second emission time by using the second angle, the initial scanning time and the scanning period, comprises: Calculating a first ratio between the first angle and a preset value, calculating a first product between the first ratio and the scanning period, and summing the first product and the initial scanning time to obtain the first emission time; A second ratio between the second angle and a preset value is calculated, a second product between the second ratio and the scanning period is calculated, and the second emission time is obtained by summing the second product and the initial scanning time.

9. The method according to claim 6, characterized in that The calculating the time interpolation of the intersection point on the connecting line by using the first transmitting time and the second transmitting time comprises: Acquire a first distance between the first center point and the intersection point, and calculate a time difference between the second emission time and the first emission time; Determine a first proportional relationship between the target time and the time difference based on a ratio between the first distance and the length of the connection line, and calculate the target time based on the proportional relationship and the time difference; The time interpolation is calculated using the target time and the first transmission time.

10. The method according to claim 4, characterized in that The step of marking the intersection point with the smallest time deviation as the center point of the 3D target in the target point cloud data includes: Obtaining the heading angle corresponding to the intersection point with the smallest time deviation; Obtaining the coordinates and size parameters of the intersection point with the smallest time deviation; The coordinates, the size parameters and the heading angle are used to mark the center point of the 3D target in the target point cloud data.

11. The method according to claim 10, characterized in that The obtaining of the heading angle corresponding to the intersection point with the smallest time deviation includes: Acquire a first distance between the first center point and the intersection point, and acquire a first heading corresponding to the first center point and a second heading corresponding to the second center point; calculating a heading difference between the second heading and the first heading; Determining a proportional relationship between a target heading and the heading difference based on a ratio between a first distance and a length of the connecting line, and calculating the target heading based on the proportional relationship and the heading difference, wherein the first distance is a distance between the first center point and the intersection point; The heading angle is calculated using the target heading and the first heading.

12. A 3D target cross-laser projection device, characterized in that: The device comprises: an acquisition module, used to acquire target point cloud data obtained by a first laser radar deployed on a vehicle to detect a current environment, and to acquire a point cloud data sequence obtained by a second laser radar deployed on the vehicle to detect the current environment, wherein the first laser radar is a laser radar for scanning a field of view of a current driving direction of the vehicle, and the second laser radar is a laser radar for scanning a full field of view around the vehicle; An extraction module, used for acquiring associated point cloud data of the target point cloud data from the point cloud data sequence, extracting a 3D target based on the associated point cloud data, and acquiring position data of the 3D target in the associated point cloud data; A labeling module is used to label the 3D target to the target point cloud data based on the emission ray corresponding to each point cloud in the target point cloud data and the position data, wherein the emission ray corresponding to the point cloud is used to represent the path formed by the laser beam emitted by the first laser radar and reflected back after encountering an object and being received.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the method according to any one of claims 1 to 11 by executing the computer instructions.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 11.

Images