A point cloud calibration method and device

Abstract

The embodiment of the present invention discloses a method and device for calibrating a point cloud, which includes: obtaining the position information of the original point cloud from a radar; rotating the original point cloud coordinate system, determining the coarse calibration parameters according to the coordinate positions before and after the rotation, and adding an affine transformation matrix based on the coarse calibration parameters to perform a coarse calibration of the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is parallel to the straight line where the scene marker is located in the horizontal direction; filtering the coarsely calibrated point cloud to obtain the height range corresponding to the coarsely calibrated point cloud ground; performing plane fitting on the points within the height range, and determining the fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the coarsely calibrated point cloud ground; adding an affine transformation matrix based on the fine calibration parameters, finely calibrating the coarsely calibrated point cloud to obtain a calibrated point cloud. By adopting the above technical solution, the calibration accuracy of the point cloud ground is improved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of automotive electronics technology, and in particular, to a point cloud calibration method and device. Background Art

[0002] The point cloud data obtained through the radar reflection information contains the precise three-dimensional position information and intensity information of the object. Among them, the radar is the carrier for generating 3D scenes, and the installation of the radar can directly determine the state of the point cloud scene.

[0003] However, due to various reasons such as wind, rain or human factors, the radar installation cannot guarantee that the horizontal plane of the point cloud based on the radar coordinate system (referred to as "point cloud ground") will maintain the same Z value in all directions when collecting point cloud data. Therefore, it is necessary to calibrate the point cloud ground, that is, convert the point cloud data to the same standard coordinate system.

[0004] In related technologies, point cloud calibration usually adopts a plane fitting method. This method is prone to point cloud ground fitting errors in rainy and foggy weather or in complex environment scenes. For example, when the roof of a vehicle is very smooth, it is easy to use the point cloud plane corresponding to the roof as the point cloud ground. This will cause a very large offset in the point cloud position of the entire scene, thereby affecting the subsequent extraction of point cloud feature points. Summary of the invention

[0005] The embodiment of the present invention provides a point cloud calibration method and device, so as to improve the calibration accuracy of the point cloud ground.

[0006] In a first aspect, an embodiment of the present invention provides a point cloud calibration method, comprising:

[0007] Get the location information of the original point cloud from the radar;

[0008] The original point cloud coordinate system is rotated, the rough calibration parameters are determined according to the coordinate positions before and after the rotation, and the affine transformation matrix is ​​added based on the rough calibration parameters to perform rough calibration of the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene;

[0009] Filter the roughly calibrated point cloud to obtain the height range of the roughly calibrated point cloud ground;

[0010] Perform plane fitting on the points within the height range, and determine the fine calibration parameters of the point cloud based on the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after rough calibration;

[0011] Based on the fine calibration parameters, the affine transformation matrix is ​​added to perform fine calibration on the coarsely calibrated point cloud to obtain the calibrated point cloud.

[0012] Optionally, the method provided by the embodiment of the present invention further includes:

[0013] Perform ground filtering on the calibrated point cloud;

[0014] The point cloud data after the ground is filtered out is input into the neural network model to extract feature information of the point cloud data after the ground is filtered out through the neural network model.

[0015] Optionally, performing ground filtering on the calibrated point cloud includes:

[0016] The calibrated point cloud ground points that satisfy the following formula are deleted, and the calibrated non-point cloud ground points that do not satisfy the following formula are retained:

[0017] |P3(z3)-Z″|<M

[0018] Among them, P3(z3) represents the position information of the point cloud in the Z coordinate axis direction after precise calibration, Z″ is the height value of the point cloud ground in the Z coordinate axis direction after calibration, and M represents the height range of M meters above and below the point cloud ground after calibration.

[0019] Optionally, performing a rough coordinate calibration on the position information includes:

[0020] If the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane of the original point cloud coordinate system meet the coarse calibration conditions, the coarse calibration parameters corresponding to the XOY plane are calculated to perform coarse calibration on the X and Y axes of the original point cloud coordinate system.

[0021] Among them, the rough calibration conditions include that the Z coordinate axis direction of the original point cloud coordinate system is parallel to the vertical direction of the scene landmark, the X coordinate axis and the Y coordinate axis are the point cloud ground coordinate axes of the point cloud coordinate system, and the Z coordinate axis is perpendicular to the X coordinate axis and perpendicular to the Y coordinate axis.

[0022] Optionally, the method provided by the embodiment of the present invention further includes:

[0023] When the coarse calibration parameters corresponding to the YOZ plane and / or the coarse calibration parameters corresponding to the XOZ plane of the original point cloud coordinate system do not meet the coarse calibration conditions, the Z coordinate axis of the original point cloud coordinate system is coarsely calibrated by respectively calculating the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane, so that the coarse calibration parameters of the YOZ plane and the XOZ plane corresponding to the Z coordinate axis after the coarse calibration both meet the coarse calibration conditions.

[0024] The calculation method of the coarse calibration parameters corresponding to the YOZ plane and the coarse calibration parameters corresponding to the XOZ plane is the same as the calculation method of the coarse calibration parameters corresponding to the XOY plane.

[0025] Optionally, coarse calibration parameters are determined according to the coordinate positions before and after the rotation, and an affine transformation matrix is ​​added based on the coarse calibration parameters to perform coarse coordinate calibration on the position information, including:

[0026] All original point clouds are rotated according to the following formula to roughly calibrate the coordinates of the position information:

[0027] The original point cloud coordinates are recorded as P 1 (x 1 ,y 1 ,z 1 ), after the original point cloud coordinate system is rotated, the point cloud P 1 The corresponding coordinate position of the rotated point cloud is P 2 (x 2 ,y 2 ,z 2 ), where the position information x, y, z satisfies the following affine transformation matrix:

[0028]

[0029] Among them, the coarse calibration parameters

[0030] Optionally, filter the roughly calibrated point cloud, including:

[0031] The ground points in the roughly calibrated point cloud that satisfy the following formula are retained, and the non-ground points in the roughly calibrated point cloud that do not satisfy the following formula are filtered, where:

[0032] |P2(z2)-Z|<N

[0033] Among them, P2(z2) represents the position information of the point cloud after rough calibration in the Z coordinate axis direction, Z represents the maximum value of the ground points obtained by comparing the relevant key points in the real scene, where the relevant key points are points at non-point cloud edge positions, and N represents the difference between the maximum height of the radar from the real ground and the minimum height of the radar from the real ground.

[0034] Optionally, plane fitting is performed on the points within the height range, and the fine calibration parameters of the point cloud are determined according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration, including:

[0035] Randomly select three points within the height range for plane fitting to obtain a candidate fitting plane;

[0036] For any candidate fitting plane, determine the sum of the distances from all points within the height range to the candidate fitting plane, and take the candidate fitting plane corresponding to the minimum sum of distances as the target fitting plane;

[0037] According to the rotation angle between the coarsely calibrated ground and the target fitting plane corresponding to the coarsely calibrated point cloud, the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud are determined.

[0038] Optionally, determining the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud according to the rotation angle between the coarsely calibrated ground corresponding to the coarsely calibrated point cloud and the target fitting plane includes:

[0039] Get all sample frames of the original point cloud;

[0040] For each sample frame, the rotation angle of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud is determined according to the rotation angle between the coarsely calibrated ground and the target fitting plane corresponding to the coarsely calibrated point cloud;

[0041] For each coordinate axis, the average value of the rotation angles corresponding to the coordinate axis in all sample frames is used as the precise calibration parameter corresponding to the coordinate axis.

[0042] Optionally, add an affine transformation matrix based on the fine calibration parameters, including:

[0043] Note θ x represents the X-axis precision calibration parameter, θ y represents the fine calibration parameter of the Y axis, θ z represents the fine calibration parameters of the Z axis, z′ represents the height information of the target fitting plane, and the affine transformation matrix added based on the above fine calibration parameters is:

[0044]

[0045] Accordingly, the roughly calibrated point cloud is finely calibrated to obtain the calibrated point cloud, including:

[0046] According to the following formula, the point cloud after rough calibration is finely calibrated to obtain the calibrated point cloud:

[0047]

[0048] Among them, (x 2 ,y 2 ,z 2 ) represents the position information of the point cloud after rough calibration, (x 3 ,y 3 ,z 3 ) represents the position information of the calibrated point cloud.

[0049] In a second aspect, an embodiment of the present invention further provides a point cloud calibration device, comprising:

[0050] A position information acquisition module is configured to acquire position information of an original point cloud from a radar;

[0051] A coarse calibration module is configured to rotate the original point cloud coordinate system, determine the coarse calibration parameters according to the coordinate positions before and after the rotation, and add an affine transformation matrix based on the coarse calibration parameters to perform a coarse calibration of the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene;

[0052] A first filtering module is configured to filter the roughly calibrated point cloud to obtain a height range corresponding to the ground of the roughly calibrated point cloud;

[0053] A fine calibration parameter determination module is configured to perform plane fitting on points within the height range, and determine fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration;

[0054] The fine calibration module is configured to add an affine transformation matrix based on the fine calibration parameters, perform fine calibration on the point cloud after the coarse calibration, and obtain a calibrated point cloud.

[0055] Optionally, the device provided by the embodiment of the present invention further includes:

[0056] A second filtering module is configured to perform ground filtering on the calibrated point cloud;

[0057] The feature extraction module is configured to input the point cloud data after the ground is filtered out into the neural network model, so as to extract feature information of the point cloud data after the ground is filtered out through the neural network model.

[0058] Optionally, the second filtering module is specifically configured as follows:

[0059] The calibrated point cloud ground points that satisfy the following formula are deleted, and the calibrated non-point cloud ground points that do not satisfy the following formula are retained:

[0060] |P3(z3)-Z″|<M

[0061] Among them, P3(z3) represents the position information of the point cloud in the Z coordinate axis direction after precise calibration, Z″ is the height value of the point cloud ground in the Z coordinate axis direction after calibration, and M represents the height range of M meters above and below the point cloud ground after calibration.

[0062] Optionally, the coarse calibration module is specifically configured as follows:

[0063] Rotate the original point cloud coordinate system and determine the rough calibration parameters according to the coordinate positions before and after the rotation;

[0064] If the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane of the original point cloud coordinate system meet the coarse calibration conditions, the coarse calibration parameters corresponding to the XOY plane are calculated, and the affine transformation matrix is ​​added based on the coarse calibration parameters to perform coarse calibration on the X-axis and Y-axis of the original point cloud coordinate system, wherein the coarse calibration conditions include that the Z-axis direction of the original point cloud coordinate system is parallel to the vertical direction of the scene landmark, the X-axis and the Y-axis are the point cloud ground coordinate axes of the point cloud coordinate system, and the Z-axis is perpendicular to the X-axis and to the Y-axis.

[0065] Optionally, the device provided by the embodiment of the present invention further includes:

[0066] The pre-calibration module is configured to perform a rough calibration on the Z coordinate axis of the original point cloud coordinate system by respectively calculating the rough calibration parameters corresponding to the YOZ plane and the XOZ plane when the rough calibration parameters corresponding to the YOZ plane and / or the rough calibration parameters corresponding to the XOZ plane of the original point cloud coordinate system do not meet the rough calibration conditions, so that the rough calibration parameters of the YOZ plane and the XOZ plane corresponding to the Z coordinate axis after the rough calibration both meet the rough calibration conditions,

[0067] The calculation method of the coarse calibration parameters corresponding to the YOZ plane and the coarse calibration parameters corresponding to the XOZ plane is the same as the calculation method of the coarse calibration parameters corresponding to the XOY plane.

[0068] Optionally, the coarse calibration module is specifically configured as follows:

[0069] All original point clouds are rotated according to the following formula to roughly calibrate the coordinates of the position information:

[0070] The original point cloud coordinates are recorded as P 1 (x 1 ,y 1 ,z 1 ), after the original point cloud coordinate system is rotated, the point cloud P 1 The corresponding coordinate position of the rotated point cloud is P 2 (x 2 ,y 2 ,z 2 ), where the position information x, y, z satisfies the following affine transformation matrix:

[0071]

[0072] Among them, the coarse calibration parameters

[0073] Optionally, the first filtering module is specifically configured as follows:

[0074] The ground points in the roughly calibrated point cloud that satisfy the following formula are retained, and the non-ground points in the roughly calibrated point cloud that do not satisfy the following formula are filtered, where:

[0075] |P2(z2)-Z|<N

[0076] Among them, P2(z2) represents the position information of the point cloud after rough calibration in the direction of the Z coordinate axis, Z represents the maximum value of the ground points obtained by comparing the relevant key points in the real scene, wherein the relevant key points are points at non-point cloud edge positions, and N represents the difference between the maximum height of the radar from the real ground and the minimum height of the radar from the real ground.

[0077] Optionally, a fine calibration parameter determination module includes:

[0078] A candidate fitting plane determining unit is configured to select three points at random from a height range for plane fitting to obtain a candidate fitting plane;

[0079] a target fitting plane determining unit configured to determine, for any candidate fitting plane, the sum of distances from all points within a height range to the candidate fitting plane, and to take the candidate fitting plane corresponding to the minimum sum of distances as the target fitting plane;

[0080] The fine calibration parameter determination unit is configured to determine the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarse calibrated point cloud according to the rotation angle between the coarse calibration ground and the target fitting plane corresponding to the coarse calibrated point cloud.

[0081] Optionally, the fine calibration parameter determination unit is specifically configured as follows:

[0082] Get all sample frames of the original point cloud;

[0083] For each sample frame, the rotation angle of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud is determined according to the rotation angle between the coarsely calibrated ground and the target fitting plane corresponding to the coarsely calibrated point cloud;

[0084] For each coordinate axis, the average value of the rotation angles corresponding to the coordinate axis in all sample frames is used as the precise calibration parameter corresponding to the coordinate axis.

[0085] Optionally, the fine calibration module is specifically configured as follows:

[0086] Note θ x represents the X-axis precision calibration parameter, θ y represents the fine calibration parameter of the Y axis, θ zrepresents the fine calibration parameters of the Z axis, z′ represents the height information of the target fitting plane, and the affine transformation matrix added based on the above fine calibration parameters is:

[0087]

[0088] Accordingly, the point cloud after rough calibration is finely calibrated according to the following formula to obtain the calibrated point cloud:

[0089]

[0090] Among them, (x 2 ,y 2 ,z 2 ) represents the position information of the point cloud after rough calibration, (x 3 ,y 3 ,z 3 ) represents the position information of the calibrated point cloud.

[0091] In a third aspect, an embodiment of the present invention further provides a computing device, including:

[0092] A memory storing executable program code;

[0093] a processor coupled to the memory;

[0094] The processor calls the executable program code stored in the memory to execute the point cloud calibration method provided by any embodiment of the present invention.

[0095] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the point cloud calibration method provided by any embodiment of the present invention.

[0096] The technical solution provided by the embodiment of the present invention can perform a rough calibration of the coordinates of the original point cloud after obtaining the position information of the original point cloud by calculating the rough calibration parameters of the point cloud and adding an affine transformation matrix based on the rough calibration parameters, so that the coordinate axis of the point cloud ground after the rough calibration is parallel to the straight line where the scene marker is located in the horizontal direction, so that the radar can collect the point cloud information of all vehicles traveling on the lane, and when the vehicle is traveling in a straight line, the value of only one coordinate axis of the position of the center point of the vehicle is changing, so as to reduce the amount of calculation in the subsequent operation process. After the rough calibration is completed, the height range corresponding to the rough calibrated point cloud ground can be obtained by filtering the rough calibrated point cloud, and the plane fitting of the points within the height range can be performed according to the positional relationship between the target fitting plane obtained by fitting and the rough calibrated point cloud ground, and the fine calibration parameters of the point cloud can be determined, and the fine calibration parameters have higher accuracy and wider application range. Based on the fine calibration parameters, the affine transformation matrix can be added to the rough calibrated point cloud to achieve further correction of the rough calibrated point cloud ground, so that the coordinate system corresponding to the point cloud ground can be converted to a standard coordinate system. When point cloud data is subsequently processed, the point cloud data can be processed based on the same standard coordinate system. Compared with the point cloud ground calibration solution in the related art, the calibration method provided by the embodiment of the present invention avoids the point cloud ground misfitting situation, and can also avoid the influence of noise points generated by radar in rainy and foggy weather on the calibration, and the obtained calibration scene is more accurate.

[0097] The technical effects of the embodiments of the present invention include:

[0098] 1. By calculating the rough calibration parameters of the point cloud and adding the affine transformation matrix based on the rough calibration parameters, the coordinates of the position information of the original point cloud can be roughly calibrated, so that the ground coordinate axis of the point cloud after rough calibration is parallel to the straight line where the scene marker is located in the horizontal direction, so that the radar can collect the point cloud information of all vehicles traveling on the lane, and when the vehicle is traveling in a straight line, only the value of one coordinate axis of the position of the vehicle center point changes, so as to reduce the amount of calculation in the subsequent operation process. After completing the rough calibration of the original point cloud, the rough calibrated point cloud can be cropped and filtered in any direction to provide a basis for further fine calibration of the point cloud ground.

[0099] 2. When obtaining the ground points of the point cloud after rough calibration, the height range of the point cloud ground can be locked by using straight-through filtering to avoid obtaining noise points other than the ground points, and can effectively reduce the amount of calculation in the filtering process, thereby reducing the code running time and improving the calculation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0100] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0101] Figure 1a A flowchart of a point cloud calibration method provided in Embodiment 1 of the present invention;

[0102] Figure 1b A port point cloud scene image before rough calibration provided in the first embodiment of the present invention;

[0103] Figure 1c The port point cloud scene image after rough calibration provided in the first embodiment of the present invention;

[0104] Figure 1d A schematic diagram of the rotation of the point cloud ground coordinate system provided in the first embodiment;

[0105] Figure 1e A schematic diagram of a point cloud scene before calibration provided in the first embodiment of the present invention;

[0106] Figure 1f A schematic diagram of a calibrated point cloud scene provided in the first embodiment of the present invention;

[0107] Figure 2a A flowchart of a point cloud calibration method provided in Embodiment 2 of the present invention;

[0108] Figure 2b A schematic diagram of a point cloud scene after calibration and filtering provided in the second embodiment of the present invention;

[0109] Figure 3 A structural block diagram of a point cloud calibration device provided in Embodiment 3 of the present invention;

[0110] Figure 4 It is a structural diagram of a computing device provided in Embodiment 4 of the present invention. DETAILED DESCRIPTION

[0111] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0112] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device including a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products or devices.

[0113] The embodiments of the present invention disclose a method and device for calibrating a point cloud, which are described in detail below.

[0114] Embodiment 1

[0115] Figure 1a This is a flow chart of a point cloud calibration method provided in the first embodiment of the present invention. The method can be executed by a point cloud calibration device, which can be implemented by software and / or hardware. The method provided in this embodiment can be applied in complex environments such as ports, or can also be applied to vehicles driving on urban roads. Figure 1a As shown, the method provided in this embodiment includes:

[0116] S110, obtaining position information of the original point cloud from the radar.

[0117] Among them, the original point cloud is the point cloud data collected by the radar. After the radar is started, under the predetermined IP address (Internet Protocol Address), the current data packet can be intercepted by using the tcpdump (packet capture) command running on the Linux platform, and the point cloud position information at the current moment can be parsed from the data packet, and the obtained point cloud position information can be saved to ROS (Robot Operating System). Among them, the point cloud position information includes the point cloud position information (x, y, z) based on the original point cloud coordinate system. The current point cloud scene can be viewed through the ROS visualization module (rviz).

[0118] S120, rotating the original point cloud coordinate system, determining coarse calibration parameters according to the coordinate positions before and after the rotation, and adding an affine transformation matrix based on the coarse calibration parameters to perform coarse coordinate calibration on the position information.

[0119] In this embodiment, the original point cloud coordinate system is the radar coordinate system when the radar collects the frame point cloud, and the horizontal coordinate axis of the radar coordinate system is the X-axis or the Y-axis. The XOY plane corresponding to the horizontal coordinate axis is the point cloud ground. In this embodiment, the original point cloud coordinate system is rotated, and its rotated coordinate position is parallel to the straight line where the point cloud ground coordinate axis is located in the horizontal direction of the scene marker. Among them, the parallel condition can be set to an angle between the two that is less than or equal to 1°.

[0120] The scene markers represent objects in the real scene. The coordinate system corresponding to the scene markers is the coordinate system used to represent the real ground in the real scene, that is, the standard coordinate system referenced during point cloud ground correction. In this embodiment, it can be defined as (0,0,0). When selecting scene markers, objects on the lane can be used as scene markers. For example, the scene marker can be a road fence. For example, Figure 1b The port point cloud scene diagram before the rough calibration provided in the first embodiment of the present invention is as follows: Figure 1b As shown in the figure, the original point cloud ground radiates outward in an arc shape. Figure 1b The rectangular area A surrounded by two road fences a1 and the rectangular area B surrounded by two road fences b1 are both lanes, and the scene marker can be the road fence a1 or the road fence b1. In addition, in this embodiment, objects parallel to the lane direction can also be used as scene markers. In this embodiment, by setting the scene markers in the above manner, the ground coordinate axis of the point cloud after rough calibration can be parallel to the straight line where the scene markers are located in the horizontal direction, so that when the vehicle is driving in a straight line, the value of only one coordinate axis of the position of the vehicle center point changes, so as to reduce the amount of calculation in the subsequent operation process.

[0121] In this embodiment, the process of rotating the original point cloud coordinate system and determining the rough calibration parameters according to the coordinate positions before and after the rotation can be achieved in the following manner:

[0122] An arbitrary point is selected on the horizontal axis of the original point cloud coordinate system, which is not the origin of the coordinate system. The rotation angle is determined by the coordinate position of the point in the coordinate system before and after the rotation, and the rotation angle is used as a rough calibration parameter to add the affine transformation matrix.

[0123] In this embodiment, the coordinate information of the selected point before and after the rotation can be determined through the search (select) of the visualization module in ROS and the search function of cloudcompare (three-dimensional point cloud editing and processing software).

[0124] For example, Figure 1bAs shown, the angle between the horizontal coordinate axis X-axis in the coordinate system corresponding to the original point cloud scene graph and the straight line where the scene marker a1 is located is approximately 45°, and the angle between the Y-axis and the straight line where the scene marker b1 is located is approximately 45°. Take a point P on the original horizontal coordinate axis and rotate the coordinate system. When the angle between the straight line equation corresponding to the horizontal coordinate axis in the rotated coordinate system and the straight line equation corresponding to the scene marker meets the parallel condition, the rotation stops, and the rotation angle of point P when the parallel condition is met can be used as the horizontal rotation angle of the original point cloud. For example, Figure 1c The port point cloud scene diagram after rough calibration provided in the first embodiment of the present invention is as follows: Figure 1c As shown in FIG. 1 , when the angle between the rotated X-axis and the straight line where the scene annotation a1 is located is less than or equal to 1°, and the angle between the rotated Y-axis and the straight line where the scene annotation b1 is located is approximately 1°, the rotation stops. At this time, the rotation angle of point P that meets the parallel condition can be used as a rough calibration parameter of the original point cloud in the horizontal direction.

[0125] Figure 1d A schematic diagram of the rotation of the point cloud ground coordinate system provided in the first embodiment. Figure 1d Will Figure 1b and Figure 1c The coordinate systems in the image are separated and combined for display. Figure 1d As shown, the coordinates of point P in the original point cloud coordinate system before rotation are P 1 (x 1 ,y 1 ,z 1 ), the coordinate after the coordinate system is rotated is P 2 (x 2 ,y 2 ,z 2 ), where the position information of point P satisfies the following affine transformation matrix:

[0126]

[0127] Among them, θ xoy Represents the coarse calibration parameters.

[0128] Let cosθ xoy is a, sinθ xoy is b, according to the above matrix, we can get the following equations:

[0129]

[0130] Based on this, the rotation angle of the XOY plane around the Z axis can be obtained This angle value can be used as a rough calibration parameter of the point cloud in the horizontal direction.

[0131] After obtaining the rough calibration parameters, the coordinates of the original point cloud position information can be roughly calibrated by rotating all point clouds according to the above affine transformation matrix.

[0132] Through rough calibration, Figure 1d The X1-Y1-Z coordinate system in is transformed into the X2-Y2-Z coordinate system.

[0133] It can be understood by those skilled in the art that, generally, after the radar is installed, the coarse calibration parameters rx(θ yoz ), ry(θ xoz ) all meet the rough calibration conditions. At this time, we only need to calculate the rough calibration parameters corresponding to the XOY plane in the above way to perform rough calibration on the X and Y axes of the original point cloud coordinate system. However, the rough calibration parameters rx(θ yoz ) does not meet the coarse calibration conditions, or the coarse calibration parameter ry(θ xoz ) does not meet the coarse calibration conditions, or rx(θ yoz ) and ry(θ xoz ) do not meet the coarse calibration conditions, it is necessary to calculate the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane respectively, so as to perform coarse calibration on the Z coordinate axis of the original point cloud coordinate system, so that the coarse calibration parameters of the YOZ plane and the XOZ plane corresponding to the Z coordinate axis after the coarse calibration both meet the coarse calibration conditions. Among them, the coarse calibration conditions include that the Z coordinate axis direction of the original point cloud coordinate system is parallel to the vertical direction of the scene landmark. Among them, the X coordinate axis and the Y coordinate axis are the point cloud ground coordinate axes of the point cloud coordinate system, and the Z coordinate axis is perpendicular to the X coordinate axis and perpendicular to the Y coordinate axis. Among them, the calculation method of the coarse calibration parameters corresponding to the YOZ plane and the coarse calibration parameters corresponding to the XOZ plane is the same as the calculation method of the coarse calibration parameters corresponding to the XOY plane.

[0134] This embodiment can achieve preliminary calibration of the coordinate system of the point cloud ground by coarsely calibrating the original point cloud. The horizontal coordinate axis (X axis or Y axis) in the coordinate system obtained after coarse calibration is parallel to the lane line, so that when the vehicle is driving in a straight line, only the value of one coordinate axis of the position of the vehicle center point changes, so as to reduce the amount of calculation in the subsequent operation process. After completing the coarse calibration of the original point cloud, the coarsely calibrated point cloud can be cropped and filtered in any direction to provide a basis for further fine calibration of the point cloud ground.

[0135] S130 , filtering the roughly calibrated point cloud to obtain a height range corresponding to the ground of the roughly calibrated point cloud.

[0136] As an optional implementation, this embodiment may filter the coarsely calibrated point cloud by using morphological filtering, such as dilation and erosion, to obtain a coarsely calibrated point cloud ground corresponding to the coarsely calibrated point cloud.

[0137] As another optional implementation, a straight-through filter can be used to lock the height range of the ground of the point cloud to filter the non-ground points. This embodiment can avoid obtaining noise points other than ground points by adopting a straight-through filter, and can effectively reduce the amount of calculation in the filtering process, thereby reducing the code running time and improving the operation speed. Specifically, the height range of the ground points in the coarsely calibrated point cloud can be determined according to the following formula, that is, the ground points in the coarsely calibrated point cloud that meet the following formula are retained, and the non-ground points in the coarsely calibrated point cloud that do not meet the following formula are filtered:

[0138] |P2(z2)-Z|<N

[0139] Among them, P2(z2) represents the position information of the point cloud in the vertical direction, that is, in the direction of the Z coordinate axis, after rough calibration. Taking into account the inequality of the real ground, by selecting key points from the area where the non-edge points are located in the real scene, and comparing the height values ​​of the radar distance from each key point, the maximum value Z of the radar distance from the real ground can be obtained. Among them, each key point includes the point corresponding to the smooth road surface and the point corresponding to the highly uneven road surface in the real scene. In addition, by referring to the real scene of the multi-frame point cloud and taking into account the inequality of the real ground, the N value can be set to: the difference between the maximum height of the radar from the real ground and the minimum height of the radar from the real ground. This setting can retain all the ground points of the point cloud during the pass-through filtering process.

[0140] S140, performing plane fitting on points within a height range corresponding to the ground of the point cloud after the coarse calibration, and determining fine calibration parameters of the point cloud according to a positional relationship between a target fitting plane obtained by fitting and the ground of the point cloud after the coarse calibration.

[0141] There are many methods for plane fitting. For example, the least square method can be used for plane fitting, or the RANSAC (Random Sample Consensus) algorithm can be used for plane fitting. The following is a detailed introduction to the RANSAC-based plane fitting algorithm. The algorithm can be implemented through the following steps 1 to 2:

[0142] 1. Randomly select three points within the height range corresponding to the ground of the point cloud after rough calibration for plane fitting to obtain the candidate fitting plane.

[0143] In this embodiment, three points can be randomly selected within a region of M meters (M < N) from the height range N, denoted as P1′(x1′, y1′, z1′, i1′), P2′(x2′, y2′, z2′, i2′), and P3′(x3′, y3′, z3′, i′3). According to the plane equation ax + by + cz + d = 0, the following system of equations can be obtained:

[0144]

[0145] When solving this system of equations in this embodiment, compared with the method of solving the unknowns of the equation by solving the matrix in the related art, in this embodiment, by using Cramer's rule, when the matrix [a, b, c, d] is a non-singular matrix, it is shown that the matrix has a unique solution, and then the following can be obtained:

[0146] a = (y2′ - y1′)*(z3′ - z1′) - (z2′ - z1′)*(y3′ - y1′)

[0147] b = (z2′ - z1′)*(x3′ - x1′) - (x2′ - x1′)*(z3′ - z1′)

[0148] c = (x2′ - x1′)*(y3′ - y1′) - (y2′ - y1′)*(x3′ - x1′)

[0149] d = -(ax1′ + by1′ + cz1′)

[0150] Based on the values of a, b, c, and d above, the plane equation corresponding to the fitted plane can be obtained. By using Cramer's rule to solve the solution of the matrix in this embodiment, the operation speed is effectively improved.

[0151] 2. For each candidate fitted plane, determine the sum of the distances from all points within the height range to the fitted plane, and use the candidate fitted plane corresponding to the minimum sum of distances as the target fitted plane.

[0152] For the plane equation corresponding to the fitted plane, the distance d′ (d′ > 0) from any point within the region of height M to the fitted plane can be defined. Substituting any point into the plane equation can obtain the value of d′. Traverse all the points in the region in a loop until the fitted plane corresponding to the minimum sum of the distances from all points to a certain fitted plane is found, which is the target fitted plane.

[0153] Furthermore, after obtaining the target fitted plane, the height information of the target fitted plane and the angle information between the target fitted plane and the coarsely calibrated point cloud ground can be obtained.

[0154] In this embodiment, the rotation angle between the target fitting plane and the roughly calibrated point cloud ground actually refers to the rotation angle of the Z axis of the roughly calibrated point cloud coordinate system. Since the rotation of the point cloud is a rotation in three-dimensional space, when the roughly calibrated point cloud is rotated according to the rotation angle between the target fitting plane and the roughly calibrated point cloud ground, the YOZ plane corresponding to the roughly calibrated point cloud, that is, the X axis of the corresponding coordinate system, and the XOZ plane, that is, the Y axis of the corresponding coordinate system, will also be rotated accordingly. The rotation angle of the X axis and the rotation angle of the Y axis can be obtained through the rotation angle of the Z axis. Among them, the rotation angle corresponding to each coordinate axis can be used as the precise calibration parameter of each coordinate axis in the coordinate system corresponding to the roughly calibrated point cloud.

[0155] Furthermore, in order to make the determination of the fine calibration parameters more accurate, the present embodiment can obtain all sample frames of the original point cloud; for each sample frame, the rotation angle of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud is determined according to the rotation angle between the coarsely calibrated ground corresponding to the coarsely calibrated point cloud and the target fitting plane; for each coordinate axis, the average value of the rotation angle corresponding to the coordinate axis in all sample frames is used as the fine calibration parameter corresponding to the coordinate axis. By adopting the above setting method, the accuracy of the calibration parameters can be effectively improved, so that the calculation of the calibration parameters satisfies all samples.

[0156] S150 , adding an affine transformation matrix based on the fine calibration parameters, and performing fine calibration on the point cloud after the coarse calibration to obtain a calibrated point cloud.

[0157] In this embodiment, θ x represents the X-axis precision calibration parameter, θ y represents the fine calibration parameter of the Y axis, θ z represents the fine calibration parameters of the Z axis, z′ represents the height information of the target fitting plane, and the affine transformation matrix added based on the above fine calibration parameters is:

[0158]

[0159] In this embodiment, the affine transformation matrix can be used to simultaneously perform rotation and translation transformations on the roughly calibrated point cloud. Compared with the method of first completing the rotation transformation and then performing the translation transformation, this configuration of the embodiment of the present invention can simplify the calculation and improve the calculation efficiency.

[0160] Accordingly, in this embodiment, fine calibration is performed on the roughly calibrated point cloud to obtain a calibrated point cloud, including:

[0161] According to the following formula, the point cloud after rough calibration is finely calibrated to obtain the calibrated point cloud:

[0162]

[0163] Among them, (x 2 ,y 2 ,z 2 ) represents the position information of the point cloud after rough calibration, (x 3 ,y 3 ,z 3 ) represents the position information of the calibrated point cloud.

[0164] Specifically, all points in the roughly calibrated point cloud are rotated and translated according to the above affine transformation matrix to obtain a calibrated point cloud set. Figure 1e This is a schematic diagram of a point cloud scene before calibration provided in the first embodiment of the present invention. Figure 1f Schematic diagram of a calibrated point cloud scene provided in the first embodiment of the present invention. Figure 1e and 1f As shown in FIG. 1 , the height of the origin O of the point cloud coordinate system has changed before and after calibration, that is, the height information of the point cloud ground is corrected to the coordinate origin. The Y axis of the horizontal coordinate system corresponding to the point cloud coordinate system is also close to being parallel to the straight line where the scene marker b1 is located.

[0165] The technical solution provided in this embodiment, after obtaining the position information of the original point cloud, calculates the rough calibration parameters of the point cloud, and adds an affine transformation matrix based on the rough calibration parameters to perform a rough calibration of the coordinates of the position information of the original point cloud, so that the ground coordinate axis of the point cloud after the rough calibration is parallel to the straight line where the scene marker is located in the horizontal direction, so that the radar can collect the point cloud information of all vehicles traveling on the lane, and when the vehicle is traveling in a straight line, the value of only one coordinate axis of the position of the center point of the vehicle is changing, so as to reduce the amount of calculation in the subsequent operation process. After completing the rough calibration, the height range corresponding to the ground of the point cloud after the rough calibration can be obtained by filtering the point cloud after the rough calibration, and the plane fitting of the points within the height range can be performed, and the fine calibration parameters of the point cloud can be determined according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration, and the fine calibration parameters have higher accuracy and wider application range. Based on the fine calibration parameters, the affine transformation matrix can be added to the point cloud after the rough calibration to achieve further correction of the ground of the point cloud after the rough calibration, so that the coordinate system corresponding to the ground of the point cloud can be converted to a standard coordinate system. When point cloud data is subsequently processed, the point cloud data can be processed based on the same standard coordinate system. Compared with the point cloud ground calibration solution in the related art, the calibration method provided by the embodiment of the present invention avoids the point cloud ground misfitting situation, and can also avoid the influence of noise points generated by radar in rainy and foggy weather on the calibration, and the obtained calibration scene is more accurate.

[0166] Embodiment 2

[0167] Figure 2aThis is a flow chart of a point cloud calibration method provided in the second embodiment of the present invention. Based on the above embodiment, this embodiment refines the application of the calibrated point cloud, such as Figure 2a As shown, the method provided in this embodiment includes:

[0168] S210: Obtain location information of the original point cloud from the radar.

[0169] S220, rotating the original point cloud coordinate system, determining coarse calibration parameters according to the coordinate positions before and after the rotation, and adding an affine transformation matrix based on the coarse calibration parameters to perform coarse coordinate calibration on the position information.

[0170] Among them, the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene.

[0171] S230 , filtering the roughly calibrated point cloud to obtain a height range corresponding to the ground of the roughly calibrated point cloud.

[0172] S240, performing plane fitting on points within a height range corresponding to the ground of the point cloud after the coarse calibration, and determining fine calibration parameters of the point cloud according to a positional relationship between a target fitting plane obtained by fitting and the ground of the point cloud after the coarse calibration.

[0173] S250, adding an affine transformation matrix based on the fine calibration parameters, and performing fine calibration on the point cloud after the coarse calibration to obtain a calibrated point cloud.

[0174] The specific implementation of steps S210 to S250 may refer to the description of the above embodiment and will not be repeated here.

[0175] S260, performing ground filtering on the calibrated point cloud.

[0176] There are many ways to filter the point cloud ground, such as morphological filtering, such as dilation and erosion, etc., or through-filtering can be used to lock the height range of the point cloud ground, and then filter the point cloud ground within the height range. This embodiment uses through-filtering to effectively reduce the amount of calculation in the filtering process, thereby reducing the code running time and improving the computing speed. Specifically, the point cloud ground in the target point cloud can be determined according to the following formula, wherein the point cloud that satisfies the following formula is a point in the point cloud ground and needs to be deleted, and the point cloud that does not satisfy the following formula is a point in the non-point cloud ground and needs to be retained.

[0177] |P3(z3)-Z″|<M

[0178] Among them, P3(z3) represents the position information of the point cloud in the vertical direction after fine calibration, that is, in the Z coordinate axis direction. Z″ is the height value of the calibrated point cloud ground in the Z coordinate axis direction, and M is the height range defined when performing plane fitting during the fine calibration process, that is, M represents the height range of M meters above and below the point cloud ground after calibration.

[0179] S270, inputting the point cloud data after the ground is filtered out into the neural network model, so as to extract feature information of the point cloud data after the ground is filtered out through the neural network model.

[0180] The neural network model may be a target detection model such as PointNet, PointNet++, etc., which is not specifically limited in this embodiment. The target detection model can be used to obtain feature point information in the filtered point cloud data, such as vehicle information and road information where the vehicle is located. Figure 2b This is a schematic diagram of a point cloud scene after calibration and filtering provided in the second embodiment of the present invention. Figure 2b As shown in Figure 1, by filtering the calibrated point cloud ground, the background points of the point cloud ground are filtered out, that is, the background points such as Figure 1e and 1f The point cloud ground points radiating outward in an arc shape retain the point cloud information of lane A and lane B. By inputting the filtered point cloud information into the target detection model, the accuracy of 3D object feature point detection can be improved.

[0181] In this embodiment, feature point information in the target point cloud can be obtained by performing target detection on the target point cloud. Before target detection, the ground points of the point cloud are filtered, which can effectively reduce the proportion of ground points and increase the sampling probability of feature points, thereby improving the accuracy of object detection.

[0182] Embodiment 3

[0183] Figure 3 A structural block diagram of a point cloud calibration device provided in Embodiment 3 of the present invention is shown in FIG. Figure 3 As shown, the device includes: a position information acquisition module 310, a coarse calibration module 320, a first filtering module 330, a fine calibration parameter determination module 340 and a fine calibration module 350, wherein:

[0184] The position information acquisition module 310 is configured to acquire the position information of the original point cloud from the radar;

[0185] The coarse calibration module 320 is configured to rotate the original point cloud coordinate system, determine the coarse calibration parameters according to the coordinate positions before and after the rotation, and add the affine transformation matrix based on the coarse calibration parameters to perform a coarse coordinate calibration on the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene;

[0186] The first filtering module 330 is configured to filter the roughly calibrated point cloud to obtain a height range corresponding to the ground of the roughly calibrated point cloud;

[0187] The fine calibration parameter determination module 340 is configured to perform plane fitting on the points within the height range, and determine the fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration;

[0188] The fine calibration module 350 is configured to add an affine transformation matrix based on the fine calibration parameters, and perform fine calibration on the point cloud after the coarse calibration to obtain a calibrated point cloud.

[0189] Optionally, the device provided by the embodiment of the present invention further includes:

[0190] A second filtering module is configured to perform ground filtering on the calibrated point cloud;

[0191] The feature extraction module is configured to input the point cloud data after the ground is filtered out into the neural network model, so as to extract feature information of the point cloud data after the ground is filtered out through the neural network model.

[0192] Optionally, the second filtering module is specifically configured as follows:

[0193] The calibrated point cloud ground points that satisfy the following formula are deleted, and the calibrated non-point cloud ground points that do not satisfy the following formula are retained:

[0194] |P3(z3)-Z″|<M

[0195] Among them, P3(z3) represents the position information of the point cloud in the Z coordinate axis direction after precise calibration, Z″ is the height value of the point cloud ground in the Z coordinate axis direction after calibration, and M represents the height range of M meters above and below the point cloud ground after calibration.

[0196] Optionally, the coarse calibration module is specifically configured as follows:

[0197] Rotate the original point cloud coordinate system and determine the rough calibration parameters according to the coordinate positions before and after the rotation;

[0198] If the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane of the original point cloud coordinate system meet the coarse calibration conditions, the coarse calibration parameters corresponding to the XOY plane are calculated, and the affine transformation matrix is ​​added based on the coarse calibration parameters to perform coarse calibration on the X-axis and Y-axis of the original point cloud coordinate system, wherein the coarse calibration conditions include that the Z-axis direction of the original point cloud coordinate system is parallel to the vertical direction of the scene landmark, the X-axis and the Y-axis are the point cloud ground coordinate axes of the point cloud coordinate system, and the Z-axis is perpendicular to the X-axis and to the Y-axis.

[0199] Optionally, the device provided by the embodiment of the present invention further includes:

[0200] The pre-calibration module is configured to perform a rough calibration on the Z coordinate axis of the original point cloud coordinate system by respectively calculating the rough calibration parameters corresponding to the YOZ plane and the XOZ plane when the rough calibration parameters corresponding to the YOZ plane and / or the rough calibration parameters corresponding to the XOZ plane of the original point cloud coordinate system do not meet the rough calibration conditions, so that the rough calibration parameters of the YOZ plane and the XOZ plane corresponding to the Z coordinate axis after the rough calibration both meet the rough calibration conditions,

[0201] The calculation method of the coarse calibration parameters corresponding to the YOZ plane and the coarse calibration parameters corresponding to the XOZ plane is the same as the calculation method of the coarse calibration parameters corresponding to the XOY plane.

[0202] Optionally, the coarse calibration module 320 is specifically configured as follows:

[0203] All original point clouds are rotated according to the following formula to roughly calibrate the coordinates of the position information:

[0204] The original point cloud coordinates are recorded as P 1 (x 1 ,y 1 ,z 1 ), after the original point cloud coordinate system is rotated, the point cloud P 1 The corresponding coordinate position of the rotated point cloud is P 2 (x 2 ,y 2 ,z 2 ), where the position information x, y, z satisfies the following affine transformation matrix:

[0205]

[0206] Among them, the coarse calibration parameters

[0207] Optionally, the first filtering module 330 is specifically configured as follows:

[0208] The ground points in the roughly calibrated point cloud that satisfy the following formula are retained, and the non-ground points in the roughly calibrated point cloud that do not satisfy the following formula are filtered, where:

[0209] |P2(z2)-Z|<N

[0210] Among them, P2(z2) represents the position information of the point cloud after rough calibration in the direction of the Z coordinate axis, Z represents the maximum value of the ground points obtained by comparing the relevant key points in the real scene, wherein the relevant key points are points at non-point cloud edge positions, and N represents the difference between the maximum height of the radar from the real ground and the minimum height of the radar from the real ground.

[0211] Optionally, the fine calibration parameter determination module 340 includes:

[0212] A candidate fitting plane determining unit is configured to select three points at random from a height range for plane fitting to obtain a candidate fitting plane;

[0213] a target fitting plane determining unit configured to determine, for any candidate fitting plane, the sum of distances from all points within a height range to the candidate fitting plane, and to take the candidate fitting plane corresponding to the minimum sum of distances as the target fitting plane;

[0214] The fine calibration parameter determination unit is configured to determine the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarse calibrated point cloud according to the rotation angle between the coarse calibration ground and the target fitting plane corresponding to the coarse calibrated point cloud.

[0215] Optionally, the fine calibration parameter determination unit is specifically configured as follows:

[0216] Get all sample frames of the original point cloud;

[0217] For each sample frame, the rotation angle of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud is determined according to the rotation angle between the coarsely calibrated ground and the target fitting plane corresponding to the coarsely calibrated point cloud;

[0218] For each coordinate axis, the average value of the rotation angles corresponding to the coordinate axis in all sample frames is used as the precise calibration parameter corresponding to the coordinate axis.

[0219] Optionally, the fine calibration module 350 is specifically configured as follows:

[0220] Note θ x represents the X-axis precision calibration parameter, θ y represents the fine calibration parameter of the Y axis, θ z represents the fine calibration parameters of the Z axis, z′ represents the height information of the target fitting plane, and the affine transformation matrix added based on the above fine calibration parameters is:

[0221]

[0222] Accordingly, the point cloud after rough calibration is finely calibrated according to the following formula to obtain the calibrated point cloud:

[0223]

[0224] Among them, (x 2 ,y 2 ,z 2 ) represents the position information of the point cloud after rough calibration, (x 3 ,y 3 ,z 3 ) represents the position information of the calibrated point cloud.

[0225] The point cloud calibration device provided in the embodiment of the present invention can execute the point cloud calibration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not described in detail in the above embodiment, please refer to the point cloud calibration method provided in any embodiment of the present invention.

[0226] Embodiment 4

[0227] See also Figure 4 , Figure 4 Schematic diagram of a computing device provided by Embodiment 4 of the present invention. Figure 4 As shown, the computing device may include:

[0228] A memory 701 storing executable program codes;

[0229] a processor 702 coupled to the memory 701;

[0230] The processor 702 calls the executable program code stored in the memory 701 to execute the point cloud calibration method provided by any embodiment of the present invention.

[0231] An embodiment of the present invention discloses a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the point cloud calibration method provided by any embodiment of the present invention.

[0232] In various embodiments of the present invention, it should be understood that the size of the serial numbers of the above-mentioned processes does not mean the necessary order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0233] In the embodiments provided by the present invention, it should be understood that "B corresponding to A" means that B is associated with A, and B can be determined according to A. However, it should also be understood that determining B according to A does not mean determining B only according to A, and B can also be determined according to A and / or other information.

[0234] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0235] If the above integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-accessible memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several requests for a computer device (which can be a personal computer, a server or a network device, etc., specifically a processor in a computer device) to perform some or all of the steps of the above methods of various embodiments of the present invention.

[0236] Those skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a random access memory (RAM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), a one-time programmable read-only memory (OTPROM), an electronically erasable rewritable read-only memory (EEPROM), a compact disc (CD-ROM) or other optical disc storage, magnetic disk storage, magnetic tape storage, or any other computer-readable medium that can be used to carry or store data.

[0237] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0238] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0239] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A point cloud calibration method, characterized in that: include: Get the location information of the original point cloud from the radar; The original point cloud coordinate system is rotated, the coarse calibration parameters are determined according to the coordinate positions before and after the rotation, and the affine transformation matrix is ​​added based on the coarse calibration parameters to perform coarse coordinate calibration on the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is approximately parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene; Filter the roughly calibrated point cloud to obtain the height range of the roughly calibrated point cloud ground; Performing plane fitting on the points within the height range, and determining fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after rough calibration; An affine transformation matrix is ​​added based on the fine calibration parameters, and fine calibration is performed on the roughly calibrated point cloud to obtain a calibrated point cloud.

2. The method according to claim 1, characterized in that The method further comprises: Performing ground filtering on the calibrated point cloud; The point cloud data after the ground is filtered out is input into the neural network model, so as to extract feature information of the point cloud data after the ground is filtered out through the neural network model.

3. The method according to claim 2, characterized in that The performing ground filtering on the calibrated point cloud comprises: The calibrated point cloud ground points that satisfy the following formula are deleted, and the calibrated non-point cloud ground points that do not satisfy the following formula are retained: |P3(z3)-Z″|<M Among them, P3(z3) represents the position information of the point cloud in the Z coordinate axis direction after precise calibration, Z″ is the height value of the point cloud ground in the Z coordinate axis direction after calibration, and M represents the height range of M meters above and below the point cloud ground after calibration.

4. The method according to claim 1, characterized in that: The performing rough coordinate calibration on the position information includes: If the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane of the original point cloud coordinate system all meet the coarse calibration conditions, the coarse calibration parameters corresponding to the XOY plane are calculated to coarsely calibrate the X coordinate axis and the Y coordinate axis of the original point cloud coordinate system, wherein the coarse calibration conditions include that the Z coordinate axis direction of the original point cloud coordinate system is parallel to the vertical direction of the scene marker, the X coordinate axis and the Y coordinate axis are the point cloud ground coordinate axes of the point cloud coordinate system, and the Z coordinate axis is perpendicular to the X coordinate axis and perpendicular to the Y coordinate axis; When the coarse calibration parameters corresponding to the YOZ plane and / or the coarse calibration parameters corresponding to the XOZ plane of the original point cloud coordinate system do not satisfy the coarse calibration conditions, the Z coordinate axis of the original point cloud coordinate system is coarsely calibrated by respectively calculating the coarse calibration parameters corresponding to the YOZ plane and the XOZ plane, so that the coarse calibration parameters of the YOZ plane and the XOZ plane corresponding to the Z coordinate axis after the coarse calibration both satisfy the coarse calibration conditions. The calculation method of the coarse calibration parameters corresponding to the YOZ plane and the coarse calibration parameters corresponding to the XOZ plane is the same as the calculation method of the coarse calibration parameters corresponding to the XOY plane.

5. The method according to any one of claims 1 to 4, characterized in that: The step of determining a rough calibration parameter according to the coordinate positions before and after the rotation, and adding an affine transformation matrix based on the rough calibration parameter to perform a rough coordinate calibration on the position information includes: All original point clouds are rotated according to the following formula to roughly calibrate the coordinates of the position information: The coordinates of the original point cloud are recorded as P1 (x1, y1, z1). After the original point cloud coordinate system is rotated, the coordinate position of the rotated point cloud corresponding to point cloud P1 is P2 (x2, y2, z2), where the position information x, y, z satisfies the following affine transformation matrix: Among them, the coarse calibration parameters 6. The method according to claim 1, characterized in that The filtering of the roughly calibrated point cloud comprises: The ground points in the roughly calibrated point cloud that satisfy the following formula are retained, and the non-ground points in the roughly calibrated point cloud that do not satisfy the following formula are filtered, where: |P2(z2)-Z|<N Among them, P2(z2) represents the position information of the point cloud after rough calibration in the direction of the Z coordinate axis, Z represents the maximum value of the ground points obtained by comparing the relevant key points in the real scene, wherein the relevant key points are points at non-point cloud edge positions, and N represents the difference between the maximum height of the radar from the real ground and the minimum height of the radar from the real ground.

7. The method according to claim 1, characterized in that The performing plane fitting on the points within the height range and determining the fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration, includes: Randomly select three points from the height range to perform plane fitting to obtain a candidate fitting plane; For any candidate fitting plane, determine the sum of the distances from all points within the height range to the candidate fitting plane, and take the candidate fitting plane corresponding to the minimum sum of the distances as the target fitting plane; According to the rotation angle between the coarsely calibrated ground corresponding to the coarsely calibrated point cloud and the target fitting plane, the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud are determined.

8. The method according to claim 7, characterized in that The step of determining the fine calibration parameters of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud according to the rotation angle between the coarsely calibrated ground surface corresponding to the coarsely calibrated point cloud and the target fitting plane comprises: Get all sample frames of the original point cloud; For each sample frame, according to the rotation angle between the coarsely calibrated ground corresponding to the coarsely calibrated point cloud and the target fitting plane, the rotation angle of each coordinate axis in the coordinate system corresponding to the coarsely calibrated point cloud is determined; For each coordinate axis, the average value of the rotation angles corresponding to the coordinate axis in all sample frames is used as the precise calibration parameter corresponding to the coordinate axis.

9. The method according to claim 1, characterized in that: The adding an affine transformation matrix based on the fine calibration parameters comprises: Note θ x represents the X-axis precision calibration parameter, θ y represents the fine calibration parameter of the Y axis, θ z represents the fine calibration parameters of the Z axis, z′ represents the height information of the target fitting plane, and the affine transformation matrix added based on the above fine calibration parameters is: Accordingly, the step of finely calibrating the roughly calibrated point cloud to obtain a calibrated point cloud includes: The roughly calibrated point cloud is finely calibrated according to the following formula to obtain the calibrated point cloud: Among them, (x2, y2, z2) represents the position information of the point cloud after the rough calibration, and (x3, y3, z3) represents the position information of the point cloud after calibration.

10. A point cloud calibration device, characterized in that: include: A position information acquisition module is configured to acquire position information of an original point cloud from a radar; A coarse calibration module is configured to rotate the original point cloud coordinate system, determine coarse calibration parameters according to the coordinate positions before and after the rotation, and add an affine transformation matrix based on the coarse calibration parameters to perform coarse coordinate calibration on the position information, wherein the ground coordinate axis of the point cloud after the rotation of the original point cloud coordinate system is approximately parallel to the straight line where the scene marker is located in the horizontal direction, and the scene marker is an object parallel to the lane direction in the real scene; A first filtering module is configured to filter the roughly calibrated point cloud to obtain a height range corresponding to the ground of the roughly calibrated point cloud; A fine calibration parameter determination module is configured to perform plane fitting on the points within the height range, and determine fine calibration parameters of the point cloud according to the positional relationship between the target fitting plane obtained by fitting and the ground of the point cloud after the rough calibration; The fine calibration module is configured to add an affine transformation matrix based on the fine calibration parameters, perform fine calibration on the coarsely calibrated point cloud, and obtain a calibrated point cloud.

Images