Point cloud target detection method based on non-uniform cylinder prime coding in foggy day scene
By using the non-uniform cylindrical pixel encoding method to process point cloud data in foggy scenes, the problem of point cloud data damage in foggy days is solved, the detection robustness and adaptability are improved, and the environment of smart vehicles is enhanced Perception ability.
Patent Information
- Application Number
- CN202411275115.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-09-12
AI Technical Summary
Lidar point cloud data is affected by extinction and scattering in foggy weather conditions, resulting in a decrease in perception accuracy and an increase in false point clouds. It is difficult for the existing technology to effectively deal with, especially the problem of increased computing power and time requirements in real-time detection.
Using the non-uniform cylindrical pixel encoding method based on foggy scenes, the foggy cloud data is simulated and encoded on sunny point cloud data to generate a highly adaptable point cloud voxel data set, and the cylindrical coordinate system and preset extinction coefficient are used to perform the volume. , adaptively adjust voxel volume to improve robustness.
It improves the robustness and adaptability of lidar point cloud detection in foggy conditions, reduces false point clouds, enhances the environmental perception capabilities of smart vehicles, and reduces computing resource requirements.
Smart Images

Figure CN119064892A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to a laser radar point cloud data processing method in the field of environmental perception technology, and specifically is a point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes. Background Art
[0002] In foggy weather, the light pulses emitted by the LiDAR will be affected by two main factors when passing through water molecules in the air, namely extinction and backscattering. Extinction refers to the large attenuation of the energy of the light pulse during propagation, resulting in an overall decrease in the power of the received signal. At the same time, the emitted light signals of objects within the measurement range cannot be received, and the effective sensing range of the LiDAR is reduced; backscattering refers to the light pulse generating a received signal peak within an incorrect range, resulting in a large number of incorrect false points in the generated LiDAR point cloud.
[0003] Compared with the original point cloud, voxels show better stability in the face of data damage, and the size of voxels has a decisive influence on the robustness of voxels. For example, larger voxels are more stable to damage caused by motion blur, while smaller voxels are more adaptable to data damage caused by fog, snow, and crosstalk. Therefore, in order to deal with the damage caused to point cloud data by foggy weather, it is necessary to divide the voxels as small as possible. However, in order to ensure that the detection range is sufficient, the reduction in voxel volume will inevitably lead to an increase in the number of voxels, resulting in a corresponding increase in the computing power and time required for the deep learning network to process voxels, which is not conducive to real-time detection under foggy conditions. Therefore, it is necessary to design a special voxel encoding method to adapt to the impact of foggy weather on lidar point clouds. Summary of the invention
[0004] The present invention aims to address the deficiencies of the above-mentioned prior art and proposes a point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes, so that the higher the concentration of fog, the smaller the volume of the cylindrical voxels divided in the near distance, thereby being able to adaptively adjust the algorithm's adaptability to fog and improve detection robustness.
[0005] In order to achieve the above-mentioned purpose, the present invention adopts the following technical scheme:
[0006] The point cloud target detection method based on non-uniform cylindrical voxel coding in foggy scenes of the present invention is characterized in that it comprises the following steps:
[0007] A. After using LiDAR to collect point cloud data of the target scene in a sunny scene and sampling, the dimension is Sunny sky point cloud dataset ;in, Represents a sunny point cloud dataset The i-th clear sky point cloud in The coordinates of , Represents a sunny point cloud dataset The number of point clouds, W represents the i-th sunny point cloud The number of messages;
[0008] B. Using the foggy weather simulation network to simulate the sunny point cloud dataset Processing is performed to generate dimensions of Foggy point cloud dataset ;in, Represents a foggy point cloud dataset The A foggy cloud. The coordinates of ;
[0009] C. Foggy point cloud dataset Perform non-uniform cylindrical voxel encoding to obtain the encoded point cloud voxel dataset in the foggy scene ;
[0010] C1. Establish a cylindrical coordinate system with the laser radar as the origin, the vehicle's forward direction as the positive direction of the X axis, the direction perpendicular to the ground and upward as the positive direction of the Z axis, and the direction pointing to the left of the driver as the positive direction of the Y axis. Use formula (1) to convert the Cartesian coordinate system into Coordinates Perform coordinate transformation to obtain the cylindrical coordinate system Coordinate encoding :
[0011] (1)
[0012] In formula (1), express The Euclidean distance from the projection on the XY plane in the Cartesian coordinate system to the origin of the coordinate system in the cylindrical coordinate system, express The angle between the projection on the XY plane in the Cartesian coordinate system to the origin of the cylindrical coordinate system and the positive direction of the X axis in the Cartesian coordinate system. express The coordinate of the Z axis in the Cartesian coordinate system;
[0013] C2. In the cylindrical coordinate system, select On the axis within the range and Axis unit step is the division angle of voxels, On the axis In the range, Axis unit step is the height of the voxel division, and the foggy point cloud dataset after preliminary division is obtained. On the axis Within the range, the foggy point cloud dataset after preliminary division is divided to obtain the final foggy point cloud dataset ;in, and , and , and They represent the coordinates of the foggy point cloud in the cylindrical coordinate system. axis, axis, Minimum and maximum values on the axis;
[0014] C3. Using the preset extinction coefficient , and the maximum sensing distance of the laser radar is obtained by formula (2) :
[0015] (2)
[0016] In formula (2), n is the detectable background noise, g is the adaptive laser gain of the lidar, is the intensity of the lidar point cloud under clean weather conditions; where n, g, and is the intrinsic parameter of the laser radar;
[0017] C4. Maximum sensing distance using LiDAR ,Will exist The length of the axis is set in sections, so that exist The voxels in the y-axis are evenly divided into S parts. exist arrive The voxels in the foggy scene are evenly divided into T parts, so as to obtain the encoded point cloud voxel dataset in the foggy scene. , where S>T;
[0018] D. Input into the neural network for feature extraction, and then output the final point cloud recognition result.
[0019] The point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes according to the present invention is also characterized in that step B comprises:
[0020] B1. According to the spatial pulse response of the laser radar, the targets hit by the laser are divided into hard targets and soft targets; wherein the hard targets refer to the time characteristics of the laser radar that do not change when hitting the target and reflecting; the soft targets refer to the fog molecules that cause distributed scattering of the light pulse;
[0021] B2. When the laser radar reaches Evenly increase point cloud, thus obtaining a dimension of Point cloud dataset , and calculated by formula (3) Middle Point Cloud Distance to LiDAR ;in, express The number of point clouds in That is Middle Sunny day with clouds :
[0022] (3)
[0023] In formula (3), for The coordinates of
[0024] B3, obtained by formula (4) Middle Point Cloud The spatial impulse response of And as the spatial impulse response of the hard target, it is obtained by formula (5) The spatial impulse response of And as the spatial impulse response of the kth soft target:
[0025] (4)
[0026] (5)
[0027] In formula (4) and formula (5), represents the scattering coefficient of the hard target, represents the scattering coefficient of the soft target, is the Dirac function, is a unit step function;
[0028] B4. Calculate the signal receiving power of the hard target using equations (6) and (7) respectively and the signal receiving power of the kth soft target :
[0029] (6)
[0030] (7)
[0031] In formula (6) and formula (7), is the half-power pulse width of the laser radar, represents the peak pulse power of the laser radar, c is the speed of light, t is the time, is the system constant of the laser radar, represents the crossover function;
[0032] B5. Calculate the maximum signal receiving power of the soft target ,in, Indicates the point cloud corresponding to the maximum signal received power Distance to the LiDAR;
[0033] like , then the point cloud corresponding to the hard target is retained As the i-th foggy point cloud Otherwise, As the i-th foggy point cloud ;
[0034] B6. After assigning i+1 to i, return to step B2 and execute sequentially to finally generate a foggy point cloud dataset. .
[0035] An electronic device of the present invention includes a memory and a processor, wherein the memory is used to store a target detection program that supports the processor to execute the non-uniform cylindrical voxel encoding of the laser radar point cloud, and the processor is configured to execute the program stored in the memory.
[0036] The present invention provides a computer-readable storage medium, wherein a computer program is stored on the computer-readable storage medium, and the computer program, when executed by a processor, executes the target detection step of the non-uniform cylindrical voxel encoding of the laser radar point cloud.
[0037] Compared with the prior art, the present invention has the following beneficial effects:
[0038] 1. In view of the problem that the laser radar perception accuracy is greatly reduced under foggy conditions, the present invention proposes a point cloud target detection method based on non-uniform cylindrical voxel coding in foggy scenes. The voxels of the laser radar point cloud show better stability in the face of data damage, and non-uniform cylindrical voxel coding is used when dividing the point cloud data into voxels. The higher the concentration of fog, the smaller the volume of the cylindrical voxels divided by the method of the present invention is in the near distance. Therefore, the adaptability of the algorithm to fog can be adaptively adjusted, which is beneficial to improving the environmental perception ability of intelligent vehicles.
[0039] 2. The present invention proposes a strategy for generating a point cloud dataset in foggy weather. Compared with the previous foggy weather dataset, the method of the present invention first collects a lidar point cloud dataset under clear weather, then establishes a physical model of lidar point cloud under foggy weather, batch generates fog that may exist in foggy weather scenes, and finally uses a point cloud fog simulation network to establish a point cloud dataset under foggy weather. This solves the problems of excessive cost, limited data volume and low generalization of real datasets, improves the adaptability of target detection algorithms to adverse weather environments, and is conducive to improving the environmental perception capabilities of intelligent vehicles. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is an overall flow chart of the point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes of the present invention;
[0041] Figure 2 This is a flow chart of the point cloud fog simulation of the present invention;
[0042] Figure 3 This is a structural diagram of the point cloud non-uniform cylindrical voxel encoding of the present invention. DETAILED DESCRIPTION
[0043] In this embodiment, a point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes is provided. Figure 1 As shown, the following steps are included:
[0044] A. After using LiDAR to collect point cloud data of the target scene in a sunny scene and sampling, the sampling process includes: recording the number of point clouds collected by LiDAR as , and specify the number of point clouds after sampling ,like , then the point cloud collected by the lidar is randomly downsampled until the number of point clouds is Otherwise, the point cloud collected by the lidar is randomly copied until the number of point clouds reaches , and the dimension is Sunny sky point cloud dataset ;in, Represents a sunny point cloud dataset The i-th clear sky point cloud in The coordinates of , Representing a point cloud dataset The number of sunny point clouds, W represents the i-th sunny point cloud The number of messages.
[0045] B. Using the foggy weather simulation network to simulate the sunny point cloud dataset Processing is performed to generate dimensions of Foggy point cloud dataset ;in, Represents a foggy point cloud dataset The A foggy cloud. The coordinates of ; Point cloud fog simulation process is as follows Figure 2 As shown;
[0046] B1. According to the spatial pulse response of the laser radar, the targets hit by the laser are divided into hard targets and soft targets. Among them, hard targets refer to the time characteristics of the laser radar that do not change when hitting the target and reflecting; soft targets refer to fog molecules that cause distributed scattering of light pulses. Among them, the sunny point cloud dataset The point cloud in the figure corresponds to the hard target, while the soft target simulates the possible fog points.
[0047] B2. When the laser radar reaches Evenly increase point cloud, thus obtaining a dimension of Point cloud dataset , and calculated by formula (3) Middle Point Cloud Distance to LiDAR ;in, express The number of point clouds in That is Middle Sunny day with clouds :
[0048] (3)
[0049] In formula (3), for The coordinates of the point cloud dataset The points in the data set are arranged in order. The first point is the point closest to the LiDAR, and then add point clouds at the same interval, so the point cloud dataset In That is Middle Sunny day with clouds .
[0050] B3, obtained by formula (4) Middle Point Cloud The spatial impulse response of And as the spatial impulse response of the hard target, it is obtained by formula (5) The spatial impulse response of And as the spatial impulse response of the kth soft target:
[0051] (4)
[0052] (5)
[0053] In formula (4) and formula (5), represents the scattering coefficient of the hard target, represents the scattering coefficient of the soft target, is the Dirac function, is a unit step function.
[0054] B4. According to the theory of electronic engineering and signal processing, the output of any linear system can be expressed as the convolution of the input signal and the system function (system impulse response). Therefore, assuming that the propagation of light in fog is inelastic scattering, that is, all reflected light is not reflected by other fog molecules, and no energy is converted into other wavelengths, then the received signal power of the lidar is , use equations (6) and (7) to calculate the signal receiving power of the hard target respectively and the signal receiving power of the kth soft target :
[0055] (6)
[0056] (7)
[0057] In formula (6) and formula (7), is the half-power pulse width of the laser radar, represents the peak pulse power of the laser radar, c is the speed of light, t is the time, is the system constant of the laser radar, and its value is ,in, is the loss in the receiver optics, represents the sensor optical aperture area, Represents the cross function, describing the area illuminated by the emitter The area observed by the receiver (sensor optical aperture area), whose value is
[0058]
[0059] B5. Calculate the maximum signal receiving power of the soft target ,in, Indicates the point cloud corresponding to the maximum signal received power Distance to the LiDAR.
[0060] In an environment where both hard and soft targets exist, the LiDAR signal receiving power of the hard target is usually greater than that of the soft target. At this time, the LiDAR can detect the object normally, but it will attenuate as the distance between the object and the LiDAR increases. Therefore, during the propagation of light, it is very likely that the signal receiving power of the soft target will exceed that of the hard target. At this time, fog will "block" the object and generate a virtual point. Therefore, the signal receiving power strength of the soft target and the hard target is compared. If , then the point cloud corresponding to the hard target is retained As the i-th foggy point cloud Otherwise, As the i-th foggy point cloud ;
[0061] B6. After assigning i+1 to i, return to step B2 and execute sequentially to finally generate a foggy point cloud dataset. , thus realizing fog simulation.
[0062] C. Foggy point cloud dataset Perform non-uniform cylindrical voxel encoding to obtain the encoded point cloud voxel dataset in the foggy scene ;
[0063] C1. Establish a cylindrical coordinate system with the laser radar as the origin, the vehicle's forward direction as the positive direction of the X axis, the direction perpendicular to the ground and upward as the positive direction of the Z axis, and the direction pointing to the left of the driver as the positive direction of the Y axis. Use formula (1) to convert the Cartesian coordinate system into Coordinates Perform coordinate transformation to obtain the cylindrical coordinate system Coordinate encoding :
[0064] (1)
[0065] In formula (1), express The Euclidean distance from the projection on the XY plane in the Cartesian coordinate system to the origin of the coordinate system in the cylindrical coordinate system, express The angle between the projection on the XY plane in the Cartesian coordinate system to the line connecting the origin of the coordinate system in the cylindrical coordinate system and the X axis in the Cartesian coordinate system, express The coordinate of the Z axis in the Cartesian coordinate system.
[0066] C2. In the cylindrical coordinate system, select On the axis within the range and Axis unit step is the division angle of voxels, On the axis In the range, Axis unit step is the height of the voxel division, and the foggy point cloud data set after preliminary division is obtained. In order to make the point cloud detection algorithm adapt to the influence of foggy conditions with different concentrations (i.e., different extinction coefficients), A non-uniform cylindrical voxel encoding is proposed. On the axis Within the range, the foggy point cloud dataset after preliminary division is divided to obtain the final foggy point cloud dataset ;in, and , and , and They represent the coordinates of the foggy point cloud in the cylindrical coordinate system. axis, axis, The minimum and maximum values on the axis.
[0067] C3. Using the preset extinction coefficient , and the maximum sensing distance of the laser radar is obtained by formula (2) :
[0068] (2)
[0069] In formula (2), n is the detectable background noise, g is the adaptive laser gain of the lidar, is the intensity of the lidar point cloud under clean weather conditions; where n, g, and is the intrinsic parameter of the laser radar;
[0070] C4, such as Figure 3 As shown, the maximum sensing distance of the laser radar ,Will exist The length of the axis is set in sections, so that exist The voxels in the y-axis are evenly divided into S parts. exist arrive The voxels in the foggy scene are evenly divided into T parts, so as to obtain the encoded point cloud voxel dataset in the foggy scene. , where S>T; due to the maximum perception distance With the extinction coefficient Therefore, for fog with higher concentration, the volume of the cylindrical voxels divided by this method is smaller in the near distance, thereby realizing non-uniform cylindrical voxel encoding, which can adaptively adjust the adaptability of the algorithm to fog and improve the robustness of the detection method;
[0071] D. Input into the neural network for feature extraction, and then output the final point cloud recognition result, including the following steps: the encoded point cloud voxel dataset in the foggy scene First, through a 3D convolutional layer and The activation function extracts preliminary features, and then reduces the size of the feature map through the 3D maximum pooling layer. This process is then repeated to further extract and compress features, and finally a high-level feature map is obtained, and then the region proposal network is used. Generate candidate regions, finally fuse the features of the candidate regions with the global features, and use the detection head to output the final detection results. The 3D convolutional network includes: three 3D convolutional layers (32, 64, and 128 channels), each convolutional layer is followed by The activation function and the maximum pooling layer, the convolution step size of the 1st, 2nd, and 3rd 3D convolutional layers is 1, and the step size of the maximum pooling layer is 2.
[0072] In this embodiment, an electronic device is designed, including a memory and a processor. The memory is used to store a target detection program that supports the processor to execute the non-uniform cylindrical voxel encoding of the lidar point cloud. The processor is configured to execute the program stored in the memory.
[0073] In this embodiment, a computer-readable storage medium is designed, on which a computer program is stored. When the computer program is executed by a processor, the target detection step of the non-uniform cylindrical voxel encoding of the laser radar point cloud is executed.
Claims
1. A point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes, characterized in that: The steps include: A. After using LiDAR to collect point cloud data of the target scene in a sunny scene and sampling, the dimension is Sunny point cloud dataset ;in, Represents a sunny point cloud dataset The i-th clear sky point cloud in The coordinates of , Represents a sunny point cloud dataset The number of point clouds, W represents the i-th sunny point cloud The number of messages; B. Using the foggy weather simulation network to simulate the sunny point cloud dataset Processing is performed to generate dimensions of Foggy point cloud dataset ;in, Represents a foggy point cloud dataset The A foggy cloud. The coordinates of ; C. Foggy point cloud dataset Perform non-uniform cylindrical voxel encoding to obtain the encoded point cloud voxel dataset in the foggy scene ; C1. Establish a cylindrical coordinate system with the laser radar as the origin, the vehicle's forward direction as the positive direction of the X axis, the direction perpendicular to the ground and upward as the positive direction of the Z axis, and the direction pointing to the left of the driver as the positive direction of the Y axis. Use formula (1) to convert the Cartesian coordinate system into Coordinates Perform coordinate transformation to obtain the cylindrical coordinate system Coordinate encoding : (1) In formula (1), express The Euclidean distance from the projection on the XY plane in the Cartesian coordinate system to the origin of the coordinate system in the cylindrical coordinate system, express The angle between the projection on the XY plane in the Cartesian coordinate system to the origin of the cylindrical coordinate system and the positive direction of the X axis in the Cartesian coordinate system. express The coordinate of the Z axis in the Cartesian coordinate system; C2. In the cylindrical coordinate system, select On the axis within the range and Axis unit step is the division angle of voxels, On the axis In the range, Axis unit step is the height of the voxel division, and the foggy point cloud dataset after preliminary division is obtained. On the axis Within the range, the foggy point cloud dataset after preliminary division is divided to obtain the final foggy point cloud dataset ;in, and , and , and They represent the coordinates of the foggy point cloud in the cylindrical coordinate system. axis, axis, Minimum and maximum values on the axis; C3. Using the preset extinction coefficient , and the maximum sensing distance of the laser radar is obtained by formula (2) : (2) In formula (2), n is the detectable background noise, g is the adaptive laser gain of the lidar, is the intensity of the lidar point cloud under clean weather conditions; where n, g, and is the intrinsic parameter of the laser radar; C4. Maximum sensing distance using LiDAR ,Will exist The length of the axis is set in sections, so that exist The voxels in the y-axis are evenly divided into S parts. exist arrive The voxels in the foggy scene are evenly divided into T parts, so as to obtain the encoded point cloud voxel dataset in the foggy scene. , where S>T; D. Input into the neural network for feature extraction, and then output the final point cloud recognition result.
2. The point cloud target detection method based on non-uniform cylindrical voxel encoding in foggy scenes according to claim 1 is characterized in that: The step B comprises: B1. According to the spatial pulse response of the laser radar, the targets hit by the laser are divided into hard targets and soft targets; wherein the hard targets refer to the time characteristics of the laser radar that do not change when hitting the target and reflecting; the soft targets refer to the fog molecules that cause distributed scattering of the light pulse; B2. When the laser radar reaches Evenly increase point cloud, thus obtaining a dimension of Point cloud dataset , and calculated by formula (3) Middle Point Cloud Distance to LiDAR ;in, express The number of point clouds in That is Middle Sunny day with clouds : (3) In formula (3), for The coordinates of B3, obtained by formula (4) Middle Point Cloud The spatial impulse response And as the spatial impulse response of the hard target, it is obtained by formula (5) The spatial impulse response And as the spatial impulse response of the kth soft target: (4) (5) In formula (4) and formula (5), represents the scattering coefficient of the hard target, represents the scattering coefficient of the soft target, is the Dirac function, is a unit step function; B4. Calculate the signal receiving power of the hard target using equations (6) and (7) respectively and the signal receiving power of the kth soft target : (6) (7) In formula (6) and formula (7), is the half-power pulse width of the laser radar, represents the peak pulse power of the laser radar, c is the speed of light, t is the time, is the system constant of the laser radar, represents the crossover function; B5. Calculate the maximum signal receiving power of the soft target ,in, Indicates the point cloud corresponding to the maximum signal received power Distance to the LiDAR; like , then the point cloud corresponding to the hard target is retained As the i-th foggy point cloud Otherwise, As the i-th foggy point cloud ; B6. After assigning i+1 to i, return to step B2 and execute sequentially to finally generate a foggy point cloud dataset. .
3. An electronic device, comprising a memory and a processor, characterized in that: The memory is used to store a target detection program that supports the processor to execute the non-uniform cylindrical voxel encoding of the laser radar point cloud as described in claim 1 or 2, and the processor is configured to execute the program stored in the memory.
4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the target detection step of non-uniform cylindrical voxel encoding of the laser radar point cloud according to claim 1 or 2 is performed.
Citation Information
Patent Citations
Multi-scale physical simulation sand and dust weather laser radar point cloud data enhancement method
CN117872316A
Point cloud detection method and system, and computing device
CN117994774A
Urban air pollutant positioning method and device
CN118151164A
Method and device for monitoring laser radar
JP2002196075A
Radar system, mobile device and radar detection method
WO2021258246A1
