Processing method of TOF point cloud, point cloud optimization method, laser radar and robot

Abstract

This application relates to the field of lidar technology, disclosing a method for processing Time-of-Flight (TOF) point clouds, a point cloud optimization method, a lidar system, and a robot. The TOF point cloud processing method compensates for the distance of each laser point in the original TOF point cloud based on real-time temperature, obtaining a TOF compensated point cloud. This effectively reduces the impact of temperature drift on the measurement distance, making the TOF compensated point cloud more accurate. The point cloud optimization method acquires a first point cloud and a second point cloud. For each angle, based on the first and second point clouds, it determines the distance of a third laser point at each angle. The third laser points at each angle constitute the optimized point cloud. This method leverages the advantages of both TOF ranging and triangulation methods while avoiding their disadvantages, resulting in an optimized point cloud with high accuracy.

Description

Technical Field

[0001] This application relates to the field of lidar technology, and in particular to a method for processing TOF point clouds, a point cloud optimization method, lidar, and a robot. Background Technology

[0002] With the continuous development of technology, LiDAR is widely used in robotics, autonomous driving, and unmanned vehicles. LiDAR (Laser Detection and Ranging) is a radar system that uses emitted laser beams to detect the position, velocity, and other characteristics of a target. A LiDAR system consists of a transmitter and a receiver. The transmitter emits a detection signal (laser) towards the target, and the receiver receives the signal reflected back from the target (reflected light). The LiDAR then compares the received signal with the emitted signal, performs appropriate processing, and obtains relevant information about the target, such as its distance, azimuth, altitude, velocity, attitude, and even shape.

[0003] Among them, lidar employs two ranging methods: triangulation and Time of Flight (TOF). These two methods have their own accuracies and limitations at different distances. Summary of the Invention

[0004] The main technical problem solved by the embodiments of this application is to provide a method for processing TOF point clouds, a point cloud optimization method, a lidar and a robot, which can make TOF point clouds more accurate and enable the optimized point clouds to have high accuracy at different distances.

[0005] Firstly, this application provides a method for processing TOF point clouds, including:

[0006] Acquire the TOF raw point cloud. The TOF raw point cloud is the point cloud data collected by the first receiver of the lidar based on the TOF ranging method. The TOF raw point cloud includes the measured distance of each lidar point.

[0007] Obtain the real-time temperature from the lidar;

[0008] Based on the real-time temperature, distance compensation is performed on each laser point in the original TOF point cloud to obtain the TOF compensated point cloud.

[0009] In some embodiments, the aforementioned distance compensation for each laser point in the original TOF point cloud based on real-time temperature is performed to obtain a TOF compensated point cloud, including:

[0010] For each laser point in the original TOF point cloud, the compensation distance of each laser point is calculated using a preset compensation model, where the preset compensation model is a function of the real-time temperature and the preset reference temperature.

[0011] The compensation distance of each laser point is added to its measurement distance to obtain the TOF compensated point cloud.

[0012] In some embodiments, the method further includes:

[0013] Distance correction is applied to the TOF compensated point cloud to obtain the TOF corrected point cloud;

[0014] The aforementioned distance correction of the TOF compensated point cloud yields the TOF corrected point cloud, including:

[0015] For each laser point in the TOF compensated point cloud, the corrected distance of each laser point is calculated using a preset correction model to obtain the TOF corrected point cloud. The preset correction model is a function of distance and brightness.

[0016] In some embodiments, the method further includes:

[0017] Smoothing is applied to the TOF-corrected point cloud.

[0018] The aforementioned smoothing process for TOF-corrected point clouds includes:

[0019] Each laser point in the TOF corrected point cloud is taken as the starting point, and m laser points with consecutive angles are taken as a smoothing unit to obtain multiple smoothing units, where m is an integer greater than or equal to 5.

[0020] For each laser point in a smoothing unit, coordinate smoothing is performed based on the coordinates of multiple laser points near the laser point to obtain the smoothed coordinates.

[0021] Secondly, this application provides a point cloud optimization method, including:

[0022] Acquire the first point cloud and the second point cloud. The first point cloud is the point cloud obtained by processing the TOF point cloud using the processing method described in the first aspect. The second point cloud is the point cloud data collected by the second receiver of the lidar based on the triangulation method.

[0023] The first laser point and the second laser point at each angle are obtained from the first point cloud and the second point cloud respectively. Based on the distance between the first laser point and the second laser point, the distance of the third laser point at each angle is determined, and the optimized point cloud is obtained.

[0024] In some embodiments, determining the distance of the third laser point at each angle based on the distance of the first laser point and the distance of the second laser point includes:

[0025] If the distance to the first laser point is less than or equal to the first threshold, the distance to the third laser point is determined to be the distance to the second laser point.

[0026] If the distance of the first laser point is greater than the first threshold and less than or equal to the second threshold, the distance of the third laser point is determined to be the fusion result of the distances of the first laser point and the distances of the second laser point;

[0027] If the distance to the first laser point is greater than the second threshold, the distance to the third laser point is determined to be the distance to the first laser point.

[0028] In some embodiments, the method further includes:

[0029] If the distance to the first laser point is less than or equal to the third threshold, and the second laser point is not displayed, then the distance to the third laser point is determined to be the distance to the first laser point, wherein the third threshold is less than the first threshold.

[0030] In some embodiments, the aforementioned determination of the distance to the third laser point as a fusion result of the distances to the first laser point and the second laser point includes:

[0031] The distance to the third laser point is calculated using the following formula;

[0032] dis = α*d1 + β*d2;

[0033] in,

[0034] Ratio=(d1-d base ) / stride;

[0035] Where dis is the distance to the third laser point, d1 is the distance to the first laser point, d2 is the distance to the second laser point, α and β are the weights, Ratio is the scaling factor, and d base is the mean of the second and third thresholds, and stride is the difference between the mean and the second or third threshold.

[0036] Thirdly, this application provides a lidar system, including:

[0037] At least one processor, and

[0038] A memory that is communicatively connected to at least one processor, wherein,

[0039] The memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform the method of the first aspect or the method of the second aspect.

[0040] Fourthly, this application provides a robot that includes a lidar as described in the third aspect.

[0041] The beneficial effects of this application's embodiments are as follows: Unlike existing technologies, the TOF point cloud processing method provided in this application's embodiments acquires the original TOF point cloud (the original TOF point cloud is point cloud data collected by the first receiver of the lidar based on the TOF ranging method) and obtains the lidar's real-time temperature. Then, based on the real-time temperature, distance compensation is performed on each laser point in the original TOF point cloud to obtain a TOF compensated point cloud. This method effectively reduces the temperature drift caused by temperature on the measurement distance, making the TOF compensated point cloud more accurate.

[0042] The point cloud optimization method provided in this application acquires a first point cloud and a second point cloud (the first point cloud is the TOF-compensated point cloud mentioned above, and the second point cloud is point cloud data collected by the second receiver of the lidar based on triangulation). It then obtains a first laser point and a second laser point at each angle from the first and second point clouds respectively. Based on the distance between the first and second laser points, it determines the distance of a third laser point at each angle, thus obtaining an optimized point cloud. This method ensures that the optimized point cloud is a combination of the TOF-compensated point cloud (first point cloud) and the triangulation point cloud (second point cloud), leveraging the advantages of both methods while avoiding their disadvantages. Consequently, the third laser point in the optimized point cloud is relatively accurate at various distances, achieving high precision across different distances, which is beneficial for the accuracy of ranging or mapping. Attached Figure Description

[0043] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0044] Figure 1 This is a schematic diagram illustrating the application environment of the TOF point cloud processing method or point cloud optimization method in some embodiments of this application;

[0045] Figure 2 A schematic diagram illustrating the principle of laser ranging provided in some embodiments of this application;

[0046] Figure 3 This is a flowchart illustrating the TOF point cloud processing method in some embodiments of this application;

[0047] Figure 4 This is a flowchart illustrating the point cloud optimization method in some embodiments of this application;

[0048] Figure 5 This application provides schematic diagrams of the structure of a lidar for some embodiments.

[0049] Figure 6 The diagram shows the structure of a robot provided in some embodiments of this application. Detailed Implementation

[0050] The present application will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any way. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application. These all fall within the protection scope of the present application.

[0051] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] It should be noted that, unless there is a conflict, the various features in the embodiments of this application can be combined with each other, all of which are within the protection scope of this application. Furthermore, although functional modules are divided in the device schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," and "third" used herein do not limit the data or execution order, but only distinguish identical or similar items with essentially the same function and effect.

[0053] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The term "and / or" as used in this specification includes any and all combinations of one or more of the associated listed items.

[0054] Furthermore, the technical features involved in the various embodiments of this application described below can be combined with each other as long as they do not conflict with each other.

[0055] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating the application environment of a TOF point cloud processing method or point cloud optimization method provided in an embodiment of this application. For example... Figure 1 As shown, robot 10 is located on the ground, which can be the floor of a living room or office. The location of the robot includes target objects such as desks, flower pots, and sofas.

[0056] The robot is equipped with a LiDAR 11, which scans target objects in space to obtain point cloud data. Based on this point cloud data, the distance from the target object to the LiDAR is calculated. Understandably, LiDAR accuracy is affected by environmental factors, such as temperature, and the ranging method also influences accuracy at different distances, resulting in insufficient accuracy at some distances. Therefore, the LiDAR further processes or optimizes the point cloud data to calculate an accurate distance. By measuring the distance to the target object, the robot can be guided to move and avoid obstacles.

[0057] The robot 10 can be configured into any suitable shape to perform specific business functions. For example, in some embodiments, the robot 10 can be a mobile robot based on a SLAM system, such as a cleaning robot, pet robot, handling robot, care robot, remote monitoring robot, or sweeping robot. The cleaning robot includes, but is not limited to, sweeping robots, vacuuming robots, mopping robots, or floor washing robots.

[0058] In some embodiments, the robot includes a main body and drive wheel components, a camera unit, a sensing unit, a lidar, and a controller. The main body can be generally elliptical, triangular, D-shaped, or other shapes. The controller is located on the main body, which is the main structure of the robot. The shape, structure, and manufacturing material (such as hard plastic or metals like aluminum or iron) can be selected according to the actual needs of the robot 10; for example, it can be set to a relatively flat cylindrical shape commonly found in robotic vacuum cleaners.

[0059] The drive wheel assembly is mounted on the main body and is used to drive the robot's movement. If the robot is a cleaning robot, the drive wheel assembly drives the robot to move on the surface to be cleaned. In some embodiments, the drive wheel assembly includes a left drive wheel, a right drive wheel, and an omnidirectional wheel. The left and right drive wheels are respectively mounted on opposite sides of the main body. The omnidirectional wheel is mounted at the front bottom of the main body and is a movable caster wheel that can rotate 360 ​​degrees horizontally, allowing the robot to turn flexibly. The mounting of the left drive wheel, right drive wheel, and omnidirectional wheel forms a triangle to improve the stability of the robot's movement.

[0060] In some embodiments, a camera unit is disposed on the robot's body and is used to acquire image data and / or video data. The camera unit is communicatively connected to a controller and is used to acquire image data and / or video data within its coverage area, for example, acquiring image data and / or video data of a specific location, and sending the acquired image data and / or video data to the controller. In this application embodiment, the camera unit includes, but is not limited to, infrared cameras, night vision cameras, webcams, digital cameras, high-definition cameras, 4K cameras, 8K high-definition cameras, and other camera devices.

[0061] In some embodiments, the sensing unit is used to collect some motion parameters of the robot and various types of environmental space data. The sensing unit includes various suitable sensors, such as gyroscopes, infrared sensors, odometers, magnetometers, accelerometers, or speedometers, etc.

[0062] In some embodiments, the lidar 11 is communicatively connected to the controller and is mounted on the body of the robot 10, for example, on the mobile chassis of the robot 10. The lidar 11 is used to sense obstacles in the environment surrounding the mobile robot 10, obtain the distances of surrounding objects, and send them to the controller so that the controller can control the robot's movement based on the distances of surrounding objects. In some embodiments, the lidar 11 includes pulse lidar, continuous wave lidar, and other types of radar, and the mobile chassis includes a universal chassis, an arched mobile chassis, and other robot mobile chassis.

[0063] In some embodiments, the controller is located inside the main body and is an electronic computing core built into the robot body. It is used to execute logical operations to achieve intelligent control of the robot. The controller is electrically connected to the left drive wheel, right drive wheel, and omnidirectional wheel. As the control core of the robot, the controller is used to control the robot's walking, backward movement, and some business logic processing. For example, the controller receives image data and / or video data sent by the camera unit and receives laser point cloud data sent by the LiDAR, and constructs an environmental map based on the laser point cloud data. The controller uses Simultaneous Localization and Mapping (SLAM), i.e., the laser SLAM algorithm, to calculate the laser point cloud data of the monitored area to construct the environmental map. In this embodiment, the laser SLAM algorithm includes Kalman filtering, particle filtering, and graph optimization methods.

[0064] Understandably, a controller can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a microcontroller, an ARM (Acorn RISC Machine) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination of these components. A controller can also be any conventional processor, controller, microcontroller, or state machine. A controller can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration, or one or more combinations of a microcontroller unit (MCU), a field-programmable gate array (FPGA), or a system-on-a-chip (SoC).

[0065] It is understood that the robot 10 in this application embodiment also includes a storage module, which includes, but is not limited to, one or more of the following devices: FLASH flash memory, NAND flash memory, vertical NAND flash memory (VNAND), NOR flash memory, resistive random access memory (RRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), spin-transfer torque random access memory (STT-RAM).

[0066] It should be noted that, depending on the task to be completed, in addition to the functional modules mentioned above, the main body of the robot can also be equipped with one or more other different functional modules (such as water tanks, cleaning devices, etc.) to work together to perform the corresponding tasks.

[0067] To facilitate understanding of the methods provided in the embodiments of this application, the relevant technologies involved in the embodiments of this application are described here:

[0068] A lidar system comprises a transmitter, a receiver, a processor, and a rotating mechanism. The transmitter is a device that emits laser light, such as a gas laser, a solid-state laser, a semiconductor laser, or a free-electron laser. The receiver is a device that receives the laser light, such as a charge-coupled device (CCD).

[0069] The processor is primarily responsible for controlling the transmitter to emit laser light and processing the laser signals received by the receiver to calculate the distance information of the target object. The processor can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0070] The rotating mechanism is the mounting frame for the lidar, used for orientation adjustment. In some embodiments, the rotating mechanism may include a rotating base driven by a belt. The transmitter, receiver, and processor are mounted on the rotating mechanism, which rotates at a stable speed, enabling the lidar to scan the surrounding environment and generate real-time point cloud information.

[0071] LiDAR employs two ranging methods: triangulation and Time-of-Flight (TOF). Please refer to [link / reference]. Figure 2 , Figure 2 Image (a) in the diagram illustrates the principle of triangulation. Triangulation primarily involves a transmitter emitting a laser beam. The laser beam illuminates the target object at a specific incident angle. The laser is reflected and scattered on the target object's surface. A lens at another angle focuses the reflected laser beam, creating an image that is then projected onto a receiver. Because the transmitter and receiver are separated by a distance *s*, the target object at different distances will be imaged at different positions on the receiver, depending on the optical path. For example... Figure 2 As shown in (a), s is the distance between the focal points of the transmitter and receiver (i.e., the baseline), d is the distance between the transmitter and the target object, α is the heading angle, p is the perpendicular distance from the target object to the baseline, and f is the focal length. The position of endpoint A, where the dashed line parallel to the laser direction intersects the receiver, is known. The distance x from the center of the imaging point B on the receiver after laser reflection to endpoint A is also known. By constructing geometrically similar triangles, the distance to the target object is p = f * s / x.

[0072] Using the triangulation method, the point cloud data generated by the lidar includes the angle, distance, and brightness of each point. The angle is the angle of the laser spot formed on the target object in polar coordinates; the distance is the distance from the laser spot to the lidar; and the brightness is the brightness of the laser spot.

[0073] Please see Figure 2 , Figure 2(b) in the diagram illustrates the principle of the Time-of-Flight (TOF) method. The TOF method involves firing a laser beam at a target object for an extremely short time and calculating the distance from the transmitter to the target object by directly measuring the flight time of the laser from emission to its impact on the target object and back. Using the TOF method, the point cloud data generated by the lidar includes the angle and distance of each point. The angle is the polar coordinate of the position where the laser hits the target object, and the distance is the distance from the position where the laser hits the target object to the lidar.

[0074] In some applications, laser ranging based on Time-of-Flight (TOF) ranging is susceptible to external environmental factors such as temperature and vibration, resulting in significant errors in the TOF point cloud and low accuracy. Furthermore, as the principle of TOF ranging indicates, laser ranging based on TOF has a short flight time when measuring close-range objects, leading to larger errors in close-range measurements.

[0075] In some application scenarios, as can be seen from the principle of triangulation, the position of the light spot on the receiver changes less as the distance increases. This leads to the receiver being unable to distinguish whether the position of the light spot has changed beyond a certain distance. In other words, the distance x from the center of the imaging point B on the receiver after laser reflection to the endpoint A changes little, resulting in a large error in the distance p of the target object.

[0076] As can be seen from the above, different distance measurement methods have their own shortcomings and errors at different distances, resulting in poor accuracy.

[0077] To address the aforementioned issues, this application provides a method for processing TOF point clouds, a point cloud optimization method, and a lidar, which enables TOF point clouds to be more accurate and allows the optimized point clouds to maintain high precision at different distances.

[0078] The TOF point cloud processing method involves acquiring the original TOF point cloud (which is point cloud data collected by the first receiver of the lidar based on the TOF ranging method) and obtaining the lidar's real-time temperature. Then, based on the real-time temperature, distance compensation is performed on each lidar point in the original TOF point cloud to obtain the TOF compensated point cloud. This method effectively reduces the impact of temperature drift on the measurement distance, making the TOF compensated point cloud more accurate.

[0079] This point cloud optimization method involves acquiring a first point cloud and a second point cloud (the first point cloud is the TOF-compensated point cloud mentioned above, and the second point cloud is point cloud data collected by the second receiver of the lidar based on triangulation). From the first and second point clouds, a first laser point and a second laser point at each angle are obtained respectively. Based on the distances of the first and second laser points, the distance of a third laser point at each angle is determined, resulting in the optimized point cloud. This method combines the TOF-compensated point cloud (first point cloud) and the triangulation point cloud (second point cloud), leveraging the advantages of both methods while avoiding their disadvantages. Consequently, the third laser point in the optimized point cloud is relatively accurate at various distances, demonstrating high precision across different distances, which is beneficial for the accuracy of ranging or mapping.

[0080] As can be understood from the above, the TOF point cloud processing method provided in this application embodiment can be implemented by various types of electronic devices with computing capabilities, such as LiDAR, robot controller, or other devices with computing capabilities.

[0081] The following describes the TOF point cloud processing method provided in this application embodiment, using exemplary applications and implementations of the lidar provided in the embodiments of this application. Please refer to... Figure 3 , Figure 3 This is a flowchart illustrating the TOF point cloud processing method provided in the embodiments of this application.

[0082] It is understandable that the lidar is installed on the robot, and specifically, the execution entity of the TOF point cloud processing method is one or more processors of the lidar.

[0083] like Figure 3 As shown, the method S100 may specifically include the following steps:

[0084] S101: Acquire the TOF raw point cloud. This TOF raw point cloud is the point cloud data collected by the first receiver of the lidar based on the TOF ranging method. The TOF raw point cloud includes the measured distance of each laser point.

[0085] In this embodiment, the first receiver is a receiver of the lidar. It can be understood that the first receiver is a device of the lidar that receives laser light, such as a charge coupled device (CCD).

[0086] The TOF raw point cloud is point cloud data acquired by the first receiver of the lidar based on the TOF ranging method. As the principle of TOF ranging is known, the lidar transmitter emits a laser, which hits the target object and returns to the first receiver, generating point cloud data. Using the TOF method, the TOF raw point cloud data generated by the lidar includes the angle and distance of each laser point. The angle is the angle of the laser's position on the target object in polar coordinates, and the distance is the distance from the laser's position on the target object to the lidar.

[0087] S102: Obtain the real-time temperature of the lidar.

[0088] Understandably, as an electronic device, lidar generates heat and its temperature rises when it operates continuously for extended periods. Higher temperatures cause temperature drift in the lidar's measurement results. Temperature drift is the shift in the distance of the laser point as the temperature increases.

[0089] Thus, the real-time temperature of the lidar is obtained. In some embodiments, a temperature sensor can be used to collect the real-time temperature of the lidar.

[0090] S103: Based on the real-time temperature, perform distance compensation on each laser point in the original TOF point cloud to obtain the TOF compensated point cloud.

[0091] It is understandable that since temperature can cause temperature drift in the distance of laser points, compensating for the distance of each laser point in the original TOF point cloud based on the real-time temperature can effectively reduce the impact of temperature drift on the measurement distance, making the TOF compensated point cloud more accurate.

[0092] In some embodiments, the aforementioned step S103 specifically includes:

[0093] For each laser point in the original TOF point cloud, a pre-defined compensation model is used to calculate the compensation distance for each laser point. This model is a function of the real-time temperature and a pre-defined reference temperature. The compensation distance for each laser point is then added to its measured distance to obtain the TOF compensated point cloud.

[0094] In this embodiment, the lidar is placed in a constant temperature chamber, and distance measurements are performed at various temperatures to obtain data sets regarding temperature and distance. Then, using these data sets, a preset compensation model is established based on the real-time temperature and a preset reference temperature. The preset reference temperature is a temperature threshold. It is understood that the preset compensation model was obtained by the applicant through experimental analysis. Therefore, the preset compensation model, obtained through testing a large amount of historical data, can calculate the compensation distance corresponding to the temperature.

[0095] In some embodiments, the preset compensation model is as follows:

[0096] d b =k b *(T i -T thr )+bias1 (1)

[0097] Where, d b To compensate for distance, k b T is the temperature compensation factor coefficient. thr T is the preset reference temperature. i Let be the real-time temperature at time i, and bias1 be the bias parameter.

[0098] Then, the compensation distance is added to the measured distance of the laser point to obtain the TOF compensated point cloud. In this embodiment, the compensated distance of the laser point at time i is:

[0099] d tc =d0+d b (2)

[0100] Where d0 is the original distance of the laser point at time i, d tc This is the distance after warming.

[0101] It is understandable that temperature drift occurs due to the inherent characteristics of the laser tube and sensor components, causing a shift in the distance of the laser point, either too large or too small. If the temperature drift causes the distance to be too large, the compensation distance is d. b If the value is negative, and temperature drift causes the distance to be too small, then the compensation distance d is... b It is a positive number.

[0102] In this embodiment, the above method can effectively reduce the temperature drift caused by temperature on the measurement distance, making the TOF compensated point cloud more accurate.

[0103] In some embodiments, the method S100 further includes:

[0104] S104: Perform distance correction on the TOF compensated point cloud to obtain the TOF corrected point cloud.

[0105] In some embodiments, at least five targets are set up, and a lidar is used to measure the distance to these five targets to obtain correction parameters. In some embodiments, five targets are set up, each made of homogeneous material, such as a white target, and the placement distance from the lidar can be 1.0m, 3.0m, 5.0m, 7.0m, or 9.0m, etc. After the lidar measures the distance to these targets, the measurement results are obtained. It can be understood that the measurement results here refer to the compensated TOF point cloud. For each laser point in the TOF compensated point cloud, correction parameters are obtained based on the difference between their distance and the placement distance. After calculating the compensated distance and the correction parameters, the obtained corrected distance is closer to the placement distance (true distance), thus, the corrected TOF point cloud is more accurate.

[0106] In this embodiment, step S104 specifically includes: for each laser point in the TOF compensated point cloud, calculating the corrected distance of each laser point using a preset correction model to obtain the TOF corrected point cloud. The preset correction model is a function of distance and brightness.

[0107] It is understandable that the preset correction model was obtained by the applicant after conducting experimental analysis of historical measurement results. Thus, by analyzing a large number of historical measurement results, the preset correction model can calculate the corrected distance of each laser point.

[0108] In some embodiments, the preset correction model is as follows:

[0109] d = k1*d tc +k2*peak+bias2 (3)

[0110] Where d is the corrected distance, and k1, k2, and bias2 are the ranging parameters obtained by the lidar through correction. tc The distance is the temperature-compensated distance, and peak represents the brightness of the light spot at that moment, which can be obtained from the lidar. In this embodiment, it is assumed that the measured distance of the laser point (the compensated distance) is affected by the brightness, and a linear combination model of distance and brightness is established.

[0111] To obtain accurate ranging information, each individual radar needs to be calibrated, and specific k1, k2, and bias2 are solved using designated targets. Specifically, for example, five targets are set, each made of homogeneous material, such as white targets, and the placement distances from the lidar can be 1.0m, 3.0m, 5.0m, 7.0m, or 9.0m, etc.

[0112] As can be seen from formula (3), at least three equations are needed to solve for three variables. For systems of equations with more than three variables, overdetermined equations can be used to solve them. Formula (3) can be rewritten in matrix form as follows:

[0113]

[0114] Let X = [d] tc peak 1], Y = d, therefore we can obtain the following:

[0115] XA=Y (5)

[0116] The optimal solution can be obtained by fitting using the least squares method:

[0117] A=inv(X′*X)*X′*Y (6)

[0118] In formula (6), X′ represents the transpose of matrix X, and inv(X′*X) means that matrix X′*X is invertible. If X′*X is a singular matrix, then the pseudo-inverse is used to solve the above formula.

[0119] The distance d between each laser point after temperature compensation tc Substituting the corresponding brightness of each laser point into formula (6), the correction parameters k1, k2 and bias2 can be obtained.

[0120] Once the corrected ranging parameters are obtained, formula (3) can be determined. In actual testing, by substituting formula (3) with the temperature-compensated distance and brightness of the current laser point, the accurate measured distance d (i.e., the corrected distance) can be calculated.

[0121] In this embodiment, the lidar is calibrated in advance to obtain correction parameters. During the calibration process, it is assumed that the corrected distance is related to the temperature-compensated distance and the brightness of the laser point, and a linear combination model of distance and brightness is established. Multiple targets are preset, the compensated distance is obtained, and the linear combination model is fitted to obtain an accurate preset correction model, making the distance correction more accurate. The accuracy of the TOF corrected point cloud obtained in the actual ranging process is higher.

[0122] Because Time-of-Flight (TOF) ranging lidar is easily affected by factors such as temperature and motor rotation, the generated point cloud is prone to undulating and wavy patterns. In addition, when TOF measures at long distances, such as in a straight object at 8m, the distances between several adjacent laser points perpendicular to the object are 8030mm, 7970mm, 8010mm, etc., making the point cloud transition unsmooth. Furthermore, when TOF measures at close distances, such as at 1m, the smaller the distance, the more stringent the timing of flight time becomes, so the generated point cloud is also prone to undulating and wavy patterns.

[0123] In some embodiments, the method S100 further includes:

[0124] S105: Smooth the TOF corrected point cloud.

[0125] It is understandable that the TOF corrected point cloud displays a series of laser points hitting an object, and the transition between some laser points is not smooth, similar to a "wave". In order to reduce the impact of the unsmooth transition on subsequent mapping, in this embodiment, the TOF corrected point cloud is smoothed to alleviate these unsmooth transition states, making the processed point cloud smoother.

[0126] In this embodiment, step S105 specifically includes:

[0127] Taking each laser point in the TOF-corrected point cloud as a starting point, and considering m consecutive laser points at a given angle as a smoothing unit, multiple smoothing units are obtained, where m is an integer greater than or equal to 5. For each laser point in a smoothing unit, coordinate smoothing is performed based on the coordinates of multiple laser points near the laser point to obtain the smoothed coordinates.

[0128] Understandably, point cloud data includes several points distributed at equal angles, for example, with a 0.5° interval between any two adjacent points. Here, angular continuity means that the angle difference is less than a threshold T1, which is generally the set scanning angle interval (e.g., 0.5°). The reason for selecting m angularly continuous laser points as smoothing units and performing smoothing operations is to perform separate smoothing operations on different objects with separate positions, which is beneficial to the accuracy of local point clouds.

[0129] Using each laser point in the TOF-corrected point cloud as a starting point, m (m≥5) consecutive laser points are grouped into a smoothing unit to obtain multiple smoothing units. For example, m can be 5, then 5 consecutive points form one smoothing unit. In some embodiments, if the TOF-corrected point cloud includes 40 points, then starting from the first point, 5 points are grouped into one smoothing unit, and each time one point is moved forward, resulting in 46 smoothing units including 5 points; if the TOF-corrected point cloud includes 6 points, starting from the first point, 5 points are grouped into one smoothing unit, and each time one point is moved forward, resulting in 2 smoothing units including 5 points.

[0130] It is understandable that the angular intervals of laser points in point cloud data are small, and the angle occupied by m laser points is small, corresponding to a small range of length on the object surface. Due to the continuity of the object surface, the same object surface is approximately smooth within a small range of length. Therefore, the distance difference between m laser points in a smooth unit is not significant. The distance here can be the distance in polar coordinates.

[0131] Therefore, for each laser point in a smoothing unit, coordinate smoothing can be performed based on the coordinates (angles and distances in polar coordinates) of multiple nearby laser points to obtain smoothed coordinates. After all laser points in each smoothing unit have been traversed and smoothed, a smoothed point cloud is obtained.

[0132] In some embodiments, taking m=5 as an example for illustration, the five-point cubic smoothing concept is adopted. The polar coordinates of the five laser points are fitted with a polynomial to obtain a smooth curve. The specific smoothing formula is as follows:

[0133]

[0134]

[0135]

[0136]

[0137]

[0138] Where y1, y2, y3, y4, and y5 represent the original distance y values ​​of the five points in the polar coordinate system. These are the smoothed polar coordinates, where y0 is the point preceding y1. -1 These are the first two points of y1.

[0139] If m is greater than 5 points, then formulas (7) and (8) are the two beginning endpoints after smoothing of the m points, and formulas (10) and (11) are the two ending endpoints after smoothing of the m points. For the smoothing results of the intermediate points, formula (9) is used. The generalization of this formula is as follows:

[0140]

[0141] The range of variable i in the above formula is: 3≤i≤m-2.

[0142] It is worth noting that the above formulas (7) to (11) were determined by the inventors of this application through experimental analysis. Experiments have confirmed that using the above formulas for smoothing operations can effectively reduce the undulating and wave-like appearance in the TOF corrected point cloud, making the point cloud of each object smoother and more stable.

[0143] In summary, the TOF point cloud processing method provided in this application obtains the original TOF point cloud (the original TOF point cloud is point cloud data collected by the first receiver of the lidar based on the TOF ranging method) and the real-time temperature of the lidar. Then, based on the real-time temperature, distance compensation is performed on each laser point in the original TOF point cloud to obtain the TOF compensated point cloud. This method effectively reduces the temperature drift caused by temperature on the measurement distance, making the TOF compensated point cloud more accurate. In some embodiments, the TOF compensated point cloud is further corrected to obtain a more accurate TOF corrected point cloud. In some embodiments, the TOF corrected point cloud is further smoothed to reduce fluctuations in the point cloud data, making the smoothed TOF corrected point cloud more accurate.

[0144] As can be understood from the above, the point cloud optimization method provided in this application embodiment can be implemented by various types of electronic devices with computing capabilities, such as by lidar, by robot controller, or by other devices with computing capabilities.

[0145] The point cloud optimization method provided in this application is illustrated below with examples of the LiDAR applications and implementations provided in the embodiments of this application. Please refer to... Figure 4 , Figure 4 This is a flowchart illustrating the point cloud optimization method provided in the embodiments of this application.

[0146] It is understandable that the lidar is installed on the robot, and specifically, the execution entity of the TOF point cloud processing method is one or more processors of the lidar.

[0147] like Figure 4 As shown, method S200 may specifically include the following steps:

[0148] S201: Acquire the first point cloud and the second point cloud. The first point cloud is a point cloud obtained by processing the TOF point cloud using any of the above-mentioned methods. The second point cloud is point cloud data collected by the second receiver of the lidar based on the triangulation method.

[0149] It is understood that in some embodiments, the first point cloud may be the aforementioned TOF compensated point cloud; in some embodiments, the first point cloud may be the aforementioned TOF corrected point cloud; and in some embodiments, the first point cloud may be the point cloud after the aforementioned TOF corrected point cloud has been processed by a smoothing operation.

[0150] In this embodiment, the second receiver is a receiver of the lidar. It can be understood that the second receiver is a device of the lidar that receives laser light, such as a charge coupled device (CCD).

[0151] The second point cloud is point cloud data generated by the second receiver of the lidar based on the triangulation method. Using triangulation, the point cloud data generated by the lidar includes the angle, distance, and brightness of each point. Specifically, the angle is the angle of the laser spot formed on the target object in polar coordinates, the distance is the distance from the laser spot to the lidar, and the brightness is the brightness of the laser spot.

[0152] In some embodiments, the distance measurement formula of the trigonometric method is as follows:

[0153] d = n1 / (n2-cx) (13)

[0154] Where n1 and n2 are the ranging parameters that can be obtained by the lidar through calibration, and cx is the center of the light spot, that is, the value of the vertical coordinate of the light spot on the second receiver, in pixels. When the width of the second receiver is 480 pixels, the value of cx is 1-480.

[0155] The magnitude of parameter n1 is related to the structure of the lidar, while the magnitude of parameter n2 is related to the position of the light spot falling on the second receiver. To obtain accurate ranging information, each individual lidar unit based on the triangulation method needs to be calibrated. The specific values ​​of n1 and n2 are then determined using a set target.

[0156] In some embodiments, the number of targets is at least five, and each target is made of homogeneous material, such as a white target. The distance between each target is within the range of the minimum distance at which one sensor of the lidar has higher ranging accuracy to the minimum distance at which another sensor has higher ranging accuracy. For example, if the ranging range of lidar sensor A is 0.1m-2.0m and the ranging range of sensor B is 1.0m-10.0m, then the distance between each target is 0.1m-1.0m. The distances of the five targets from the lidar can be 0.1m, 0.3m, 0.6m, 1.0m, and 1.5m, respectively.

[0157] As can be seen from formula (13), at least two equations are needed to solve for two variables. For a system of equations with more equations than variables, overdetermined equations can be used for solving. Formula (13) can be written in matrix form as shown in formula (14) below:

[0158]

[0159] Let X = [-1d], Y = [d*cx], therefore we can obtain the following:

[0160] XA=Y (15)

[0161] The optimal solution can be obtained by fitting using the least squares method:

[0162] A=inv(X′*X)*X′*Y (16)

[0163] In the above formula, X′ represents the transpose of matrix X, and inv(X′*X) means finding the invertibility of matrix X′*X. If X′*X is a singular matrix, then the pseudo-inverse is used to solve the above formula.

[0164] Substituting the distance value d for each target and the corresponding cx for each target into formula (16), the calibration parameters n1 and n2 can be obtained. After obtaining the calibrated ranging parameters, formula (13) can be determined. Thus, in actual testing, by using formula (13) and substituting cx for the current laser point, the accurate distance d can be calculated.

[0165] After each laser point acquired by the second receiver is calibrated according to formula (13), the calibrated triangulation point cloud is obtained. In some embodiments, this triangulation point cloud can be used as the aforementioned second point cloud.

[0166] S202: Obtain the first laser point and the second laser point at each angle from the first point cloud and the second point cloud respectively. Determine the distance of the third laser point at each angle based on the distance of the first laser point and the distance of the second laser point to obtain the optimized point cloud.

[0167] It is understandable that when a lidar emits lasers at a certain scanning frequency, the resulting first or second point cloud contains laser points distributed according to a certain angular resolution. Therefore, it can traverse various angles. For example, if the lidar's scanning range is 30° and its angular resolution is 0.5°, there are 60 angles, and theoretically, the first or second point cloud will contain laser points at these 60 angles.

[0168] The first and second laser points at each angle are obtained from the first and second point clouds, respectively. Since both Time-of-Flight (TOF) and triangulation methods have their advantages and disadvantages, specifically, as the principle of TOF ranging shows, laser ranging based on TOF has a short flight time when measuring close-range objects, leading to a larger error at close range. As the principle of triangulation shows, the position of the laser spot on the receiver changes less with increasing distance, meaning that beyond a certain distance, the receiver cannot distinguish whether the position of the laser spot has changed. That is, the distance x from the center of the imaging point B on the receiver after laser reflection to the endpoint A changes little, resulting in a larger error in the distance p of the target object.

[0169] Therefore, based on the distances of the first and second laser points, the distance of the third laser point at each angle is determined. In other words, the distance of the third laser point is obtained by comprehensively considering the distances of the first laser point using the Time-of-Flight (TOF) ranging method and the distance of the second laser point using the triangulation ranging method. This approach leverages the advantages of both methods while avoiding their disadvantages, resulting in a more accurate and precise distance for the third laser point.

[0170] In some embodiments, the aforementioned step S202 specifically includes:

[0171] 1) If the distance of the first laser point is less than or equal to the first threshold, the distance of the third laser point is determined to be the distance of the second laser point.

[0172] 2) If the distance of the first laser point is greater than the first threshold and less than or equal to the second threshold, the distance of the third laser point is determined to be the fusion result of the distance of the first laser point and the distance of the second laser point.

[0173] 3) If the distance to the first laser point is greater than the second threshold, the distance to the third laser point is determined to be the distance to the first laser point.

[0174] In this embodiment, the first threshold is the near-range measurement accuracy of the TOF ranging method, for example, the first threshold is 1050mm. When the measurement distance is less than the first threshold, the measurement result of the TOF ranging method has a large error. Therefore, when the distance of the first laser point is less than or equal to the first threshold, the result of the triangulation method (the second laser point) is used as the ranging result of the current angle. That is, the distance of the third laser point is determined to be the distance of the second laser point.

[0175] The second threshold is the long-distance measurement accuracy of the triangulation method, for example, the second threshold is 1150mm. When the measurement distance is greater than the second threshold, the measurement result of the triangulation method has a large error. Therefore, when the distance of the first laser point is greater than the second threshold, the result of the TOF ranging method (first laser point) is used as the ranging result of the current angle. That is, the distance of the third laser point is determined to be the distance of the first laser point.

[0176] If the distance to the first laser point is greater than the first threshold and less than or equal to the second threshold, both the triangulation and Time-of-Flight (TOF) ranging methods have relatively small errors. However, due to the anisotropy of lidar and the susceptibility of TOF ranging to temperature variations, there are discrepancies between the distances measured by the triangulation and TOF methods. To reduce errors and make the measurement between the first and second thresholds more accurate, the results of the triangulation and TOF ranging methods are fused. In other words, the distance to the third laser point is determined as a fusion of the distances to the first and second laser points.

[0177] In some embodiments, the aforementioned "determining the distance of the third laser point as a fusion result of the distance of the first laser point and the distance of the second laser point" specifically includes:

[0178] The distance to the third laser point is calculated using the following formula;

[0179] dis=α*d1+β*d2 (17)

[0180] in,

[0181] Ratio=(d1-d bade) / stride;

[0182] Where dis is the distance to the third laser point, d1 is the distance to the first laser point, d2 is the distance to the second laser point, α and β are the weights, Ratio is the scaling factor, and d bade is the mean of the second and third thresholds, and stride is the difference between the mean and the second or third threshold.

[0183] In this embodiment, based on d base The mean of the second and third thresholds is denoted by , and the stride is the difference between the mean and either the second or third threshold. Both determine the range within which the first and second laser points are weighted and fused. In some embodiments, d base The value can be 1100mm, and the stride can be 50mm. Therefore, the second threshold can be determined to be 1150mm, and the third threshold to be 100mm.

[0184] Based on the scaling factor Ratio, the weighting coefficients α and β are obtained. It can be understood that the inventors of this application established the above-mentioned fusion model, namely formula (17), by analyzing historical data, and weighted the distance of the first laser point and the distance of the second laser point to make the distance of the third laser point more accurate and effectively reduce the error.

[0185] In some embodiments, the method S200 specifically includes:

[0186] (4) If the distance of the first laser point is less than or equal to the third threshold and the second laser point is not displayed, then the distance of the third laser point is determined to be the distance of the first laser point, wherein the third threshold is less than the first threshold.

[0187] Here, the third threshold is the critical value of the near-range blind zone of the triangulation method, for example, 100mm. When the measurement distance is less than the third threshold, the object may enter the blind zone of the triangulation method, and the second receiver may not be able to collect the light spot or may only collect a partial light spot, resulting in no display of the second laser point. In some embodiments, encountering special materials (such as low-reflectivity materials) may also cause the second laser point to not be displayed.

[0188] When the distance to the first laser point is less than or equal to the third threshold, and the second laser point is not displayed, in order to inform the robot that there is an obstacle, the result of the Time-of-Flight (TOF) ranging method (the first laser point) is used as the measurement result of the current angle. That is, the distance to the third laser point is determined to be the distance to the first laser point. Therefore, when facing a target object, it can ensure that corresponding point cloud data is generated, thereby guiding the robot to avoid obstacles.

[0189] It is understandable that after processing the laser points at various angles of the first and second point clouds using the above method, third laser points at each angle are obtained. The third laser points at each angle constitute the optimized point cloud.

[0190] In summary, the point cloud optimization method provided in this application obtains a first point cloud (a point cloud processed by temperature compensation, correction, and smoothing based on TOF ranging) and a second point cloud (a point cloud corrected based on triangulation ranging). It then obtains a first laser point and a second laser point at each angle from the first and second point clouds, respectively. Based on the distances of the first and second laser points, it determines the distance of a third laser point at each angle, thus obtaining the optimized point cloud. This method ensures that the optimized point cloud is a combination of the TOF-compensated point cloud (first point cloud) and the triangulation-based point cloud (second point cloud), leveraging the advantages of both methods while avoiding their disadvantages. Consequently, the third laser point in the optimized point cloud is relatively accurate at various distances, demonstrating high precision across different distances, which is beneficial for the accuracy of ranging or mapping.

[0191] This application also provides a lidar system; please refer to [link / reference]. Figure 5 , Figure 5 This is a schematic diagram of the hardware structure of a lidar provided in an embodiment of this application;

[0192] like Figure 5 As shown, the lidar 300 includes at least one processor 301 and a memory 302 connected in communication. Figure 5 (Taking a bus connection and a single processor as an example).

[0193] The processor 301 provides computing and control capabilities to control the lidar 300 to perform corresponding tasks. In some embodiments, the processor 301 controls the lidar 300 to perform a TOF point cloud processing method according to any of the above method embodiments. This method includes acquiring a raw TOF point cloud, which is point cloud data collected by the lidar's first receiver based on the TOF ranging method. The processor 301 also acquires the lidar's real-time temperature and, based on the real-time temperature, performs distance compensation on each laser point in the raw TOF point cloud to obtain a compensated TOF point cloud.

[0194] This method effectively reduces the impact of temperature drift on the measurement distance, making the TOF compensated point cloud more accurate. In some embodiments, the TOF compensated point cloud is further corrected to obtain a more accurate TOF corrected point cloud. In some embodiments, the TOF corrected point cloud is further smoothed to reduce fluctuations in the point cloud data, making the smoothed TOF corrected point cloud more accurate.

[0195] In some embodiments, the processor 301 controls the lidar 300 to execute the point cloud optimization method in any of the above method embodiments. This method includes: acquiring a first point cloud and a second point cloud, wherein the first point cloud is a point cloud obtained by processing using any of the above-described TOF point cloud processing methods, and the second point cloud is point cloud data acquired by the lidar's second receiver based on triangulation. A first laser point and a second laser point at each angle are respectively acquired from the first and second point clouds; based on the distances of the first and second laser points, the distance of a third laser point at each angle is determined, resulting in an optimized point cloud.

[0196] This method ensures that the optimized point cloud is a combination of the TOF-compensated point cloud (first point cloud) and the triangulation point cloud (second point cloud). It leverages the advantages of both TOF ranging and triangulation ranging while avoiding their disadvantages. As a result, the third laser point in the optimized point cloud is relatively accurate at various distances, achieving high precision at different distances, which is beneficial for the accuracy of ranging or mapping.

[0197] Processor 301 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a hardware chip, or any combination thereof; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The aforementioned PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0198] The memory 302, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to the TOF point cloud processing method in the embodiments of this application, and the program instructions / modules corresponding to the point cloud optimization method in the embodiments of this application. The processor 301, by running the non-transitory software programs, instructions, and modules stored in the memory 302, can implement the TOF point cloud processing method or point cloud optimization method in any of the above-described method embodiments, that is, it can achieve... Figures 3-4 To avoid repetition, the various processes involved will not be described in detail here.

[0199] Specifically, memory 302 may include volatile memory (VM), such as random access memory (RAM); memory 302 may also include non-volatile memory (NVM), such as read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), or other non-transitory solid-state storage devices; memory 302 may also include combinations of the above types of memory.

[0200] In this embodiment, memory 302 may further include memory remotely configured relative to the processor, and this remote memory may be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0201] In this embodiment, the lidar 300 may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The lidar 300 may also include other components for implementing device functions, which will not be elaborated here.

[0202] Please refer to the following: Figure 6 , Figure 6 This is a schematic diagram of the structure of a robot provided in an embodiment of this application;

[0203] like Figure 6 As shown, the robot 400 includes a lidar 300 and a controller 401. The lidar 300 is communicatively connected to the controller 401, which sends ranging commands to the lidar 300 to enable it to perform distance measurement. It is understood that the ranging command can be sent to the robot 400 from an external terminal, and the controller 401 forwards the command to the lidar 300. The external terminal can be a fixed terminal or a mobile terminal, such as a computer, mobile phone, tablet, or other electronic device; no specific limitation is made here.

[0204] It should be noted that the specific hardware structure of the robot can be referred to the content mentioned in the above embodiments, and will not be repeated here.

[0205] This application also provides a computer-readable storage medium, such as a memory including program code, which can be executed by a processor to complete the TOF point cloud processing method or the point cloud optimization method in the above embodiments. For example, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CDROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0206] This application also provides a computer program product comprising one or more lines of program code stored in a computer-readable storage medium. The processor of an electronic device reads the program code from the computer-readable storage medium and executes the program code to complete the method steps of the TOF point cloud processing method provided in the above embodiments, or to complete the point cloud optimization method provided in the above embodiments.

[0207] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware, or by a program or program code related to hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0208] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0209] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software and a general-purpose hardware platform, or of course, using hardware. Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and not to limit them; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of this application as described above, which are not provided in detail for the sake of brevity; although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A point cloud optimization method, characterized in that, include: Acquire a first point cloud and a second point cloud. The first point cloud is a TOF-compensated point cloud, and the second point cloud is point cloud data collected by the second receiver of the lidar based on the triangulation method. The first laser point and the second laser point at each angle are obtained from the first point cloud and the second point cloud respectively. Based on the distance of the first laser point and the distance of the second laser point, the distance of the third laser point at each angle is determined to obtain the optimized point cloud. The step of determining the distance of the third laser point at each angle based on the distances of the first laser point and the second laser point includes: If the distance of the first laser point is greater than the first threshold and less than or equal to the second threshold, the distance of the third laser point is determined to be the fusion result of the distance of the first laser point and the distance of the second laser point; If the distance to the first laser point is less than or equal to the first threshold, the distance to the third laser point is determined to be the distance to the second laser point. If the distance to the first laser point is greater than the second threshold, the distance to the third laser point is determined to be the distance to the first laser point. The determination of the distance of the third laser point as the fusion result of the distances of the first laser point and the second laser point includes: The distance to the third laser point is calculated using the following formula; ； in, ; ; in, The distance to the third laser point. The distance to the first laser point. The distance to the second laser point. and These are the weights, It is a scaling factor. The average of the second and third thresholds. The difference between the mean and the second threshold or the third threshold, wherein the third threshold is the near-range blind zone threshold of the triangulation method.

2. The method according to claim 1, characterized in that, The method further includes: If the distance to the first laser point is less than or equal to the third threshold, and the second laser point is not displayed, then the distance to the third laser point is determined to be the distance to the first laser point, wherein the third threshold is less than the first threshold.

3. The method according to claim 1, characterized in that, The first point cloud is obtained using the following method: Acquire the TOF raw point cloud, which is the point cloud data collected by the first receiver of the lidar based on the TOF ranging method. The TOF raw point cloud includes the measured distance of each laser point. Obtain the real-time temperature of the lidar; Based on the real-time temperature, distance compensation is performed on each laser point in the original TOF point cloud to obtain a TOF compensated point cloud.

4. The method according to claim 3, characterized in that, The step of performing distance compensation on each laser point in the original TOF point cloud based on the real-time temperature to obtain a TOF compensated point cloud includes: For each laser point in the original TOF point cloud, the compensation distance of each laser point is calculated using a preset compensation model, wherein the preset compensation model is a function of the real-time temperature and the preset reference temperature; The compensation distance of each laser point is added to its measurement distance to obtain the TOF compensated point cloud.

5. The method according to claim 3, characterized in that, The method further includes: Distance correction is performed on the TOF compensated point cloud to obtain the TOF corrected point cloud; The step of performing distance correction on the TOF compensated point cloud to obtain the TOF corrected point cloud includes: For each laser point in the TOF compensated point cloud, the corrected distance of each laser point is calculated using a preset correction model to obtain the TOF corrected point cloud, wherein the preset correction model is a function of distance and brightness.

6. The method according to claim 5, characterized in that, The method further includes: The TOF-corrected point cloud is smoothed. The smoothing process for the TOF-corrected point cloud includes: Taking each laser point in the TOF corrected point cloud as the starting point, and taking m laser points with consecutive angles as a smoothing unit, multiple smoothing units are obtained, where m is an integer greater than or equal to 5. For each laser point in the smoothing unit, coordinate smoothing is performed based on the coordinates of multiple laser points near the laser point to obtain the smoothed coordinates.

7. A lidar, characterized in that, At least one processor, and A memory communicatively connected to the at least one processor, wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method according to any one of claims 1-6.

8. A robot, characterized in that, Including the lidar as described in claim 7.

Images