A time-division imaging lidar ranging method and system

Abstract

This invention discloses a time-division imaging lidar ranging method and system. The method is applied to lidar ranging, where the lidar has a front laser and two side lasers, with polarizers positioned on both sides of a camera. This lidar ranging method uses a time-division imaging time-axis strategy to control the activation of the front and side lasers, as well as the image processing by the data processor, thereby obtaining the positions of the front and side laser lines, the distances to the front and side targets, and finally fusing and stitching the time-division imaging front and side laser images to obtain a wide-angle laser image. This invention achieves time allocation between the camera sending the image to the data processor and the data processor performing image processing, ensuring that the two processes do not conflict in time. When the camera sends an image, the data processor receives the image; when the camera captures the next frame, the data processor processes the currently received image.

Description

Technical Field

[0001] This invention belongs to the field of radar data processing technology, specifically relating to a time-division imaging lidar ranging method and system. Background Technology

[0002] LiDAR ranging technology based on triangulation emits a laser beam and receives the reflected light signal using a receiver. Combining the angle between the sensor and the target object with a known baseline length, it calculates the distance using the principles of triangle geometry. Specifically, when the LiDAR sensor emits a laser pulse and receives the reflected signal, the precise distance can be derived by measuring the angle change between the laser beam and the receiver and using trigonometric functions. Compared to traditional time-of-flight (ToF) ranging, this method offers higher spatial resolution and accuracy, especially in dynamic scenarios requiring the measurement of the relative position of target objects, providing more accurate three-dimensional ranging data. Therefore, triangulation ranging has wide applications in autonomous driving, robot navigation, and building surveying, effectively improving ranging accuracy and environmental perception capabilities.

[0003] However, since the field of view (FOV) of a lidar system is typically determined by the laser emission angle and scanning range of the sensor, many lidar devices have a relatively narrow FOV. This means they can only perceive objects within a limited area, and ranging accuracy may be lower, especially in low-angle or localized areas. This narrow FOV limits the comprehensive coverage capability of lidar, particularly in complex urban environments or dynamic scenes, easily creating blind spots that prevent timely detection of certain obstacles, thus affecting the overall perception capability of the system. Even with a wider FOV, many traditional imaging systems typically optimize spatially by increasing the number of sensors to improve the image viewing angle. However, this approach can lead to a sharp increase in system cost, or an increase in system size and power consumption. Summary of the Invention

[0004] To address the problems of incomplete detection or excessively high system costs caused by small field-of-view perception in existing technologies, this invention provides a time-division imaging lidar ranging method and system. Specifically, the time-division imaging lidar ranging method is applied to a lidar system. This lidar system has polarizers on both sides of the camera, causing the lasers emitted by the lasers on both sides to be refracted by the polarizers and captured by the camera, effectively expanding the field of view and solving the problems of blind spots and small field of view inherent in existing technologies. Furthermore, this invention also achieves time-division imaging, i.e., acquiring the forward laser image in the first moment and the laser images from both sides in the second moment, and then merging the two images to obtain a wide-angle field of view. Simultaneously, this system uses only one camera, combined with the time-division imaging method, to achieve a wide-angle field of view, solving the problems of drastically increased costs, system size, and power consumption.

[0005] Therefore, the present invention provides the following technical solution:

[0006] On the one hand, the present invention provides a time-division imaging lidar ranging method, which is applied to lidar ranging. The lidar is equipped with a front laser and two side lasers, and polarizers are respectively set on both sides of the lidar camera, so that the laser emitted by the two side lasers is refracted by the polarizers and captured by the lidar camera.

[0007] The lidar ranging method is based on the time axis strategy of time-division imaging to control the activation of the front laser and the two side lasers and the image processing of the data processor, thereby obtaining the position of the front laser line, the position of the two side laser lines, the distance to the front target, and the distance to the two side targets. Then, the time-division imaging of the front laser image and the two side laser images are fused and stitched together to obtain a wide-angle laser image.

[0008] The time-axis strategy based on time-division imaging is as follows:

[0009] The two side lasers and the front laser are activated at staggered times. That is, when the data processor uses the front laser image to identify the distance to the front target, the two side lasers are activated and the front laser is deactivated; when the data processor uses the two side laser images to identify the distance to the targets on both sides, the front laser is activated and the two side lasers are deactivated; and when the camera of the lidar sends a laser image, the data processor receives the laser image, and when the camera captures the next frame, the data processor processes the currently received laser image.

[0010] Preferably, the process of controlling the activation of the front laser, the two side lasers, and the image processing of the data processor based on the time-division imaging time-axis strategy is as follows:

[0011] First, the two side lasers are turned off, the front laser is turned on, and the camera captures the front laser image and transmits it to the data processor.

[0012] Next, after the front laser image is transmitted, the data processor identifies the distance to the front target based on the front laser image; at the same time, the front laser is turned off, the two side lasers are turned on, and the camera captures the laser images of the two side lasers.

[0013] Furthermore, the laser images from both sides are transmitted to the data processor;

[0014] Finally, after the transmission of the laser images on both sides is completed, the data processor identifies the distance between the targets on both sides based on the laser images on both sides. At the same time, the lasers on both sides are turned off, the front laser is turned on, and the camera captures the front laser image of the front laser.

[0015] Repeat the above process to perform cross-shooting and distance recognition.

[0016] Preferably, before the laser radar ranging method performs ranging, front-side calibration and lateral calibration are performed, specifically as follows:

[0017] When performing the front calibration or the two-sided calibration, the front laser or the two-sided laser of the lidar is set to the corresponding calibration position, and the camera then captures the laser image. The calibration position is the maximum irradiation distance of the laser.

[0018] Traverse the laser line positions of each column in the laser image. If a column has no laser line, fill the position of the laser line. That is, find the first and last non-zero columns and determine the laser line position in the corresponding columns. Then, use two points to calculate the laser line slope k and constant b in the following formula, and then calculate the filling position of the laser line in each column with no laser line.

[0019] y = kx + b

[0020] In the formula, y represents the position information of the laser line in each column, and x represents the column number of the laser image;

[0021] After the filling is completed, the laser line fills the entire image, thus obtaining the position of the laser line in the laser image, which is regarded as obtaining the position of the baseline;

[0022] The baseline is a reference line for the ranging process. During the ranging process, the pixel difference between the laser line and the corresponding baseline in the front laser image and the laser images on both sides, as well as the maximum illumination distance corresponding to the baseline, are used to determine the front target distance and the target distance on both sides corresponding to the front laser image and the laser images on both sides.

[0023] The preferred embodiment of the present invention also takes into account that due to the installation deviation of the laser, the distance d between the laser and the camera may be different, which may lead to inconsistent baselines and inaccurate subsequent distance measurements. Therefore, calibration is required before measuring the distance, that is, front calibration and side calibration are performed separately. In addition, it is also considered that the laser line may not fill the screen during the calibration process. Therefore, the slope of the laser line is calculated to fill in the laser line that does not fill the screen, effectively solving the application problem.

[0024] Preferably, when the data processor identifies the distance to a target in front using the front laser image or identifies the distance to targets on both sides using the laser images on both sides, it determines the position of the laser line in the laser image using the following process:

[0025] First, a brightness threshold is set. If the gray value of a pixel is greater than the brightness threshold, the position information of the pixel is saved. The position information is the y-axis coordinate of the pixel, i.e., the row number.

[0026] Then, the position information saved in each column is filtered accordingly, that is, the position information of the pixel with the largest gray value in each column is used as the laser line position of the corresponding column, or the median of the position information of several pixels with the largest gray value in each column is used as the laser line position of the corresponding column.

[0027] This allows us to obtain the position of the laser line in the laser image.

[0028] Preferably, when the data processor identifies the distance to the front target using the front laser image or the distance to the targets on both sides using the laser images on both sides, it uses the pixel deviation between the laser line position and the baseline in the laser image to look up the pre-stored distance table to obtain the distance to the front target or the distance to the targets on both sides.

[0029] The index of the pre-stored distance table is the pixel deviation between the laser line position and the baseline, and the content is the distance value. The baseline is the reference line for the ranging process, and the distances to the front target and the two side targets corresponding to the baseline are known parameters.

[0030] Preferably, the lidar ranging method further includes: adjusting the brightness of the laser according to the thickness of the laser line;

[0031] The thicker the laser line, the closer the laser is to the target, which tends to reduce the laser brightness; the thinner the laser line, the farther the laser is from the target, which tends to increase the laser brightness.

[0032] The preferred embodiment of the present invention can also realize the function of adaptively changing the brightness of the laser. When the laser radar is far away from the obstacle, the laser brightness will increase, and when it is close to the obstacle, the laser brightness will decrease. This can avoid the phenomenon of light scattering when the laser radar is close to the obstacle, which would lead to a decrease in measurement accuracy.

[0033] Secondly, the present invention provides a ranging system based on the above method, comprising: a lidar and a data processor disposed within the lidar or an external data processor connected to the lidar;

[0034] The lidar is equipped with at least a front laser, two side lasers, and a camera. Polarizers are respectively set on both sides of the camera of the lidar, so that the laser emitted by the two side lasers is refracted by the polarizers and captured by the camera of the lidar.

[0035] Preferably, both the lidar and the data processor are located inside the robot vacuum cleaner.

[0036] In addition, the present invention provides a computer-readable storage medium based on the above method, which stores a computer program that is called by a processor to implement the steps of a time-division imaging lidar ranging method.

[0037] The activation of the front laser and the two side lasers is controlled by the time axis strategy of time-division imaging, and the image processing of the data processor is carried out to obtain the position of the front laser line, the position of the two side laser lines, the distance of the front target, and the distance of the two side targets. Then, the front laser image and the two side laser images of time-division imaging are fused and stitched to obtain a wide-angle laser image.

[0038] Specifically, the system controls the lasers on both sides to be turned on at different times than the laser on the front side. That is, when the data processor uses the laser image on the front side to identify the distance to the target on the front side, it controls the lasers on both sides to be turned on and the laser on the front side to be turned off; when the data processor uses the laser images on both sides to identify the distance to the targets on both sides, it controls the laser on the front side to be turned on and the lasers on both sides to be turned off; and when the camera of the lidar sends a laser image, the data processor receives the laser image, and when the camera captures the next frame, the data processor processes the currently received laser image.

[0039] Beneficial effects

[0040] Compared with existing methods, the advantages of the present invention are:

[0041] 1. Since cameras can only capture a 90-degree field of view directly in front, they cannot measure a wide field of view. This invention addresses this limitation by adding two polarizers in front of the camera. These polarizers, through the principle of light refraction, enable the camera to capture a wider field of view. This simple and effective solution solves the technical limitations and defects of small field-of-view perception.

[0042] 2. The technical solution of this invention adopts a time-sharing strategy based on time-division imaging to control the activation of the front laser, the two side lasers, and the image processing of the data processor. That is, the two side lasers are controlled to activate at different times than the front laser. After the camera of the lidar captures the front laser image or the two side laser images, it transmits them to the data processor. After the transmission is completed, when the data processor uses the front laser image to identify the distance of the front target, it controls the two side lasers to activate; when the data processor uses the two side laser images to identify the distance of the two side targets, it controls the front laser to activate.

[0043] 3. This invention fully considers that data processing requires operation on the entire image. Therefore, data processing must be performed only after the entire image is received. After data processing, data must be received immediately to avoid data loss. Therefore, this invention proposes the above strategy, which reasonably arranges the time for receiving and processing images, so as to receive complete image information without losing a single frame, and can process the previous frame data in a timely manner during the camera's image capture, thereby improving the frame rate.

[0044] Among the advantages of time-sharing imaging are:

[0045] Traditional imaging systems typically optimize spatially by increasing the number of sensors or improving resolution to enhance image quality. However, this approach can lead to a sharp increase in system cost, size, and power consumption. Time-division imaging, on the other hand, allows the system to alternate between different viewpoints at different times. This effectively improves the spatial resolution of the image while reducing hardware investment.

[0046] Current technologies do not adequately consider the temporal resolution of image processing. In applications with extremely high real-time requirements, the response speed of the imaging system is crucial. Without time-division imaging, the system may struggle to acquire enough image data within a limited timeframe due to hardware limitations. Time-division imaging can improve image acquisition efficiency by optimizing the sensor's duty cycle.

[0047] With the development of imaging technology, the amount of data acquired by imaging devices is becoming increasingly massive. Traditional imaging systems have not fully considered how to effectively process this massive amount of data, leading to problems such as slow processing speed and high storage pressure. In systems that do not employ time-sharing imaging, data flow is typically continuous and uninterrupted, making flexible scheduling difficult. Time-sharing imaging can control the working hours of the imaging device, rationally allocating the timing of data acquisition and processing. This reduces fluctuations in data flow and avoids delays or data loss due to system overload during peak processing periods. Furthermore, time-sharing imaging can optimize data storage and transmission paths by processing data from different time periods in batches, thereby improving processing efficiency. Attached Figure Description

[0048] Figure 1 This is a schematic diagram of the operation of the lidar provided in the embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the laser image provided by the present invention;

[0050] Figure 3 This is a schematic diagram of time-division imaging provided by the present invention;

[0051] Figure 4 This is a schematic diagram of light scattering provided by the present invention;

[0052] Figure 5 This is a schematic diagram of the installation deviation provided by the present invention;

[0053] Figure 6 This is a flowchart of a time-division imaging lidar ranging method provided by the present invention;

[0054] Figure 7 This is a calibration flowchart provided by the present invention;

[0055] Figure 8 This is a schematic diagram of the laser brightness adjustment process provided by the present invention. Detailed Implementation

[0056] This invention provides a time-division imaging lidar ranging method, applicable to lidar ranging. The invention will be further described below with reference to embodiments.

[0057] Since a camera can only capture a 90-degree field of view directly in front, it cannot measure a large field of view. Therefore, the technical solution of this invention optimizes the structure of the lidar by placing polarizers on both sides of the lidar's camera, causing the laser emitted by the lasers on both sides to be refracted by the polarizers and captured by the lidar's camera. Figure 1As shown, the lidar includes a front laser and two side lasers. Polarizers are positioned on both sides of the lidar's camera. These polarizers, through the principle of light refraction, enable the camera to capture a wider field of view. The size and position of the polarizers can be adjusted adaptively. This invention only constrains them to meet the requirement that the laser emitted by the side lasers is refracted by the polarizers and captured by the lidar's camera, thus expanding the field of view, and preferably without affecting the laser propagation of the front laser.

[0058] Based on the aforementioned lidar, the present invention provides a lidar ranging method using time-division imaging. This method controls the activation of the front and side lasers and the image processing of the data processor using a time-division imaging time-axis strategy. This yields the positions of the front and side laser lines, as well as the distances to the front and side targets. Finally, the front and side laser images from the time-division imaging are fused and stitched together to obtain a wide-angle laser image (preferably containing both the laser line image and the distance information to the front / side targets). The time-division imaging time-axis strategy is as follows:

[0059] The side lasers and the front laser are activated at staggered times. Specifically, when the data processor uses the front laser image to identify the distance to the target in front, the side lasers are activated, and the front laser is deactivated; conversely, when the data processor uses the side laser images to identify the distance to targets on both sides, the front laser is activated, and the side lasers are deactivated. Furthermore, when the lidar's camera transmits a laser image, the data processor receives the laser image; when the camera captures the next frame, the data processor processes the currently received laser image. This system ensures that when the lidar's camera transmits a laser image, the data processor receives the laser image; and when the camera captures the next frame, the data processor processes the currently received laser image.

[0060] Specifically as follows:

[0061] First, the front laser is activated to emit a front laser beam. When the laser encounters an object, it is reflected and captured by the camera, resulting in a front laser image. An example image of the laser image is attached. Figure 2 The white line in the diagram is the laser line.

[0062] Then, the front laser image is sent to the microcontroller (data processor, preferably a microcontroller in this invention), and the microcontroller receives the front image data.

[0063] After transmission is complete, the two side lasers are turned on, while the front laser is turned off. When the laser encounters an object, it is reflected by a polarizer, and then refracted by the polarizer before being captured by the camera as images of the two side lasers. At this time, the camera is capturing images of the two side lasers. Simultaneously, the microcontroller extracts the position of the laser line in the image from the front laser image after transmission and converts the position information into distance information to obtain the distance to the front target.

[0064] Once the camera has captured the laser images from both sides, it sends these images to the microcontroller, which receives them. After transmission, the lasers on both sides are turned off, while the front laser is turned on. The camera then captures the front laser image. Simultaneously, the microcontroller extracts the position of the laser line within the image from both sides and converts this position information into distance information to obtain the distance to the targets on both sides. This process is repeated continuously.

[0065] In summary, after obtaining the distances to the front target and the targets on both sides, the distance information of the front laser line and the laser lines on both sides is combined. The distance information of the left laser line is placed on the left as the distance information between the device and the target on the left, the distance information of the middle laser line is placed in the middle as the distance information between the device and the target in front, and the distance information of the right laser line is placed on the right as the distance information between the device and the target on the right. This captures distance information with a wide field of view, realizes the time-division imaging function, and obtains distance information with a large field of view. The principle is as follows. Figure 3 As shown.

[0066] Based on the above theoretical statements, the following Example 1 is a preferred example of a time-division imaging lidar ranging method provided by the present invention.

[0067] Example 1:

[0068] The process of a time-division imaging lidar ranging method provided in this embodiment of the invention is as follows:

[0069] S1: Calibration and establishment of distance tables.

[0070] Due to laser installation deviation, i.e. Figure 5 The varying distance *d* between the laser and the camera can lead to inconsistent baselines, resulting in inaccurate distance measurements. Therefore, calibration is necessary before distance measurement. This embodiment provides an automatic calibration function. By placing the lidar at the default calibration position (maximum laser measurement distance), a calibrated baseline corresponding to the maximum measurement distance can be automatically obtained. Specifically, front and side calibrations are performed:

[0071] S11: When performing front-side calibration or side-side calibration, the front laser or side lasers of the lidar are set to the calibration position respectively, and the camera then captures the laser image. The calibration position is the maximum illumination distance of the laser. The lidar in this embodiment can measure a maximum distance of 400mm, so 400mm is used as the reference point in this example.

[0072] S12: First, iterate through the laser line position information of each column of the image. If the value is zero, it means that there is no laser line in that column and it needs to be added. Continue until the first non-zero column is found, and save the laser line position information and column number of that column. Then find the last non-zero column and save the laser line position information and column number of that column as well. Use these two points and the formula below to calculate the slope and constant b. The slope is the slope of the laser line. Then, add the laser line position information to the columns that have no laser lines according to this formula. The process is as follows. Figure 7 As shown.

[0073] y = kx + b

[0074] In the formula, y represents the position information of the laser line in each column, x represents the column number of the laser image, k represents the slope of the laser line, and b is a constant;

[0075] S13: After filling, the laser line fills the entire image, thus obtaining the position of the laser line in the laser image, which is considered as obtaining the position information of the baseline. Then, a distance table is established according to the triangulation method. The distance table is an array, where the array index is the pixel difference between the baseline of the laser line (position information of the 400mm laser line) and the actual measured position information of the laser line, and the content is the distance value to the target. It should be understood that establishing a distance table based on the triangulation method is existing technology, therefore it will not be described in detail.

[0076] It should be noted that step S12 is the preferred method in this embodiment, which fully considers the possibility that the length of the laser line may not fill the entire screen during application. Therefore, the slope of the laser line is calculated to fill in the gaps in the screen. The identification of the laser line position in the laser image will be described in detail below.

[0077] S2: Based on the time-sharing imaging time axis strategy, control the activation of the front laser, the lasers on both sides, and the image processing of the data processor.

[0078] First, the front laser is turned on, and the side lasers are turned off. The camera captures the image from the front laser and sends the image information to the microcontroller. After this, the side lasers are turned on, the front laser is turned off, and the camera begins to capture the next frame (during this period, image capture is performed, and image information is not sent to the microcontroller). During this period, the microcontroller processes the received front laser image information and finds the position information of the laser in the image. This achieves a time allocation between the camera sending the image to the microcontroller and the microcontroller processing the image, ensuring that the two do not conflict in time. When the camera sends the image, the microcontroller receives the image; when the camera captures the next frame, the microcontroller processes the currently received image.

[0079] Since there is a time interval between the camera sending image data and the camera starting to capture images (approximately 50ms), this embodiment preferably sets a flag bit. This flag bit increments by 1 every 1ms, and is cleared to 0 when the microcontroller receives data. As long as the flag bit remains 0, it indicates that the microcontroller is in the image receiving phase. When the flag bit is greater than 5, meaning no data has been received for 5ms, it can be determined that the camera has finished sending the entire image data, and the microcontroller can then process the received image data. It should be understood that the 5ms interval is self-set; in other feasible embodiments, it can be adaptively adjusted according to application requirements.

[0080] The technical solution of this invention fully considers that data processing requires operation on the entire image. Therefore, data processing must be performed only after the entire image is received, and data reception must begin immediately after data processing to prevent data loss. This fully utilizes hardware resources, rationally arranges the time for image reception and image processing, ensures complete reception of image information without losing a single frame, and enables timely processing of the previous frame's data during image capture by the camera, thereby improving the frame rate.

[0081] The process by which the microcontroller identifies laser lines in a laser image in this embodiment is as follows:

[0082] 1. Perform this operation on each column and each row of the laser image. Since the captured image is a black and white image, and the laser line has the highest brightness, while the rest of the image is basically black, a brightness threshold (a grayscale value ranging from 0 to 255) can be set. For any pixel whose grayscale value is greater than the brightness threshold, the corresponding position information of that pixel is stored. This position information is the y-axis coordinate of the pixel in the image, i.e., the row number. Because the rest of the image is basically black except for the laser line, its pixel value cannot be greater than the brightness threshold, while the brightness value of the laser line can be greater than the brightness threshold. Thus, the position information of the laser line can be found.

[0083] 2. Then, filter the position information saved in each column accordingly. Since the laser line has the highest brightness value in the whole image, first extract the position information of the pixel with the highest brightness value and locate the pixel on the laser line. Since the laser line is thick, each column may have multiple pixels with the highest brightness value. Take the position information of all these pixels and take the median value of these position information as the laser line position information of this column of the image.

[0084] 3. After operating on all columns, the position information of the front laser line in the image is obtained.

[0085] 4. Then, iterate through each column, subtract the laser line position information of each column from the pixel value of the pixel point corresponding to the baseline to obtain the pixel deviation, and convert the front laser position information into front distance information by looking up the distance table (the index and content of the distance table are applicable to the pixel difference of each column).

[0086] It should be understood that the camera captures the laser images from both sides and sends this information to the microcontroller. After transmission, the side lasers are turned off, the front laser is turned on, and the camera captures the front laser image again, repeating this cycle. Simultaneously, the microcontroller identifies the position information of the laser lines on both sides and obtains the distance information by looking up a distance table. Then, the front distance information is combined with the side distance information. The distance information of the left laser line is placed on the left side as the distance between the device and the target on the left, the distance information of the middle laser line is placed in the middle as the distance between the device and the target in front, and the distance information of the right laser line is placed on the right as the distance between the device and the target on the right. The specific process is as follows: Figure 6 As shown.

[0087] In some embodiments, the present invention also implements an adaptive function to change the laser brightness. When the lidar is far from the obstacle, the laser brightness increases; when it is close to the obstacle, the laser brightness decreases. This avoids light scattering when the lidar is close to the obstacle, which would reduce measurement accuracy. The light scattering phenomenon is as follows: Figure 4 .

[0088] In this embodiment, it is preferable to record the number of pixels greater than a threshold in each column of the image, and extract the maximum number of pixels in each column to represent the thickness of the laser line. The image captured by the camera has a pixel value of 240×320, meaning the image has 240 lines. Therefore, if the laser line fills less than 30 lines of the image, it can be considered thin, and the laser brightness needs to be increased. If the laser line fills more than 50 lines of the image, it can be considered thick, and the laser brightness needs to be decreased. If the laser line fills more than 30 lines but less than 50 lines of the image, it can be considered just right, and the laser brightness remains unchanged. The process is as follows... Figure 8 As shown. It should be noted that in practical applications, the standard for judging the thickness of the laser line is adjusted according to application requirements and accuracy requirements, and is an empirical value. This invention does not impose specific limitations or constraints on it.

[0089] In some embodiments, the present invention also provides a ranging system based on the above ranging method, the ranging system comprising: a lidar and a data processor disposed within the lidar or an external data processor connected to the lidar;

[0090] The lidar is equipped with at least a front laser, two side lasers, and a camera. Polarizers are set on both sides of the lidar camera so that the laser emitted by the two side lasers is refracted by the polarizers and captured by the lidar camera.

[0091] Preferably, both the lidar and the data processor are located within the robotic vacuum cleaner. That is, the technical solution of this invention is applied to a robotic vacuum cleaner. Furthermore, the data processor is preferably a microcontroller.

[0092] In some embodiments, the present invention also provides a computer-readable storage medium based on the above-described ranging method, storing a computer program that is called by a processor to implement the steps of a time-division imaging lidar ranging method. The implementation process of Embodiment 1 can be referred to.

[0093] The system employs a time-division imaging strategy to control the activation of the front and side lasers, as well as image processing by the data processor. This yields the positions of the front and side laser lines, the distances to the front and side targets, and the distances to the targets on both sides. The front and side laser images from the time-division imaging are then fused and stitched together to obtain a wide-angle laser image. Specifically, the system controls the side lasers to activate at staggered times compared to the front laser. After the lidar's camera captures either the front or side laser image, it transmits it to the data processor. Once transmission is complete, the data processor activates the side lasers when it uses the front laser image to identify the distance to the front target, and vice versa. This system enables the data processor to receive laser images as the lidar's camera transmits them, and processes the currently received laser image as the camera captures the next frame.

[0094] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the hardware and software device described in any of the foregoing embodiments, such as the hard drive or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both internal storage units and external storage devices of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0095] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0096] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0097] It should be emphasized that the examples of this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A time-division imaging lidar ranging method, characterized in that: This technology is applied to lidar ranging. The lidar is equipped with a front laser and two side lasers, and polarizers are respectively set on both sides of the lidar camera, so that the laser emitted by the two side lasers is refracted by the polarizers and captured by the lidar camera. The lidar ranging method is based on the time axis strategy of time-division imaging to control the activation of the front laser and the two side lasers and the image processing of the data processor, thereby obtaining the position of the front laser line, the position of the two side laser lines, the distance to the front target, and the distance to the two side targets. Then, the time-division imaging of the front laser image and the two side laser images are fused and stitched together to obtain a wide-angle laser image. The time-axis strategy based on time-division imaging is as follows: The lasers on both sides are turned on at staggered times with the laser on the front side. That is, when the data processor uses the laser image on the front side to identify the distance to the target on the front side, the lasers on both sides are turned on and the laser on the front side is turned off; when the data processor uses the laser images on both sides to identify the distance to the targets on both sides, the laser on the front side is turned on and the lasers on both sides are turned off; and when the camera of the lidar sends a laser image, the data processor receives the laser image, and when the camera captures the next frame, the data processor processes the currently received laser image. Before the laser radar ranging method performs ranging, it first performs front-side calibration and lateral calibration, specifically as follows: When performing the front calibration or the two-sided calibration, the front laser or the two-sided laser of the lidar is set to the corresponding calibration position, and the camera then captures the laser image. The calibration position is the maximum irradiation distance of the laser. Traverse the laser line positions of each column in the laser image. If a column has no laser line, fill the position of the laser line. That is, find the first and last non-zero columns and determine the laser line position in the corresponding columns. Then, use two points to calculate the laser line slope k and constant b in the following formula, and then calculate the filling position of the laser line in each column with no laser line. y = kx + b In the formula, y represents the position information of the laser line in each column, and x represents the column number of the laser image; After the filling is completed, the laser line fills the entire image, thus obtaining the position of the laser line in the laser image, which is regarded as obtaining the position of the baseline; The baseline is a reference line for the ranging process. During the ranging process, the pixel difference between the laser line and the corresponding baseline in the front laser image and the laser images on both sides, as well as the maximum illumination distance corresponding to the baseline, are used to determine the front target distance and the target distance on both sides corresponding to the front laser image and the laser images on both sides.

2. The method according to claim 1, characterized in that: The process of controlling the activation of the front laser, the two side lasers, and the image processing of the data processor based on the time-sharing imaging time-axis strategy is as follows: First, the two side lasers are turned off, the front laser is turned on, and the camera captures the front laser image and transmits it to the data processor. Next, after the front laser image is transmitted, the data processor identifies the distance to the front target based on the front laser image; at the same time, the front laser is turned off, the two side lasers are turned on, and the camera captures the laser images of the two side lasers. Furthermore, the laser images from both sides are transmitted to the data processor; Finally, after the transmission of the laser images on both sides is completed, the data processor identifies the distance between the targets on both sides based on the laser images on both sides. At the same time, the lasers on both sides are turned off, the front laser is turned on, and the camera captures the front laser image of the front laser. Repeat the above process to perform cross-shooting and distance recognition.

3. The method according to claim 1, characterized in that: When the data processor identifies the distance to a target in front using the front laser image or identifies the distance to targets on both sides using the laser images on both sides, it determines the position of the laser line in the laser image using the following process: First, a brightness threshold is set. If the gray value of a pixel is greater than the brightness threshold, the position information of the pixel is saved. The position information is the y-axis coordinate of the pixel, i.e., the row number. Then, the position information saved in each column is filtered accordingly, that is, the position information of the pixel with the largest gray value in each column is used as the laser line position of the corresponding column, or the median of the position information of several pixels with the largest gray value in each column is used as the laser line position of the corresponding column. This allows us to obtain the position of the laser line in the laser image.

4. The method according to claim 3, characterized in that: When the data processor identifies the distance to the front target using the front laser image or the distance to the targets on both sides using the laser images on both sides, it uses the pixel deviation between the laser line position and the baseline in the laser image to look up the pre-stored distance table to obtain the distance to the front target or the distance to the targets on both sides. The index of the pre-stored distance table is the pixel deviation between the laser line position and the baseline, and the content is the distance value. The baseline is the reference line for the ranging process, and the distances to the front target and the two side targets corresponding to the baseline are known parameters.

5. The method according to claim 1, characterized in that: The lidar ranging method further includes: adjusting the brightness of the laser according to the thickness of the laser line; The thicker the laser line, the closer the laser is to the target, which tends to reduce the laser brightness; the thinner the laser line, the farther the laser is from the target, which tends to increase the laser brightness.

6. A ranging system based on the method of any one of claims 1-5, characterized in that: Includes: a lidar and a data processor disposed within the lidar or an external data processor connected to the lidar; The lidar is equipped with at least a front laser, two side lasers, and a camera. Polarizers are respectively set on both sides of the camera of the lidar, so that the laser emitted by the two side lasers is refracted by the polarizers and captured by the camera of the lidar.

7. The ranging system according to claim 6, characterized in that: Both the lidar and the data processor are located inside the robot vacuum cleaner.

8. A computer-readable storage medium based on the method of any one of claims 1-5, characterized in that: A computer program is stored, which is called by a processor to implement the steps of a time-division imaging lidar ranging method; The activation of the front laser and the two side lasers is controlled by the time axis strategy of time-division imaging, and the image processing of the data processor is carried out to obtain the position of the front laser line, the position of the two side laser lines, the distance of the front target, and the distance of the two side targets. Then, the front laser image and the two side laser images of time-division imaging are fused and stitched to obtain a wide-angle laser image. Specifically, the system controls the lasers on both sides to be turned on at staggered times with the laser on the front side. That is, when the data processor uses the laser image on the front side to identify the distance to the target on the front side, it controls the lasers on both sides to be turned on and the lasers on the front side to be turned off; when the data processor uses the laser images on both sides to identify the distance to the targets on both sides, it controls the laser on the front side to be turned on and the lasers on both sides to be turned off; and when the camera of the lidar sends a laser image, the data processor receives the laser image, and when the camera captures the next frame, the data processor processes the currently received laser image.

Images