Method and device for detecting diaphragm of laser radar and storage medium
By analyzing the point cloud data of the lidar and identifying the occlusion area of the window sheet, the problem of degradation of detection performance caused by the contamination of the lidar window sheet is solved, real-time detection and early warning of dirty occlusion areas is achieved, and the safety of autonomous driving is improved.
Patent Information
- Application Number
- CN202311614306.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-28
- Publication Date
- 2025-05-30
AI Technical Summary
In adverse weather and complex road conditions, the lidar windows are easily contaminated, resulting in reduced detection performance or failure, affecting the safety of autonomous driving.
By obtaining the point cloud data of the lidar, extracting the distance information and reflectivity information of two adjacent data frames, calculating the hollow area data, identifying and marking the occlusion area of the window slice, real-time detection and early warning are achieved.
Effectively identify and mark the dirty blocking areas of the window sheet to prevent missed or mis-checked obstacles from lidar equipment, and improve the safety of autonomous driving.
Smart Images
Figure CN120065182A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of lidar, and in particular, to a method, device, and storage medium for detecting a window sheet of a lidar. Background Art
[0002] A lidar is an active remote sensing device that uses optoelectronic detection technology. Its working principle is to emit a detection signal to a target and then receive and process the echo signal to obtain information such as the distance, speed, and reflectivity of the target. It has the advantages of high resolution, high sensitivity, strong anti-interference ability, and being unaffected by dark conditions, and is widely used in fields such as autonomous driving, logistics vehicles, robots, vehicle-road coordination, and public intelligent transportation. However, during the operation of an in-vehicle lidar, it is inevitable that the window sheet of the lidar will be contaminated due to bad weather and complex road conditions, resulting in a reduction or even complete failure of the lidar detection performance, and thus it is impossible to normally obtain the surrounding environmental information, seriously affecting the safety level of autonomous driving. Therefore, it is necessary to perform real-time detection on the window sheet and issue a warning when it is dirty and blocked, to avoid missed detection and misdetection of obstacles by the lidar device.
[0003] Existing methods for detecting the dirt and blockage of the lidar window sheet are mainly implemented based on a hardware system: one is to use a fixed window sheet dirt detection device, the disadvantage of which is that the detection device cannot change the detection position correspondingly as the scanning position of the lidar changes, and the detection range of the window sheet is limited; the other is to add an additional light-emitting device, but there are problems such as a relatively complex device structure and low detection accuracy. In actual operation, when the window sheet is blocked, the dirty area will present corresponding point cloud features on the continuous dynamic point cloud map. Summary of the Invention
[0004] The embodiments of the present application provide a method, device, and storage medium for detecting a window sheet of a lidar, aiming to detect the situation of the window sheet in real time based on the point cloud signal in the case of the absence of the lidar leading signal, and identify and display the dirty and blocked area.
[0005] In a first aspect, the embodiments of the present application provide a method for detecting a window sheet of a lidar, the method including the following steps:
[0006] Obtain the point cloud data of the lidar;
[0007] Obtain a plurality of feature information of two adjacent data frames according to the point cloud data, where the plurality of feature information includes distance information and reflectivity information;
[0008] Obtain the data of the void area of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames;
[0009] Obtain the data of the occluded area of the window slice according to the data of the void areas of the two adjacent data frames;
[0010] Mark the data of the occluded area.
[0011] In some embodiments, the two adjacent data frames include a current data frame and a previous data frame, and obtaining the data of the void areas of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames includes:
[0012] Obtain the binary matrix of the set area of the current data frame according to the distance information of the current data frame and the reflectivity information of the current data frame, where the set area is composed of data points with both distance values and reflectivity values being zero; obtain the binary matrix of the set area of the previous data frame according to the distance information of the previous data frame and the reflectivity information of the previous data frame; perform opening operation and closing operation on the binary matrix of the set area of the current data frame to obtain the data of the void area of the current data frame; perform opening operation and closing operation on the binary matrix of the set area of the previous data frame to obtain the data of the void area of the previous data frame. The opening operation and the closing operation are to move a structuring element (filter window) in the image, and then perform set operations such as intersection and union on the structuring element and the following binary image. The opening operation has the functions of eliminating small objects, separating objects at thin places, and smoothing the boundaries of larger objects. The closing operation has the functions of filling small voids in objects, connecting adjacent objects, and smoothing the boundaries.
[0013] In some embodiments, the two adjacent data frames include a current data frame and a previous data frame, and obtaining the data of the occluded area of the window slice according to the data of the void areas of the two adjacent data frames includes: using the data of the void area of the current data frame as the data of the occluded area of the window slice, where the Hamming distance between the two adjacent data frames is less than a first preset value, and the valid frame count value is greater than or equal to a second preset value. The Hamming distance between two adjacent data frames being less than the first preset value ensures the validity of the void area, and the limitation of the second preset value on the valid frame count value increases the credibility of the occluded area data. The combination of the two can improve the accuracy of the judgment result, thereby preventing false alarms caused by short-distance targets.
[0014] In some embodiments, obtaining the data of the occluded area of the window slice according to the data of the void areas of the two adjacent data frames further includes:
[0015] Obtain the Hamming distance matrix of the hole regions of the two adjacent data frames according to the data of the hole region of the previous data frame and the data of the hole region of the current data frame; sum according to the Hamming distance matrix to obtain the Hamming distance between the two adjacent data frames.
[0016] In some embodiments, before using the data of the hole region of the current data frame as the data of the occluded region of the window slice, the method further includes:
[0017] Obtain the current value of the valid frame count; when the Hamming distance between the two adjacent data frames is less than a first preset value, increment the current value of the valid frame count by one.
[0018] In some embodiments, the two adjacent data frames include the current data frame and the previous data frame, and the method further includes:
[0019] Obtain the average distance value of the current data frame according to the distance information of the current data frame; obtain the average distance value of the previous data frame according to the distance information of the previous data frame; obtain the binary matrix of the current data frame according to the average distance value of the current data frame; obtain the binary matrix of the previous data frame according to the average distance value of the previous data frame; obtain the scene state of the point cloud data according to the binary matrix of the current data frame and the binary matrix of the previous data frame. Measuring the frame - to - frame similarity by the average hashing algorithm takes less time and can meet the requirements of real - time detection in the autonomous driving scenario.
[0020] In some embodiments, the obtaining the scene state of the point cloud data according to the binary matrix of the current data frame and the binary matrix of the previous data frame includes:
[0021] Obtain the Hamming distance between the two adjacent data frames according to the binary matrix of the current data frame and the binary matrix of the previous data frame; when the Hamming distance between the two adjacent data frames is greater than a third preset value, the scene state of the point cloud data is dynamic; or when the Hamming distance between the two adjacent data frames is less than or equal to the third preset value, the scene state of the point cloud data is static. Identifying the scene state can exclude the point cloud data in the static scene, thereby ensuring the effectiveness of subsequent detection results.
[0022] In a second aspect, an embodiment of the present application provides a device for detecting the window slice of a lidar, and the device includes:
[0023] An acquisition module, configured to acquire the point cloud data of the lidar;
[0024] A feature extraction module, configured to obtain a plurality of feature information of two adjacent data frames according to the point cloud data, where the plurality of feature information includes distance information and reflectivity information;
[0025] An occlusion processing module, configured to obtain data of a void area of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames; the occlusion processing module is further configured to obtain data of an occlusion area of the window slice according to the data of the void area of the two adjacent data frames;
[0026] A marking module, configured to mark the data of the occlusion area.
[0027] The implementation manner of the device embodiment of the present invention may refer to the above method embodiment.
[0028] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is used to implement the steps of the method for detecting a window slice of a lidar in the above embodiment. Description of the Drawings
[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.
[0030] Figure 1 is a flowchart of a method for detecting a window slice of a lidar provided by an embodiment of the present application;
[0031] Figure 2 is a flowchart of a method for detecting a window slice of a lidar provided by an embodiment of the present application;
[0032] Figure 3 is a block diagram of a method for detecting a window slice of a lidar provided by an embodiment of the present application;
[0033] Figure 4 is a schematic diagram of functional modules of a device for detecting a window slice of a lidar provided by an embodiment of the present application;
[0034] Figure 5 is a schematic diagram of the structure of a computer-readable storage medium provided by an embodiment of the present application.
[0035] Description of the Reference Numerals
[0036] 210. Acquisition module; 220. Feature extraction module; 230. Occlusion processing module; Marking module 240; 310. Memory; 311. Program for detecting the window slice of the lidar; 312. Operating system; 320. Processor; 330. Communication bus; 340. User interface; 350. Network interface. Detailed implementation manners
[0037] To make the objectives, technical solutions and advantages of the present application clearer, the following will further describe in detail the implementation manners of the embodiments of the present application in conjunction with the accompanying drawings.
[0038] When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.
[0039] Figure 1 The flowchart of a method for detecting the window slice of a lidar according to an embodiment of the present application is shown. The specific implementation steps of this method are as follows:
[0040] Step S10: Obtain the point cloud data of the lidar. The window slice dirt detection algorithm is implemented based on the point cloud data generated during the operation of the lidar. Therefore, at least two frames of point cloud data need to be continuously cached. It should be understood that the point cloud data can be collected when the lidar performs detection. The types of the lidar include, but are not limited to, mechanical lidar, MEMS (Micro-Electro-Mechanical Systems) lidar, hybrid scanning lidar, solid-state lidar, etc. The method for obtaining the point cloud data and the type of lidar for obtaining the point cloud data in the window slice detection method disclosed in this embodiment are not limited.
[0041] Step S20: Obtain multiple feature information of two adjacent data frames based on the point cloud data. Among them, the multiple feature information includes distance information and reflectivity information. The feature information of each data frame in the point cloud data includes at least one or more of data such as coordinate information, distance information, reflectivity information, and velocity information. The distance information and reflectivity information can be two-dimensional data. By processing the distance information and reflectivity information of the point cloud data, the distance value and reflectivity value of two adjacent data frames can be obtained. In some embodiments, two adjacent data frames include the current data frame and the previous data frame. For the point cloud data of two adjacent data frames, traverse all points of the point cloud data, and use the data point with row number m and column number n as the target processing point to obtain the distance value Pre_dist(m,n) and reflectivity value Pre_ref(m,n) of the previous data frame; the distance value Cur_dist(m,n) and reflectivity value Cur_ref(m,n) of the current data frame.
[0042] In some embodiments, the inter-frame similarity of two adjacent data frames can be judged based on the average hashing algorithm, so as to judge the scene state of the point cloud data. The specific steps are as follows:
[0043] Calculate the average distance values of two adjacent data frames respectively. According to the distance information of the previous data frame, obtain the average distance value Pre_dist_ave of the previous data frame; according to the distance information of the current data frame, obtain the average distance value Cur_dist_ave of the current data frame.
[0044] In an example,
[0045]
[0046]
[0047] Among them, m is the number of rows of the point cloud data, n is the number of columns of the point cloud data, (i, j) is the data point with row number i and column number j, and both m and n are positive integers.
[0048] According to the average distance value of the previous data frame, the binary matrix Pre_mat of the previous data frame is obtained. In some embodiments, when the distance value of the point with row number i and column number j is greater than or equal to the average distance value of the previous data frame, that is, Pre_dist(i,j)≥Pre_dist_ave, Pre_mat(i,j) = 1. When the distance value of the point with row number i and column number j is less than the average distance value of the previous data frame, that is, Pre_dist(i,j)<Pre_dist_ave, Pre_mat(i,j) = 0. According to the average distance value of the current data frame, the binary matrix Cur_mat of the current data frame is obtained. When the distance value of the point with row number i and column number j is greater than or equal to the average distance value of the current data frame, that is, Cur_dist(i,j)≥Cur_dist_ave, Cur_mat(i,j) = 1. When the distance value of the point with row number i and column number j is less than the average distance value of the current data frame, that is, Cur_dist(i,j)<Cur_dist_ave, Cur_mat(i,j) = 0.
[0049] In one embodiment, the Hamming distance between two adjacent data frames can be obtained according to the binary matrix of the previous data frame and the binary matrix of the current data frame. In some embodiments, when Pre_mat(m,n) is equal to Cur_mat(m,n), Hamming_Distance_mat(m,n) = 0; when Pre_mat(m,n) is not equal to Cur_mat(m,n), Hamming_Distance_mat(m,n) = 1; where Hamming_Distance_mat is the Hamming distance matrix of two adjacent data frames. Summing the Hamming distance matrix Hamming_Distance_mat of two adjacent data frames can obtain the Hamming distance Hamming_Distance between two adjacent data frames.
[0050] In one embodiment, the Hamming distance between two adjacent data frames, Hamming_Distance, is compared with a third preset value TH. When the Hamming distance between two adjacent data frames is greater than the third preset value, the scene state of the point cloud data is dynamic; when the Hamming distance between two adjacent data frames is less than or equal to the third preset value, the scene state of the point cloud data is static. Among them, the third preset value can be an integer. For example, the third preset value can be 1, 50, 100, etc.; the setting of the third preset value is related to the number of data points in the point cloud data frame. In one embodiment, the third preset value TH = 121, the number of rows m of the point cloud data = 63, the number of columns n of the point cloud data = 192, and TH = [0.01mn]. If it is determined that the current point cloud scene is a static scene, the two adjacent data frames selected this time are invalid, and the calculation of the current data frame is ended. Obtain the next data frame and the current data frame of the point cloud data as the next set of adjacent data frames and start the operation again from step S20. If the current point cloud scene is determined to be a dynamic scene, the two adjacent data frames selected this time are valid, and step S30 is performed.
[0051] In one embodiment, the average distance value between the first frame and the second frame in the point cloud data frame sequence is calculated. The binary matrix of the first data frame is obtained according to the average distance value of the first data frame, and the binary matrix of the second data frame is obtained according to the average distance value of the second data frame. According to the binary matrix of the first data frame and the binary matrix of the second data frame, the Hamming distance between the first data frame and the second data frame is obtained. Compare this distance with the third preset value. If the Hamming distance between the first data frame and the second data frame is less than the third preset value, it indicates that the scene state of the current point cloud data is static, and the current round of calculation is ended. Next, multiple feature information of the second data frame and the third data frame is extracted. Based on the distance information of the second data frame and the distance information of the third data frame, the Hamming distance between the second data frame and the third data frame is obtained by using the above steps. If the Hamming distance between the second data frame and the third data frame is greater than the third preset value, it is determined that the current point cloud scene is a dynamic scene, and step S30 is continued to extract the void regions of the second data frame and the third data frame. The Hamming distance between two adjacent data frames can reflect the similarity between two adjacent data frames. The greater the Hamming distance between two adjacent data frames, the higher the similarity of the two data frames. Based on this method, the scene state of the point cloud data can be judged.
[0052] Step S30, according to the distance information of two adjacent data frames and the reflectivity information of two adjacent data frames, obtain the data of the void regions of two adjacent data frames. In some embodiments, as Figure 2 shown, step S30 includes:
[0053] Step S301: Obtain the binarized matrix Pre_frame_hole of the set region of the previous data frame based on the distance information and reflectivity information of the previous data frame. The set region consists of data points where both the distance value and the reflectivity value are 0. When Pre_dist(m,n) = 0 and Pre_ref(m,n) = 0, Pre_frame_hole(m,n) = 1; when Pre_dist(m,n) is not equal to 0 or Pre_ref(m,n) is not equal to 0, Pre_frame_hole(m,n) = 0.
[0054] In an example,
[0055]
[0056] Step S302: Perform opening operation and closing operation on the binary matrix Pre_frame_hole of the set region of the previous data frame to obtain the data of the hole region of the previous data frame. Since the matrix Pre_frame_hole of the set region of the extracted previous data frame is a binary two-dimensional matrix, an opening operation is first performed on it, and then a closing operation is performed. The opening operation is a filter based on geometric operations. First, the highlighted part of the matrix (the part where Pre_frame_hole(m,n)=1) is eroded by the filtering structural element, and the highlighted part area is reduced; then the highlighted part of the matrix is dilated by the filtering structural element, and the highlighted part area is expanded. Dilation means scanning each element in the image with a structural element, and performing an AND operation between each pixel in the structural element and the pixels it covers. If the operation results are all 0, the pixel value is 0; if there is a result that is not 0, the pixel value is 1. Dilation is an operation to find the local maximum value, which can merge all background points in contact with the object into the object, making the target larger to fill the holes in the target. Erosion means scanning each element in the image with a structural element, and performing an AND operation between each pixel in the structural element and the pixels it covers. If the operation results are all 1, the pixel value is 1; if there is a result that is not 1, the pixel value is 0. Erosion is an operation to find the local minimum value, which can eliminate the boundary points of the object, making the target smaller, thereby eliminating the noise points smaller than the structural element. Erosion and dilation are mainly used to extract the image components that are meaningful for expressing and depicting the region shape from the image, such as boundaries and connected regions, etc., to facilitate subsequent image recognition and other work. The opening operation can filter the set region of the point cloud, remove the isolated noise points, burrs, etc. in the matrix, and prevent the noise points from interfering with the set region. The closing operation filters by filling the concave corners of the set region. If there are discontinuous parts in the set region, this method can be used to repair them. The closing operation first dilates and then erodes the highlighted part of the matrix (the part where Pre_frame_hole(m,n)=1), and can fuse the slightly connected tiles. After the above operations, the position and shape of the hole region of the previous data frame can be obtained.
[0057] Step S303: According to the distance information of the current data frame and the reflectivity information of the current data frame, the binary matrix Cur_frame_hole of the set region of the current data frame can be obtained. In one embodiment, when Cur_dist(m,n)=0 and Cur_ref(m,n)=0, Cur_frame_hole(m,n)=1; when Cur_dist(m,n) is not equal to 0 or Cur_ref(m,n) is not equal to 0, Cur_frame_hole(m,n)=0.
[0058] In an example,
[0059]
[0060] Step S304: Perform opening operation and closing operation on the binary matrix Cur_frame_hole of the set region of the current data frame, and then the data of the hole region of the current data frame can be obtained. When acquiring point cloud data, noise and outliers may occur in the acquired point cloud due to various reasons. Therefore, it is necessary to preprocess the point cloud through filtering and resampling to facilitate subsequent operations such as feature extraction.
[0061] Step S40: Obtain the data of the occlusion region of the window slice according to the data of the hole regions of two adjacent data frames. In some embodiments, according to the data of the hole regions of two adjacent data frames, a Hamming distance matrix hole_Hamming_Distance_mat of the hole regions of the two adjacent data frames is set. In one embodiment, when Pre_hole(m,n) is equal to Cur_hole(m,n), Hamming_Distance_mat(m,n) = 0; when Pre_hole(m,n) is not equal to Cur_hole(m,n), Hamming_Distance_mat(m,n) = 1, where Pre_hole is the binary matrix of the hole region of the previous data frame, and Cur_hole is the binary matrix of the hole region of the current data frame. Sum the Hamming distance matrix hole_Hamming_Distance_mat of the hole regions of two adjacent data frames to obtain the Hamming distance hole_Hamming_Distance of the hole regions of two adjacent data frames.
[0062] Compare the Hamming distance of the hole regions of two adjacent data frames with a first preset value similar_th. Here, the setting of the first preset value is related to the number of data points in the point cloud data frame. During the operation of the algorithm, the first preset value can be adjusted according to the actual situation. The first preset value and the third preset value can be equal or not equal, and the first preset value can be 0, 1, 50, 112, 150, etc. In one embodiment, the first preset value similar_th = 121, the number of rows m of the point cloud data = 63, the number of columns n of the point cloud data = 192, and TH = [0.01mn]. When the Hamming distance of the hole regions of two adjacent data frames is greater than or equal to the first preset value, that is, hole_Hamming_Distance ≥ similar_th, the hole region of the current data frame is inconsistent with the hole region of the previous data frame. The hole region of the current data frame may be a point cloud hole caused by a specific scenario, and the data in the hole region of the current data frame is invalid. When the Hamming distance of the hole regions of two adjacent data frames is less than the first preset value similar_th, that is, hole_Hamming_Distance < similar_th, the hole region of the current data frame is consistent with the hole region of the previous data frame. The data in the hole region of the current data frame is valid, and the current data frame is a valid frame. In some embodiments, when hole_Hamming_Distance < similar_th, obtain the current value of the valid frame count, and then add 1 to the current value of the valid frame count. Here, the valid frame count represents the number of data frames determined to be valid frames, and the starting value of the valid frame count is 0. In some embodiments, when hole_Hamming_Distance ≥ similar_th, end the current round of calculation, clear the valid frame count, and start the operation again from step S20 for the next frame. In one embodiment, the current value of the valid frame count is 4, and the Hamming distance of the hole regions of the 5th data frame and the 6th data frame is greater than the first preset value. Clear the valid frame count, obtain multiple feature information of the 7th data frame and the 6th data frame, and obtain the data in the hole regions of the 6th data frame and the 7th data frame through the operation of step S30. The Hamming distance of the hole regions of the 6th data frame and the 7th data frame is less than the first preset value, the 7th data frame is a valid frame, the current value of the valid frame count is 0, add 1 to the valid frame count, and the valid frame count for this round is 1.
[0063] Determine whether the valid frame count value is greater than or equal to a second preset value, where the second preset value is a positive integer, such as 5, 6, 8, 10, 20, 50, etc. When the valid frame count value is greater than or equal to the second preset value, the data in the hole area of the current data frame is the data in the occluded area of the window slice. In one embodiment, when the valid frame count value is less than the second preset value, steps S20 - S30 are repeated. In one embodiment, the lidar point cloud data contains 100 data frames, the 20th and 21st data frames of the point cloud data frame sequence are obtained for detection, the current valid frame count value is 19, and the second preset value is 20. If the Hamming distance between the hole areas of the 20th and 21st data frames is less than the first preset value, then the 21st data frame is a valid frame, and the current value of the valid frame count value is incremented by 1 to obtain the current valid count value for this round as 20. At this time, the valid frame count value is equal to the second preset value, so the data in the hole area of the current data frame is the data in the occluded area of the window slice.
[0064] In one embodiment, the 1st and 2nd data frames of the point cloud data are obtained, the starting value of the valid frame count value is 0, and the second preset value is 20. If the Hamming distance between the 1st and 2nd data frames is less than the first preset value, the 2nd data frame is a valid frame, and the valid frame count value is incremented by 1 to obtain the current valid frame count value for this round as 1. At this time, the valid frame count value is less than the second preset value, and step S20 is returned to perform operations on the 3rd and 2nd data frames. If the Hamming distance between the 2nd and 3rd data frames is less than the first preset value, then the 3rd data frame is a valid frame, the current value of the valid frame count value is 1, and after incrementing the valid frame count value by 1, the current valid frame count value for this round is 2. At this time, the valid frame count value is less than the second preset value, and the operations on the 3rd and 4th adjacent frames continue.
[0065] In the embodiments of the present application, the similarity of the hole areas of two adjacent data frames is measured by the average hashing algorithm. The average hashing algorithm has a short single - frame calculation time, which can meet the real - time requirements of autonomous driving. Similarity can also be measured by methods such as cosine similarity, histogram, mutual information, mean square error (MSE) algorithm, SSIM structural similarity, and feature matching. The present application does not limit the similarity measurement method. Considering that the dirt occlusion of the window slice generally exists stably, the method provided in the embodiments of the present application further sets a limit condition for valid frames. When the hole areas of a continuous preset number of data frames are consistent, the hole area is determined to be the occluded area of the window slice, which helps to avoid misjudgment caused by close - range targets.
[0066] Step S50, mark the data of the occluded area. In one embodiment, the current data frame is the 21st frame in the point cloud data frame sequence. After setting the RGB value of the data points in the occluded area to (255, 0, 0), the operation proceeds to the next data frame. Obtain multiple feature information of the 22nd data frame and the 21st data frame. Based on the distance information and reflectivity information of the 22nd data frame and the 21st data frame, obtain the data of the void area of these two data frames. When the Hamming distance between the 22nd data frame and the 21st data frame is less than the first preset value, the valid frame count value is incremented by 1 based on the current value, where the current value of the valid frame count value is 20. The second preset value is 20. When the valid frame count value is greater than the second preset value, the data of the void area of the 22nd data frame is used as the data of the occluded area of the window slice, and the RGB value of the data points in the occluded area is set to (255, 0, 0). After obtaining the data of the occluded area of the window slice, the detection method provided by the embodiment of the present application continues to detect the point cloud data, realizing real-time update and continuous warning of the dirty occluded area.
[0067] As Figure 4 shown, the embodiment of the present application provides a device for detecting the window slice of a lidar, including an acquisition module 210, a feature extraction module 220, an occlusion processing module 230, and a marking module 240. The acquisition module 210 is configured to obtain the point cloud data of the lidar and upload it to the extraction module for processing. The feature extraction module 220 is configured to obtain multiple feature information of two adjacent data frames based on the point cloud data, where the multiple feature information includes distance information and reflectivity information. The occlusion processing module 230 is not only configured to obtain the data of the void area of two adjacent data frames based on the distance information of two adjacent data frames and the reflectivity information of two adjacent data frames, but also configured to obtain the data of the occluded area of the window slice based on the data of the void area of two adjacent data frames. The marking module 240 is configured to mark the data of the occluded area and display the dirty occluded area. The display method of the dirty occluded area can be to display the point cloud of the dirty occluded area in red or to make the point cloud at the edge of the dirty occluded area flash to show the position and shape of the dirty occluded area. In some embodiments, the marking module 240 is further configured to execute a warning instruction, and the warning instruction will output the data of the occluded area of the window slice through a text pop-up window to play a reminder and warning role.
[0068] As Figure 5As shown in the figure, an embodiment of the present application provides a computer-readable storage medium, including a memory 310, a processor 320, a communication bus 330, a user interface 340, a network interface 350, and a program 311 for detecting a window piece of a lidar stored on the memory 310 and executable on the processor 320. The program 311 for detecting the window piece of the lidar is used to implement the steps of the method for detecting the window piece of the lidar in the above embodiment. A computer device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions. In some embodiments, the computer device may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a monitoring device, a video conferencing system, a rack server, a blade server, a tower server, or a cabinet server (including an independent server or a server cluster composed of multiple servers), etc. The computer device includes, but is not limited to, a memory 310, a processor 320, and a network interface 350 that can communicate with each other, and a lidar window piece dirt detection program 311 stored on the memory 310 and executable on the processor 320.
[0069] In some embodiments, the memory 310 includes at least one type of computer-readable storage medium. The readable storage medium includes flash memory, a hard disk, a multimedia card, a card-type memory (such as an SD or DX memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, etc. In some embodiments, the memory 310 may be an internal storage module of the computer device, such as the hard disk or memory of the computer device. In some embodiments, the memory 310 may also be an external storage device of the computer device, such as a plug-in hard disk equipped on the computer device, a Smart Media Card (SMC for short), a Secure Digital (SD for short) card, a Flash Card, etc. In some embodiments, the memory 310 may also include both the internal storage module and the external storage device of the computer device. In one embodiment, the memory 310 is used to store the operating system 312 and various application software installed on the computer device, including the lidar window piece dirt detection program 311. In addition, the memory 310 may also be used to temporarily store various data that have been output or will be output.
[0070] In some embodiments, the processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips. The processor 320 is generally used to control the overall operation of the computer device, such as performing control and processing related to data interaction or communication with the computer device. In one embodiment, the processor 320 is used to run the program code stored in the memory 310 or process data.
[0071] In some embodiments, the network interface 350 includes a wireless network interface or a wired network interface. The network interface 350 is used to establish a communication connection between the computer device and other computer devices. In one embodiment, the network interface 350 is used to connect the computer device to an external terminal through a network and establish a data transmission channel and a communication connection between the computer device and the external terminal. In some embodiments, the network may be an intranet, the Internet, a Global System of Mobile communication (GSM), a Wideband Code Division Multiple Access (WCDMA), a 4G network, a 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks.
[0072] In one embodiment, the lidar window sheet contamination detection program 311 stored in the memory 310 may be divided into one or more program modules and executed by one or more processors 320 to complete the steps shown in the lidar window sheet contamination detection method of the embodiments of the present application.
[0073] Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the embodiments can be implemented by a general computing device. They can be concentrated on a single computing device or distributed on a network composed of multiple computing devices. In some embodiments, they can be implemented by program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some embodiments, the above-mentioned modules or steps of the embodiments can be separately fabricated into individual integrated circuit modules, or multiple modules or steps among them can be fabricated into a single integrated circuit module for implementation. The embodiments of the present application are not limited to any specific combination of hardware and software.
[0074] In the description of the present application, it should be understood that the terms "first", "second", etc. are only used for descriptive purposes and cannot be construed as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific circumstances. In addition, in the description of the present application, unless otherwise specified, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.
[0075] It should be noted that the workflow described above is only illustrative and does not limit the protection scope of the present invention. In actual applications, those skilled in the art can select some or all of them according to actual needs to achieve the purpose of the solution of this embodiment, and no limitation is made here. In addition, in this article, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent in such process, method, article or system. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of another identical element in the process, method, article or system including that element.
[0076] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs.
[0077] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.
Claims
1. A method for detecting the window piece of a lidar, characterized in that, the method includes: acquiring the point cloud data of the lidar; obtaining multiple feature information of two adjacent data frames according to the point cloud data, wherein the multiple feature information includes distance information and reflectivity information; obtaining the data of the void area of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames; obtaining the data of the occlusion area of the window piece according to the data of the void area of the two adjacent data frames; marking the data of the occlusion area.
2. The method for detecting the window piece of a lidar according to claim 1, wherein the two adjacent data frames include a current data frame and a previous data frame, characterized in that, the obtaining the data of the void area of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames includes: obtaining a binary matrix of the set area of the current data frame according to the distance information of the current data frame and the reflectivity information of the current data frame, wherein the set area is composed of data points with both distance values and reflectivity values being zero; performing opening operation and closing operation on the binary matrix of the set area of the current data frame to obtain the data of the void area of the current data frame; obtaining a binary matrix of the set area of the previous data frame according to the distance information of the previous data frame and the reflectivity information of the previous data frame; performing opening operation and closing operation on the binary matrix of the set area of the previous data frame to obtain the data of the void area of the previous data frame.
3. The method for detecting the window piece of a lidar according to claim 1, wherein the two adjacent data frames include a current data frame and a previous data frame, characterized in that, the obtaining the data of the occlusion area of the window piece according to the data of the void area of the two adjacent data frames includes: taking the data of the void area of the current data frame as the data of the occlusion area of the window piece, wherein the Hamming distance between the two adjacent data frames is less than a first preset value, and the valid frame count value is greater than or equal to a second preset value.
4. The method for detecting the window piece of a lidar according to claim 3, characterized in that, the obtaining the data of the occlusion area of the window piece according to the data of the void area of the two adjacent data frames further includes: obtaining a Hamming distance matrix of the void areas of the two adjacent data frames according to the data of the void area of the previous data frame and the data of the void area of the current data frame; summing according to the Hamming distance matrix to obtain the Hamming distance between the two adjacent data frames.
5. The method for detecting the window piece of a lidar according to claim 3, characterized in that, before taking the data of the void area of the current data frame as the data of the occlusion area of the window piece, the method further includes: obtaining the current value of the valid frame count value; When the Hamming distance between the two adjacent data frames is less than a first preset value, increment the current value of the valid frame count.
6. The method for detecting a window sheet of a lidar according to claim 1, wherein the two adjacent data frames include a current data frame and a previous data frame. Characterized in that: The method further includes: Obtaining an average distance value of the current data frame according to the distance information of the current data frame; Obtaining an average distance value of the previous data frame according to the distance information of the previous data frame; Obtaining a binarization matrix of the current data frame according to the average distance value of the current data frame; Obtaining a binarization matrix of the previous data frame according to the average distance value of the previous data frame; Obtaining a scene state of the point cloud data according to the binarization matrix of the current data frame and the binarization matrix of the previous data frame.
7. The method for detecting a window sheet of a lidar according to claim 6, Characterized in that: The obtaining the scene state of the point cloud data according to the binarization matrix of the current data frame and the binarization matrix of the previous data frame includes: Obtaining the Hamming distance between the two adjacent data frames according to the binarization matrix of the current data frame and the binarization matrix of the previous data frame; When the Hamming distance between the two adjacent data frames is greater than a third preset value, the scene state of the point cloud data is dynamic, or When the Hamming distance between the two adjacent data frames is less than or equal to the third preset value, the scene state of the point cloud data is static.
8. A device for detecting a window sheet of a lidar, Characterized in that: The device includes: An acquisition module for acquiring point cloud data of the lidar; A feature extraction module for obtaining a plurality of feature information of two adjacent data frames according to the point cloud data, wherein the plurality of feature information includes distance information and reflectivity information; An occlusion processing module for obtaining data of a void area of the two adjacent data frames according to the distance information of the two adjacent data frames and the reflectivity information of the two adjacent data frames; the occlusion processing module is further configured to obtain data of an occlusion area of the window sheet according to the data of the void area of the two adjacent data frames; A marking module for marking the data of the occlusion area.
9. A computer-readable storage medium, Characterized in that: A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, it is used to implement the steps of the method for detecting a window sheet of a lidar according to any one of claims 1-7.