Laser radar detection method and device, electronic equipment and storage medium

Abstract

The invention discloses a laser radar detection method and device, electronic equipment and a storage medium, and relates to the field of automatic driving and radar detection. The method comprises the following steps: acquiring a frame of point cloud data acquired by a laser radar; projecting the point cloud data into the two-dimensional image based on the attribute of the laser radar to obtain a depth occupation projection drawing and an intensity projection drawing, the depth occupation projection drawing being used for representing depth values of different data points in the point cloud data, and the intensity projection drawing being used for representing intensity values of different data points in the point cloud data; mixing the depth occupation projection drawing and the intensity projection drawing to obtain a mixed projection drawing; and determining a detection result of the laser radar based on the pixel value of the mixed projection drawing, the detection result being used for representing whether the laser radar is shielded by an object. According to the invention, the technical problems of low detection efficiency and high cost of the laser radar in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to the fields of autonomous driving and radar monitoring, and in particular to a detection method, device, electronic equipment and storage medium of a laser radar. Background Art

[0002] At present, LiDAR is widely used in scenarios such as unmanned driving in mining areas, robot navigation, drone-assisted flight, and virtual reality. In mining operations, due to the large amount of sand and dust on site, if the LiDAR is not cleaned for a long time, a large amount of dust will accumulate, which will cause the LiDAR to be unable to transmit and receive laser beams normally, affecting the normal operation of the LiDAR. Therefore, it is necessary to detect whether the LiDAR is blocked.

[0003] However, in the related art, laser radar monitoring is usually carried out by regular inspection to avoid the problem of laser radar being blocked by mud and dirt, but this method is inefficient and is greatly affected by human factors. Alternatively, the laser radar echo data is analyzed by an occlusion detection algorithm to determine whether the laser radar is blocked, which requires the use of echo data, but this method increases the complexity of detection and computational overhead, resulting in low detection efficiency and high cost of laser radar.

[0004] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0005] The embodiments of the present invention provide a laser radar detection method, device, electronic device and storage medium to at least solve the technical problems of low laser radar detection efficiency and high cost in related technologies.

[0006] According to one aspect of an embodiment of the present invention, a laser radar detection method is provided, including: acquiring a frame of point cloud data collected by the laser radar; based on the properties of the laser radar, projecting the point cloud data into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map, wherein the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data; mixing the depth occupancy projection map and the intensity projection map to obtain a mixed projection map; based on the pixel value of the mixed projection map, determining the detection result of the laser radar, wherein the detection result is used to characterize whether the laser radar is blocked by an object.

[0007] According to one aspect of an embodiment of the present invention, a laser radar detection device is provided, including: an acquisition module, used to acquire a frame of point cloud data collected by the laser radar; a projection module, used to project the point cloud data into a two-dimensional image based on the properties of the laser radar, and obtain a depth occupancy projection map and an intensity projection map, wherein the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data; a mixing module, used to mix the depth occupancy projection map and the intensity projection map to obtain a mixed projection map; a determination module, used to determine the detection result of the laser radar based on the pixel value of the mixed projection map, wherein the detection result is used to characterize whether the laser radar is blocked by an object.

[0008] According to another aspect of an embodiment of the present invention, there is further provided an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the method in each embodiment of the present invention is executed when the program is running.

[0009] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0010] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0011] According to another aspect of an embodiment of the present invention, a computer program product is provided, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0012] According to another aspect of the embodiments of the present invention, a computer program is further provided. When the computer program is executed by a processor, the methods in the embodiments of the present invention are implemented.

[0013] In an embodiment of the present invention, a frame of point cloud data collected by a laser radar is first obtained. Based on the properties of the laser radar, the point cloud data is projected into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map. Among them, the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data. The depth occupancy projection map and the intensity projection map are mixed to obtain a mixed projection map. Based on the pixel values ​​of the mixed projection map, the detection result of the laser radar is determined. It is easy to notice that the method of using a hybrid projection map to detect whether the laser radar is blocked by an object is to project the point cloud data generated by the laser radar into a two-dimensional image through the laser radar's own properties, thereby reducing the data dimension of the detection, and obtaining a depth occupancy projection map and an intensity projection map, so that the data depth value and intensity value of the point cloud data are used to characterize the data quality generated by the laser radar, and then the hybrid projection map of the depth occupancy projection map and the intensity projection map is used to determine whether the laser radar is blocked. That is, it is possible to determine whether the laser radar is blocked by judging the pixel quality corresponding to the data generated by the laser radar, thereby achieving the purpose of simply and efficiently detecting whether the laser radar is blocked by an object, thereby realizing the technical effect of reducing the detection cost and improving the detection efficiency of the laser radar, and then solving the technical problems of low laser radar detection efficiency and high cost in related technologies. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0015] Figure 1 is a flow chart of a laser radar detection method according to an embodiment of the present invention;

[0016] Figure 2 is a schematic diagram of projecting point cloud data onto a two-dimensional image according to an embodiment of the present application;

[0017] Figure 3 is a schematic diagram of a two-dimensional image according to an embodiment of the present invention;

[0018] Figure 4 is a schematic diagram of a sliding window according to an embodiment of the present invention;

[0019] Figure 5 It is a schematic diagram of a laser radar detection device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0021] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0022] According to an embodiment of the present invention, a detection method embodiment of a laser radar is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0023] Figure 1 is a flow chart of a laser radar detection method according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0024] Step S102, obtaining a frame of point cloud data collected by the laser radar.

[0025] The above-mentioned laser radar can be mounted on a mining drone to provide environmental perception, positioning and obstacle information, and is a core component of the drone. The laser radar can detect the surrounding environment and collect point cloud data. The laser radar can be a mechanical laser radar, which determines the distance of the target object by the difference in reflection time, or a solid-state laser radar, which determines the distance of the target object by the number of refractions. However, the detection capability of the laser radar is affected by the surrounding environment, especially in mining operation scenarios. The operating environment is sandy and dusty, and the laser radar is easily blocked by dust and sludge. Therefore, it is necessary to detect the laser radar simply and quickly.

[0026] The above-mentioned point cloud data can be data of the surrounding environment collected by the laser radar. Point cloud data is a representation of three-dimensional spatial data and can be composed of multiple points. Point cloud data can include but is not limited to coordinate data, color data, intensity data, etc., which are not limited here and can be determined as needed. Obtaining point cloud data provides a reliable data source for subsequent detection. The amount of point cloud data is large. By obtaining a frame of point cloud data, it is convenient to process each frame of point cloud data separately to improve the accuracy of detection.

[0027] In an optional embodiment, a frame of point cloud data can be obtained in real time according to the point cloud data collected by the laser radar when it is working. Optionally, the point cloud data is subscribed through the robot operating system, or the point cloud data is read using a software development kit. A frame of point cloud data can also be intercepted from the point cloud data stored in the cloud platform or the local storage device.

[0028] Step S104, based on the properties of the laser radar, project the point cloud data into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map.

[0029] Among them, the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data.

[0030] The above-mentioned properties of the LiDAR can be used to characterize the operating parameters of the LiDAR. Through different operating parameters, the range and direction of LiDAR detection can be adjusted, thereby affecting the LiDAR detection results. The properties of the LiDAR may include but are not limited to the pitch angle, horizontal angle, number of horizontal points and lines, etc., which are not limited here and can be determined as needed. According to the properties of the LiDAR, the point cloud data can be accurately projected into a two-dimensional image, so as to reflect the current sensing situation of the LiDAR, which is convenient for judging whether the LiDAR is blocked by the sensing situation.

[0031] The above-mentioned depth occupancy projection map can be used to represent the depth values ​​of different data points in the point cloud data. The depth value is used to represent whether the laser radar scans the object. If the laser radar does not scan the object, it may be because there are no obstacles around the current laser radar, or the laser radar is blocked and cannot effectively perceive the surrounding environment. The depth occupancy projection map can clearly show the situation of the laser radar scanning the surrounding objects.

[0032] The above intensity projection diagram can be used to characterize the intensity values ​​of different data points in the point cloud data. The intensity value can be the image pixel value after point projection. The intensity value can be used to characterize the signal intensity reflected by the laser beam of the LiDAR. The intensity signal can reflect the reflection characteristics of the target object, which are affected by the material, smoothness or environment of the object. The higher the pixel value, the higher the reflection intensity and the stronger the perception ability of the LiDAR. Conversely, the weaker the perception ability of the LiDAR. The intensity projection diagram can reflect the reflection intensity of the LiDAR, which helps to determine whether the LiDAR is blocked.

[0033] The above-mentioned projection can be a process of converting three-dimensional point cloud data into a two-dimensional image. The depth value and intensity value of each data point in the point cloud data are projected into a two-dimensional image, thereby obtaining a depth occupancy projection map and an intensity projection map, and then judging whether the laser radar is blocked by an object based on the depth occupancy projection map and the intensity projection map. Through projection, the data dimension of the detection can be reduced, thereby improving the efficiency of the detection.

[0034] In an optional embodiment, the size of the two-dimensional image can be first determined based on the properties of the laser radar, and the depth values ​​and intensity values ​​of different data points in the point cloud data can be further projected into the two-dimensional image to obtain a depth occupancy projection map and an intensity projection map. Alternatively, the three-dimensional coordinates of each data point in the point cloud data can be converted into two-dimensional coordinates to form a two-dimensional image, and further, based on the properties of the laser radar, the two-dimensional image can be annotated by projecting the depth values ​​and intensity values ​​of each data point to obtain a depth occupancy projection map and an intensity projection map.

[0035] Step S106, mixing the depth occupancy projection map and the intensity projection map to obtain a mixed projection map.

[0036] The above-mentioned mixing can be a process of weighting the depth occupancy projection map and the intensity projection map. Based on different influencing parameters, the same or different weights can be set for the depth occupancy projection map and the intensity projection map, so as to emphasize the space occupancy or pay more attention to the reflection signal. Through mixing, the recognition accuracy of occlusion can be enhanced while retaining the depth value and the intensity value.

[0037] The above-mentioned hybrid projection image can be a two-dimensional image obtained by mixing the depth occupancy projection image and the intensity projection image, that is, the hybrid projection image is a two-dimensional image combining the depth value and the intensity value of the data point. The hybrid projection image can be used to detect whether the laser radar is blocked.

[0038] In an optional embodiment, a first weight factor and a second weight factor may be set for the depth occupancy projection map and the intensity projection map, respectively, and the sum of the first weight factor and the second weight factor is 1. Further, the products of the depth occupancy projection map and the intensity projection map with the first weight factor and the second weight factor are summed to obtain a mixed projection map. The weight rule may be set in advance, and the weight factor may be adjusted in real time according to the weight rule, thereby improving the detection accuracy.

[0039] Step S108, determining the detection result of the laser radar based on the pixel value of the mixed projection image.

[0040] Among them, the detection result is used to characterize whether the laser radar is blocked by an object.

[0041] The above pixel values ​​can be pixel values ​​of a two-dimensional hybrid projection image. By using two-dimensional data dimensions to determine the occlusion of the laser radar, that is, using the pixel values ​​of the hybrid projection image, the data dimension of the detection is reduced, which is convenient for improving the detection efficiency.

[0042] The above detection results can be used to characterize whether the laser radar is blocked by an object. The detection result can be determined by the pixel value of the mixed projection image. The detection result can intuitively display the blockage of the laser radar, which is convenient for the staff to repair and handle in time to ensure the normal operation of the laser radar.

[0043] In an optional embodiment, the pixel value can be input into a pre-trained detection model, and the output value of the model can be used to characterize the detection result. The detection model can be obtained by training a classification model based on historical data of image pixel values ​​and occlusion conditions. The detection model can be one of the following: a logistic regression model, a support vector machine, a decision tree, a random forest, etc., which are not limited here and can be determined as needed. Through the detection model, the detection result of the laser radar can be obtained according to the pixel value of the mixed projection image.

[0044] In another optional embodiment, a sliding window may be used to slide on the mixed projection image multiple times according to preset sliding parameters to obtain multiple target sliding windows. Then, the pixel sum of each target sliding window is determined. Further, the pixel sum of each target sliding window is determined respectively, and in response to the pixel sum of the target sliding window being greater than the preset pixel sum, the detection result is determined based on the number of target sliding windows in the target set.

[0045] In an embodiment of the present invention, a frame of point cloud data collected by a laser radar is first obtained. Based on the properties of the laser radar, the point cloud data is projected into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map. Among them, the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data. The depth occupancy projection map and the intensity projection map are mixed to obtain a mixed projection map. Based on the pixel value of the mixed projection map, the detection result of the laser radar is determined. It is easy to notice that the mixed projection map is used to detect whether the laser radar is blocked by an object. Through the properties of the laser radar itself, the point cloud data generated by the laser radar is projected into a two-dimensional image, which reduces the data dimension of the detection, and obtains the depth occupancy projection map and the intensity projection map, so that the data depth value and intensity value of the point cloud data can be used to characterize the data quality generated by the laser radar, and then the mixed projection map of the depth occupancy projection map and the intensity projection map can be used to judge whether the laser radar is blocked. That is, it is possible to judge whether the laser radar is blocked by judging the pixel quality corresponding to the data generated by the laser radar, thereby achieving the purpose of simply and efficiently detecting whether the laser radar is blocked by objects, thereby realizing the technical effect of reducing the detection cost and improving the detection efficiency of the laser radar, and then solving the technical problems of low laser radar detection efficiency and high cost in related technologies.

[0046] Figure 2 is a schematic diagram of projecting point cloud data onto a two-dimensional image according to an embodiment of the present application, such as Figure 2 As shown in the figure, a data point in the point cloud data is used as an example to illustrate the projection. The left side is a data point r in the point cloud data. p The three-dimensional coordinates (x, y, z) are shown in Figure 1, and the right side is the projected two-dimensional image, where the width of the two-dimensional image corresponds to the number of horizontal points of the laser radar, and the height of the two-dimensional image corresponds to the number of lines of the laser radar.

[0047] Point r p Take the projection process as an example: point r p The three-dimensional coordinates are (x p ,y p , z p ), α p is the horizontal angle, β p is the pitch angle. p When projecting to a two-dimensional image, based on α p Determine the horizontal coordinate of the data point in the two-dimensional image, based on β p Determine the vertical coordinate of the data point in the two-dimensional image, thereby determining the data point r p The coordinates in the two-dimensional image are (α p , β p ).

[0048] Optionally, the attributes of the laser radar include: pitch angle, horizontal angle, number of horizontal points and number of lines. The number of horizontal points is used to represent the number of data points collected by the laser radar in the horizontal direction, and the number of lines is used to represent the number of scanning lines scanned by the laser radar in the vertical direction.

[0049] The pitch angle can be used to characterize the offset angle of the laser beam relative to the vertical axis of the laser radar. Based on the pitch angle, the vertical offset range that the laser radar can scan can be determined.

[0050] The above horizontal angle can be used to characterize the offset angle of the laser beam relative to the horizontal axis of the laser radar. Based on the horizontal angle, the offset range in the horizontal direction that the laser radar can scan can be determined.

[0051] The above-mentioned number of horizontal points can be used to characterize the number of data points collected by the laser radar in the horizontal direction. Based on the number of horizontal points, the range scanned by the laser radar in the horizontal direction can be determined.

[0052] The above-mentioned line number can be used to characterize the number of scanning lines scanned by the laser radar in the vertical direction. Based on the line number, the scanning range of the laser radar in the vertical direction of the scanning can be determined.

[0053] Optionally, based on the properties of the lidar, the point cloud data is projected into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map, including: generating a two-dimensional image based on the number of horizontal points and the number of lines, wherein the width of the two-dimensional image corresponds to the number of horizontal points, and the height of the two-dimensional image corresponds to the number of lines; based on the pitch angle and the horizontal angle, the depth values ​​of different data points are projected into the two-dimensional image to obtain a depth occupancy projection map; based on the pitch angle and the horizontal angle, the intensity values ​​of different data points are projected into the two-dimensional image to obtain an intensity projection map.

[0054] In an optional embodiment, the size of the projected two-dimensional image can be determined first, and the size of the projected two-dimensional image can be determined by the number of horizontal points and lines of the laser radar, that is, the range scanned by the laser radar in the horizontal direction and the vertical direction can be projected onto the two-dimensional image. Specifically, the width of the two-dimensional image is determined based on the number of horizontal points of the laser radar, and the height of the two-dimensional image is determined based on the number of lines of the laser radar. Further, the depth value and intensity value of each data point are projected into the two-dimensional image, so that the two-dimensional image is used to reflect the laser radar scanning situation, and then determine whether the laser radar is blocked.

[0055] Optionally, based on the pitch angle and the horizontal angle, the depth values ​​of different data points are projected into a two-dimensional image to obtain a depth occupancy projection map, including: based on the pitch angle and the horizontal angle, determining the target pixel points corresponding to the different data points; in response to the depth value of the different data points being greater than a preset depth value, determining the pixel value of the target pixel point to be a first preset value; in response to the depth value of the different data points being less than or equal to the preset depth value, determining the pixel value of the target pixel point to be a second preset value, wherein the first preset value is greater than the second preset value.

[0056] The above-mentioned target pixel point can be the pixel point of the data point projected onto the two-dimensional depth occupancy projection map. A two-dimensional image can be formed by the target pixel points. The position of the pixel point corresponding to the data point is determined by the pitch angle and the horizontal angle, and further, the pixel value of the pixel point is determined by the depth value of the data point, thereby obtaining the depth occupancy projection map.

[0057] The above-mentioned preset depth value can be used to characterize that the laser radar has scanned an object. The preset depth value can be determined according to the environment or detection requirements. In response to the depth value of the data point being greater than the preset depth value, indicating that the laser radar has scanned an object, the target pixel point corresponding to the projection of the data point is set to the first preset value. Conversely, in response to the depth value of the data point being less than or equal to the preset depth value, indicating that the laser radar has not scanned an object, the target pixel point corresponding to the projection of the data point is set to the second preset value, thereby distinguishing the depth value of the data point by the pixel value, thereby reflecting whether the laser radar has scanned an object.

[0058] The pixel value of the above-mentioned target pixel point can be used to reflect the depth value of different data points. The pixel value of the depth occupancy projection map may include a first preset value and a second preset value. The first preset value and the second preset value can reflect whether the laser radar scans an object, so as to facilitate the use of the depth occupancy projection map determined by the pixel value to judge the occlusion of the laser radar. The first preset value can be used to reflect that the laser radar scans an object. The second preset value can be used to reflect that the laser radar does not scan an object.

[0059] In an optional embodiment, the position of the pixel point in the two-dimensional image corresponding to each data point of the point cloud data is determined by the pitch angle and the horizontal angle, and further, the pixel value of the corresponding pixel point is determined by the depth value of each data point, so as to obtain a depth occupancy projection map.

[0060] Optionally, based on the pitch angle and the horizontal angle, the intensity values ​​of different data points are projected into a two-dimensional image to obtain an intensity projection diagram, including: determining the target pixel points corresponding to the different data points based on the pitch angle and the horizontal angle; and determining that the pixel value of the target pixel point is the intensity value of the data point.

[0061] When projecting point cloud data, the data points can be projected to the pixels in the two-dimensional image. The position of the pixel corresponding to the data point can be determined by the pitch angle and the horizontal angle. Furthermore, the pixel value of the pixel can be determined by the intensity value of the data point to obtain an intensity projection map. The intensity value is then reflected by the pixel value of the two-dimensional image, that is, the intensity of the reflected signal of the laser radar is reflected by the pixel value of the two-dimensional image. By using the intensity value as the pixel value, the material and surface characteristics of the obstruction can be more clearly identified, which is particularly important for occlusion detection under different lighting conditions and can enhance the adaptability of laser radar detection.

[0062] Optionally, the depth occupancy projection map and the intensity projection map are mixed to obtain a mixed projection map, including: determining a first weight value corresponding to the depth occupancy projection map and a second weight value corresponding to the intensity projection map based on the usage scenario of the laser radar; and performing weighted processing on the depth occupancy projection map and the intensity projection map based on the first weight value and the second weight value to obtain a mixed projection map.

[0063] The above-mentioned usage scenarios may include but are not limited to unmanned driving, environmental monitoring, etc., which are not limited here and can be determined as needed. Different usage scenarios have different requirements for the use of LiDAR. By setting different weight values, the performance of LiDAR occupancy detection can be improved, especially in specific environments, such as rainy and snowy weather or at night. The weight value can be adjusted according to the actual situation to improve the accuracy of detection. Therefore, according to the current usage scenario of the LiDAR, the weight values ​​corresponding to the depth occupancy projection map and the intensity projection map can be determined, so as to perform weighted processing on the depth occupancy projection map and the intensity projection map to obtain a mixed projection map, and then use the mixed projection map to determine whether the LiDAR is blocked, thereby improving the accuracy of detection.

[0064] In an optional embodiment, a weight correspondence table may be pre-set to facilitate determining the weight value according to the current usage scenario of the laser radar, or the weight value may be set manually. Furthermore, the sum of a first product obtained by multiplying the pixel value of each pixel point in the depth occupancy projection map by the first weight value and a second product obtained by multiplying the pixel value of each pixel point in the intensity projection map by the second weight value may be used as the pixel value of each pixel point in the mixed projection map, thereby obtaining the mixed projection map.

[0065] Optionally, the above method also includes: normalizing the intensity projection map to obtain a normalized projection map; performing histogram equalization on the normalized projection map to obtain an equalized projection map; based on the first weight value and the second weight value, weighted processing is performed on the depth occupancy projection map and the intensity projection map to obtain a mixed projection map, including: based on the first weight value and the second weight value, weighted processing is performed on the depth occupancy projection map and the equalized projection map to obtain a mixed projection map.

[0066] Since the reflection intensity value of the laser radar is relatively concentrated, the pixel values ​​of each pixel in the obtained intensity projection map are all within a certain range, which is not easy to distinguish accurately. Therefore, the intensity projection map can be processed by histogram equalization to obtain an equalized intensity map, so as to facilitate the mixing of the depth occupancy projection map and the equalized projection map to obtain a mixed projection map. Optionally, the intensity projection map can be processed by normalization and histogram equalization to improve the contrast of the pixels in the image, thereby improving the accuracy of the subsequent detection steps.

[0067] Figure 3 is a schematic diagram of a two-dimensional image according to an embodiment of the present invention, such as Figure 3 As shown, from top to bottom are the intensity map, the depth occupancy projection map, the equalized projection map and the mixed projection map.

[0068] Optionally, based on the pixel values ​​of the mixed projection image, the detection result of the laser radar is determined, including: using a sliding window of a preset size to slide multiple times on the mixed projection image according to preset sliding parameters to obtain multiple target sliding windows; determining the sum of the pixel values ​​of multiple pixel points in each target sliding window to obtain the pixel sum of the target sliding window; and determining the detection result based on the pixel sum of the multiple target sliding windows.

[0069] Optionally, the detection result is determined based on the pixel sum of multiple target sliding windows, including: comparing the pixel sum of each target sliding window with a preset pixel sum, and in response to the pixel sum of the target sliding window being greater than the preset pixel sum, storing the target sliding window in a target set; and determining the detection result based on the number of target sliding windows in the target set.

[0070] Optionally, the detection result is determined based on the number of target sliding windows in the target set, including: in response to the number being greater than a preset number, determining that the laser radar is blocked by the object; in response to the number being less than or equal to a preset number, determining that the laser radar is not blocked by the object.

[0071] The above-mentioned preset size is smaller than the size of the mixed projection image, that is, the width and height of the sliding window are smaller than the width and height of the mixed projection image, so as to obtain multiple target sliding windows, and further judge whether the laser radar is blocked by the object by counting the sum of the pixel values ​​of the pixel points corresponding to the target sliding windows.

[0072] Through sliding processing, the pixel points of the target sliding window are determined, and then the sum of the pixel values ​​of multiple pixel points of the target sliding window is determined, and the target sliding windows whose pixel sum is greater than the preset pixel sum are stored. Further, the number of target sliding windows whose pixel sum is greater than the preset pixel sum is determined. When the number is greater than the preset number, it is determined that the laser radar is blocked by an object. Through sliding window processing, the blocked area can be effectively identified. By calculating the pixel sum, the degree of occlusion can be quantified to provide accurate occlusion detection feedback. By setting the preset pixel sum, non-blocked interference can be filtered out to ensure the accuracy of the detection result.

[0073] Figure 4 is a schematic diagram of a sliding window according to an embodiment of the present invention, such as Figure 4 As shown, Figure 4 The figure shows the process of sliding a sliding window of size (m, n) four times on the mixed projection image. The width of the sliding window is m and the height is n. x and y are the preset sliding parameters, x is the step size of sliding in the horizontal direction, and y is the step size of sliding in the vertical direction.

[0074] According to one aspect of an embodiment of the present invention, a laser radar detection device is provided, which can execute the laser radar detection method in the above embodiment. The specific implementation scheme and application scenario in this embodiment are the same as those in the above embodiment and will not be repeated here.

[0075] Figure 5 is a schematic diagram of a detection device of a laser radar according to an embodiment of the present invention, such as Figure 5 As shown, the device includes: an acquisition module 52, which is used to acquire a frame of point cloud data collected by the laser radar; a projection module 54, which is used to project the point cloud data into a two-dimensional image based on the properties of the laser radar, and obtain a depth occupancy projection map and an intensity projection map, wherein the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data; a mixing module 56, which is used to mix the depth occupancy projection map and the intensity projection map to obtain a mixed projection map; a determination module 58, which is used to determine the detection result of the laser radar based on the pixel value of the mixed projection map, wherein the detection result is used to characterize whether the laser radar is blocked by an object.

[0076] Optionally, the attributes of the laser radar include: pitch angle, horizontal angle, number of horizontal points and number of lines. The number of horizontal points is used to represent the number of data points collected by the laser radar in the horizontal direction, and the number of lines is used to represent the number of scanning lines scanned by the laser radar in the vertical direction.

[0077] Optionally, the projection module is also used to generate a two-dimensional image based on the number of horizontal points and the number of lines, wherein the width of the two-dimensional image corresponds to the number of horizontal points, and the height of the two-dimensional image corresponds to the number of lines; based on the pitch angle and the horizontal angle, the depth values ​​of different data points are projected into the two-dimensional image to obtain a depth occupancy projection map; based on the pitch angle and the horizontal angle, the intensity values ​​of different data points are projected into the two-dimensional image to obtain an intensity projection map.

[0078] Optionally, the projection module is also used to determine the target pixel points corresponding to different data points based on the pitch angle and the horizontal angle; in response to the depth value of the different data point being greater than the preset depth value, determine the pixel value of the target pixel point to be a first preset value; in response to the depth value of the different data point being less than or equal to the preset depth value, determine the pixel value of the target pixel point to be a second preset value, wherein the first preset value is greater than the second preset value.

[0079] Optionally, the projection module is further used to determine the target pixel points corresponding to different data points based on the pitch angle and the horizontal angle; and determine that the pixel value of the target pixel point is the intensity value of the data point.

[0080] Optionally, the mixing module is also used to determine a first weight value corresponding to the depth occupancy projection map and a second weight value corresponding to the intensity projection map based on the usage scenario of the laser radar; based on the first weight value and the second weight value, the depth occupancy projection map and the intensity projection map are weighted to obtain a mixed projection map.

[0081] Optionally, the device also includes: a balancing module.

[0082] The equalization module is used to normalize the intensity projection map to obtain a normalized projection map; perform histogram equalization on the normalized projection map to obtain an equalized projection map; based on a first weight value and a second weight value, perform weighted processing on the depth occupancy projection map and the intensity projection map to obtain a mixed projection map, including: based on the first weight value and the second weight value, perform weighted processing on the depth occupancy projection map and the equalized projection map to obtain a mixed projection map.

[0083] Optionally, a determination module is used to use a sliding window of a preset size to slide multiple times on the mixed projection image according to preset sliding parameters to obtain multiple target sliding windows; determine the sum of pixel values ​​of multiple pixel points in each target sliding window to obtain the pixel sum of the target sliding window; and determine the detection result based on the pixel sum of the multiple target sliding windows.

[0084] Optionally, the determination module is also used to compare the pixel sum of each target sliding window with a preset pixel sum, and in response to the pixel sum of the target sliding window being greater than the preset pixel sum, store the target sliding window in a target set; and determine the detection result based on the number of target sliding windows in the target set.

[0085] Optionally, the determination module is further used to determine that the laser radar is blocked by an object in response to the number being greater than a preset number; and to determine that the laser radar is not blocked by an object in response to the number being less than or equal to a preset number.

[0086] An embodiment of the present application further provides an electronic device, comprising: a memory storing an executable program; and a processor for running the program, wherein the program executes the methods in various embodiments of the present invention when running.

[0087] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored executable program, wherein when the executable program is running, the device where the computer-readable storage medium is located is controlled to execute the methods in various embodiments of the present invention.

[0088] An embodiment of the present application further provides a computer program product, including a computer program, which implements the methods in various embodiments of the present invention when executed by a processor.

[0089] An embodiment of the present application further provides a computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium is used to store a computer program, and when the computer program is executed by a processor, the method in each embodiment of the present invention is implemented.

[0090] The embodiments of the present application further provide a computer program, which implements the methods in the above-mentioned embodiments of the present invention when executed by a processor.

[0091] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0092] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0093] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0094] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0095] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0096] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A laser radar detection method, characterized in that: include: Get a frame of point cloud data collected by the laser radar; Based on the properties of the laser radar, the point cloud data is projected into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map, wherein the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data; Mixing the depth occupancy projection map and the intensity projection map to obtain a mixed projection map; Based on the pixel values ​​of the mixed projection image, the detection result of the laser radar is determined, wherein the detection result is used to characterize whether the laser radar is blocked by an object.

2. The method according to claim 1, characterized in that The attributes of the laser radar include: pitch angle, horizontal angle, number of horizontal points and number of lines, wherein the number of horizontal points is used to characterize the number of data points collected by the laser radar in the horizontal direction, and the number of lines is used to characterize the number of scanning lines scanned by the laser radar in the vertical direction; based on the attributes of the laser radar, the point cloud data is projected into a two-dimensional image to obtain a depth occupancy projection map and an intensity projection map, including: Based on the number of horizontal dots and the number of lines, generating the two-dimensional image, wherein the width of the two-dimensional image corresponds to the number of horizontal dots, and the height of the two-dimensional image corresponds to the number of lines; Based on the pitch angle and the horizontal angle, projecting the depth values ​​of the different data points into the two-dimensional image to obtain the depth occupancy projection map; Based on the pitch angle and the horizontal angle, the intensity values ​​of the different data points are projected into the two-dimensional image to obtain the intensity projection graph.

3. The method according to claim 2, characterized in that The step of projecting the depth values ​​of the different data points into the two-dimensional image based on the pitch angle and the horizontal angle to obtain the depth occupancy projection map includes: Based on the pitch angle and the horizontal angle, determining target pixel points corresponding to the different data points; In response to the depth value of the different data point being greater than a preset depth value, determining the pixel value of the target pixel point to be a first preset value; In response to the depth value of the different data point being less than or equal to the preset depth value, determining the pixel value of the target pixel point to be a second preset value, wherein the first preset value is greater than the second preset value.

4. The method according to claim 2 or 3, characterized in that: The step of projecting the intensity values ​​of the different data points onto the two-dimensional image based on the pitch angle and the horizontal angle to obtain the intensity projection graph comprises: Based on the pitch angle and the horizontal angle, determining target pixel points corresponding to the different data points; Determine the pixel value of the target pixel point as the intensity value of the data point.

5. The method according to any one of claims 1 to 4, characterized in that: The step of mixing the depth occupancy projection map and the intensity projection map to obtain a mixed projection map includes: Based on the usage scenario of the laser radar, determine a first weight value corresponding to the depth occupancy projection map and a second weight value corresponding to the intensity projection map; Based on the first weight value and the second weight value, weighted processing is performed on the depth occupancy projection map and the intensity projection map to obtain the mixed projection map.

6. The method according to claim 5, characterized in that The method further comprises: Normalizing the intensity projection map to obtain a normalized projection map; Performing histogram equalization on the normalized projection image to obtain a equalized projection image; The method of performing weighted processing on the depth occupancy projection map and the intensity projection map based on the first weight value and the second weight value to obtain the mixed projection map comprises: performing weighted processing on the depth occupancy projection map and the equalized projection map based on the first weight value and the second weight value to obtain the mixed projection map.

7. The method according to any one of claims 1 to 6, characterized in that: The step of determining the detection result of the laser radar based on the pixel value of the mixed projection image includes: Using a sliding window of a preset size to slide multiple times on the mixed projection image according to preset sliding parameters to obtain multiple target sliding windows; Determine the sum of pixel values ​​of multiple pixel points in each target sliding window to obtain the pixel sum of the target sliding window; The detection result is determined based on the pixel sums of the multiple target sliding windows.

8. The method according to claim 7, characterized in that The step of determining the detection result based on the pixel sum of the multiple target sliding windows includes: Comparing the pixel sum of each target sliding window with a preset pixel sum, and in response to the pixel sum of the target sliding window being greater than the preset pixel sum, storing the target sliding window into the target set; The detection result is determined based on the number of target sliding windows in the target set.

9. The method according to claim 8, characterized in that The determining the detection result based on the number of target sliding windows in the target set includes: In response to the number being greater than a preset number, determining that the laser radar is blocked by an object; In response to the number being less than or equal to the preset number, it is determined that the laser radar is not blocked by the object.

10. A laser radar detection device, characterized in that: include: The acquisition module is used to obtain a frame of point cloud data collected by the laser radar; A projection module, used to project the point cloud data into a two-dimensional image based on the properties of the laser radar to obtain a depth occupancy projection map and an intensity projection map, wherein the depth occupancy projection map is used to characterize the depth values ​​of different data points in the point cloud data, and the intensity projection map is used to characterize the intensity values ​​of different data points in the point cloud data; A mixing module, used for mixing the depth occupancy projection map and the intensity projection map to obtain a mixed projection map; A determination module is used to determine the detection result of the laser radar based on the pixel value of the mixed projection image, wherein the detection result is used to characterize whether the laser radar is blocked by an object.

11. An electronic device, characterized in that: include: A memory storing an executable program; A processor, configured to run the program, wherein the program executes the method according to any one of claims 1 to 9 when running.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored executable program, wherein when the executable program is executed, the device where the storage medium is located is controlled to execute the method according to any one of claims 1 to 9.

13. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images