Laser radar blind compensation method, device and equipment and storage medium

Abstract

The invention provides a laser radar blind compensation method and device, equipment and a storage medium, relates to the technical field of laser radars, and solves the problem that unmanned equipment is difficult to expand the equipment sensing range in the related art. Through calibration of mirror plane parameters and rasterization of a sensing area corresponding to the laser radar, a two-dimensional grid map associated with the grid area and the mirror plane parameters is established, so that real coordinates of a laser beam reflected by a reflector are determined, blind area compensation is realized, and the blind area detection accuracy is improved. Therefore, the sensing range of the equipment when the equipment uses the laser radar for sensing is effectively expanded, accurate detection is realized, a large number of laser radars do not need to be arranged at the same time, and the landing cost of commercial equipment is reduced.

Description

Technical Field

[0001] The present application relates to the field of laser radar technology, and in particular to a laser radar blind spot filling method, device, equipment and storage medium. Background Art

[0002] In the field of autonomous driving, LiDAR plays an important role in mapping, positioning and perception. Unmanned vehicles, robots and other unmanned driving equipment use LiDAR's high ranging accuracy and high stability. It actively transmits and receives lasers to measure the laser flight time to detect the environment around the equipment. However, LiDAR is relatively expensive. As unmanned vehicles, robots and other equipment gradually move toward commercial use, they place higher requirements on product costs and environmental perception capabilities.

[0003] Due to the influence of product structure, LiDAR device process and other aspects, the deployment location of LiDAR for unmanned vehicles, robots and other unmanned driving equipment may not necessarily reflect their ability to perceive the environment well, resulting in problems that cannot be perceived. For example, the scanning area of ​​LiDAR cannot completely cover the area of ​​interest of the equipment, too many laser beams are irradiated on areas that do not need to be paid attention to during the movement of the equipment, or the laser beams are blocked by the structure of the equipment itself.

[0004] In this regard, in order to further enhance the perception capabilities of unmanned vehicles, robots and other unmanned driving equipment, related technologies usually adopt a multi-sensor fusion and complementary approach, such as installing other sensors such as laser radar or ultrasonic radar at multiple locations on the equipment, and then complementing each other in space to achieve large-scale or even blind-angle perception. However, the solutions of related technologies are relatively costly, and as the number of sensors increases, the complexity of the equipment's processing of data obtained by different sensors increases dramatically, making it difficult to implement the technical solutions, making it difficult to achieve low-cost and efficient expansion of the equipment's perception range, and not conducive to the promotion and commercial use of the product. Summary of the invention

[0005] The present application provides a laser radar blind spot filling method, device, equipment and storage medium, which solves the problem of difficulty in expanding the perception range of unmanned driving equipment in related technologies. This solution can effectively improve the perception range of the equipment when using laser radar for perception, achieve accurate detection, and reduce the cost of commercial use of the equipment.

[0006] In a first aspect, the present application provides a laser radar blind spot filling method, which is applied to a laser positioning device, wherein the laser positioning device includes a laser radar and a reflector, wherein the reflector is arranged on a radiation path of the laser radar, and the reflector is used to reflect a laser beam to an area outside the irradiation range of the laser radar, and the laser radar blind spot filling method includes:

[0007] Extracting ROI point cloud data acquired by the laser radar in the region of interest, and clustering the ROI point cloud data to determine several groups of clustered point cloud data;

[0008] Performing plane fitting on the clustered point cloud data, and determining the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting;

[0009] Rasterizing the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area;

[0010] Determining a target grid region corresponding to the reflector on the two-dimensional grid map, and associating the mirror plane parameters with the target grid region;

[0011] When it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, the real coordinates of the laser point are determined based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area.

[0012] In a second aspect, the present application further provides a laser radar blind spot filling device, which is applied to a laser positioning device. The laser positioning device includes a laser radar and a reflector. The reflector is arranged on the radiation path of the laser radar. The reflector is used to reflect the laser beam to an area outside the irradiation range of the laser radar. The laser radar blind spot filling device includes:

[0013] A point cloud data acquisition module is configured to extract ROI point cloud data acquired by the laser radar in the region of interest, and cluster the ROI point cloud data to determine a plurality of groups of clustered point cloud data;

[0014] A plane parameter acquisition module is configured to perform plane fitting on the clustered point cloud data, and determine the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting;

[0015] A rasterization module configured to rasterize a sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area;

[0016] a parameter association module configured to determine a target grid region corresponding to the reflector on the two-dimensional grid map and associate the mirror plane parameters with the target grid region;

[0017] The coordinate output module is configured to determine the real coordinates of the laser point based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area when it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map.

[0018] In a third aspect, the present application also provides a laser positioning device, which includes:

[0019] one or more processors;

[0020] A storage device is used to store one or more programs. When the one or more programs are executed by one or more processors, the one or more processors implement the above-mentioned laser radar blind spot filling method.

[0021] In a fourth aspect, the present application also provides a storage medium storing computer executable instructions, which are used to execute the above-mentioned laser radar blind spot filling method when executed by a processor.

[0022] The present application scheme utilizes the strong reflective ability of the mirror to the laser beam to reflect the laser beam to the blind spot. By calibrating the mirror plane parameters and rasterizing the perception area corresponding to the laser radar, a two-dimensional grid map associating the grid area with the mirror plane parameters is established. This is to facilitate determining the true coordinates of the laser beam reflected by the reflector, thereby filling the blind spot, and effectively improving the perception range of the equipment when using the laser radar for perception. While achieving accurate detection, there is no need to deploy a large number of laser radars, which helps to reduce the cost of commercializing the equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A schematic diagram of the structure of a laser radar and a reflector in a laser positioning device provided in one embodiment of the present application;

[0024] Figure 2 A schematic diagram of the structure of a laser radar and a reflector in another laser positioning device provided in an embodiment of the present application;

[0025] Figure 3 A flowchart of the steps of a laser radar blind spot filling method provided in one embodiment of the present application;

[0026] Figure 4 A schematic diagram of rasterization of a perception area provided in an embodiment of the present application;

[0027] Figure 5 A flowchart of filtering steps provided in an embodiment of the present application;

[0028] Figure 6 A flowchart of the steps for determining mirror plane parameters provided in one embodiment of the present application;

[0029] Figure 7 A flowchart of the steps for calculating the real coordinates of a laser point provided in an embodiment of the present application;

[0030] Figure 8 A schematic diagram of the structure of a laser radar blind spot compensation device provided in one embodiment of the present application;

[0031] Fig. 9 A schematic diagram of the structure of a laser positioning device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are only used to explain the embodiments of the present application, rather than to limit the embodiments of the present application. It should also be noted that, for ease of description, only the parts related to the embodiments of the present application rather than all structures are shown in the accompanying drawings, and those skilled in the art should be able to think of it after reading the specification of this application that as long as the technical features do not contradict each other, any combination of the technical features can constitute an optional implementation method.

[0033] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable when appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally a class, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally represents that the objects associated before and after are in an "or" relationship. In the description of the present application, "multiple" means two or more, and "several" means one or more.

[0034] LiDAR is widely used in the field of autonomous driving for mapping, positioning, and perception due to its high ranging accuracy and high stability. However, LiDAR is expensive, and in the process of commercializing LiDAR, better requirements are put forward for its use, such as achieving stronger environmental perception capabilities at the lowest cost.

[0035] When using LiDAR on unmanned vehicles, robots and other equipment, there may be problems with not being able to perceive the environment well due to the deployment location of the LiDAR, such as blind spots, the scanning area cannot cover the area of ​​interest of the device (such as the area on the left and right sides of the forward direction), and the laser beam is blocked by the structure of the device itself. In related technologies, in order to improve the perception ability of the device, sensors such as LiDAR and ultrasonic radar are usually installed at multiple locations on the device to complement each other in space and achieve a wider range of detection and perception. However, as the number of sensors increases, the cost of the equipment is also higher, the data it obtains is also more and there is duplicate data, which increases the complexity of data processing, making it more difficult to implement the technical solution and difficult to achieve a low-cost and efficient expansion of the device perception range.

[0036] In this regard, the present application provides a laser radar blind spot filling method, which is applied to a laser positioning device, wherein the laser positioning device includes a laser radar and a reflector, wherein the reflector is arranged on the radiation path of the laser radar, and the reflector is used to reflect the laser beam to an area outside the irradiation range of the laser radar. It is conceivable that the laser positioning device can be the above-mentioned unmanned vehicle, robot and other unmanned driving equipment (hereinafter referred to as equipment). The following shows the structure of the laser radar and the reflector in some laser positioning devices and a schematic diagram of the corresponding laser beam optical path.

[0037] Figure 1 A schematic diagram of the structure of a laser radar and a reflector in a laser positioning device provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the laser radar 101 is arranged above the device body, that is, the irradiation direction of the laser radar 101 is from top to bottom to irradiate the surface 201 of the object (such as the ground), and the reflector 102 is arranged on the side of the laser radar to reflect the corresponding laser beam to a partial area, thereby achieving blind spot filling. The number of the reflectors 102 can be set according to the blind area to be filled. If the blind area is small, 1 to 2 reflectors can be set, and if the blind area is large, more than 2 reflectors can be set. Figure 1 The first dotted area 301 corresponds to the blocked part of the device body. Due to the blocking of the laser beam by the blocked part of the device body, such as the device housing, some extended structures on the device housing, etc., it is difficult for the laser beam to directly irradiate the corresponding area (i.e., the blind area), such as the area where point 1 and point 2 are located. Figure 1 The second dotted area 302 represents the region of interest. Accordingly, if the left direction is taken as the forward direction, the region of interest can cover the forward direction and the blind spot of the device.

[0038] Figure 2 A schematic diagram of the structure of a laser radar and a reflector in another laser positioning device provided in an embodiment of the present application, such as Figure 2 As shown, the illumination direction of the laser radar 101 is from bottom to top and illuminates the surface 201 of the object (such as the ground) through the reflectors 102 arranged on it and on the surrounding sides. Figure 2 Similarly, the first dotted area 301 corresponds to the shielding portion on the device body. Due to the shielding of the laser beam by the shielding portion, it is difficult for the laser beam to directly irradiate the corresponding area (ie, the blind area), such as the area where point 1 and point 2 are located. Figure 2 The second dotted area 302 represents the region of interest. Accordingly, if the left direction is taken as the forward direction, the region of interest can cover the forward direction and the blind spot of the device.

[0039] It should be noted that the above example is a partial setting method of setting a laser radar and a reflector on the device. In actual applications, the laser radar and the reflector can also be set on the side of the device or other directions, so as to directly illuminate the area to be detected by the laser radar and / or reflect the laser beam by the reflector. Accordingly, the inclination angle of the reflector can be set according to the application scenario of the device so that the reflector can reflect the laser beam to the blind spot.

[0040] It is understandable that the device can use a reflector to reflect the laser beam to an area outside the irradiation range of the laser radar (i.e., a blind area). In this regard, during the data processing process, the acquired point cloud data is clustered to determine the mirror plane where the corresponding mirror point is located, and then the mirror plane parameters corresponding to the reflector are determined, so that the target grid area corresponding to the reflector is marked on the two-dimensional grid map obtained after rasterizing the sensing area of ​​the laser radar, so as to determine the coordinates of the laser point reflected by the laser beam irradiated on the reflector, thereby achieving blind spot compensation for the laser radar. As shown in the figure, Figure 3 This is a flowchart of a laser radar blind spot filling method provided in an embodiment of the present application. The laser radar blind spot filling method includes steps S110-S150, which are specifically as follows:

[0041] Step S110: extracting ROI point cloud data acquired by the laser radar in the region of interest, and clustering the ROI point cloud data to determine a plurality of groups of clustered point cloud data.

[0042] When laser radar is used in unmanned vehicles, robots and other unmanned driving equipment, laser radar and reflectors are set on the unmanned driving equipment, such as Figure 1 or Figure 2 Accordingly, for the setting of the region of interest (ROI) of the device, it can be imagined that the region of interest is a spatial area outside the device, and when setting the region of interest, the blind area of ​​the laser radar can be included in the region of interest. Optionally, in one embodiment, it can also include the detection area of ​​the laser positioning device.

[0043] Then, the region of interest is scanned by a laser radar, and the ROI (Region of Interest) point cloud data is extracted, that is, the point cloud data corresponding to the region of interest is obtained. The point cloud data is a vector set in a three-dimensional coordinate system, which is recorded in the form of points, and each point contains three-dimensional coordinates. In addition, in some embodiments, it may also include color information or reflection intensity information. It can be imagined that the amount of data included in the ROI point cloud data is large and corresponds to many objects, such as point cloud data corresponding to reflectors, point cloud data corresponding to mirror points, point cloud data corresponding to the ground, point cloud data corresponding to obstacles, etc.

[0044] In this regard, the obtained ROI point cloud data is clustered to obtain several groups of clustered point cloud data. It is conceivable that the corresponding clustering algorithms such as K-mears algorithm, FCM algorithm, etc. are used to classify the data according to corresponding parameters such as distance, similarity, etc., so as to determine the data belonging to the same cluster and use it as a group of clustered point cloud data. Therefore, the ROI point cloud data is clustered to facilitate segmentation of several groups of clustered point cloud data, so that the required point cloud data can be separated therefrom, that is, the same group of clustered point cloud data corresponding to the points reflected by the same mirror is determined therefrom.

[0045] Step S120 , performing plane fitting on the clustered point cloud data, and determining the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting.

[0046] According to the preset plane fitting algorithm, such as RANSAC (Random Sample Consensus) algorithm, PCA (Principal Component Analysis) algorithm, least squares method, etc., the divided clustered point cloud data is plane fitted to determine the plane parameters of each group of clustered point cloud data, wherein the plane parameters include the normal vector and intercept corresponding to the X, Y, and Z planes. Therefore, the corresponding plane can be uniquely determined by the above parameters in the three-dimensional coordinate system.

[0047] Then, a plane fitting algorithm is used to fit the plane corresponding to the clustered point cloud data to obtain the corresponding plane parameters, thereby determining the mirror plane parameters corresponding to the reflector. It is conceivable that, in some embodiments, the point cloud data corresponding to the reflector is relatively complex, and is mixed with a lot of point cloud data corresponding to other objects, making it difficult to extract. Clustering segmentation can then be used to obtain better extracted point cloud data of the corresponding mirror points. When fitting, the device can perform plane fitting on the point cloud data corresponding to the mirror points, thereby determining the plane parameters of the mirror plane. The mirror plane is the image of the surface of an object on the reflector. For example, in a scenario where the device uses a laser radar and a reflector to detect a blind spot in the forward direction, the reference plane detected is the ground. Correspondingly, the mirror plane is the image of the ground formed on the reflector. Figure 1 , Figure 2 The midpoint 1 and point 2 (both are the points on the ground where the laser beam is reflected by the reflector) are the mirror points of point 1 and point 2 formed by the reflector. The plane where the mirror points of point 1 and point 2 are located is the mirror plane that is mirror-symmetrical to the reference plane (i.e., the ground).

[0048] In this regard, after determining the plane parameters of the mirror plane, the plane parameters of the reflector (i.e., the mirror plane parameters) can be calculated based on the plane parameters. For example, in one embodiment, the mirror symmetry relationship between the mirror plane and the reference plane is used to combine the plane parameters of the two to calculate the mirror plane parameters of the corresponding reflector. It can be imagined that the plane parameters can be stored in the form of an array, or in the form of a vector, a matrix, etc., as long as they can record the corresponding parameters. For example, a plane parameter is represented by [a, b, c, d], where a, b, c represent the normal vector of the plane, and d represents the plane intercept. Of course, in some embodiments, in order to better distinguish different plane parameters, such as distinguishing the plane parameters of the reflector, the plane parameters of the mirror plane formed, etc., it can be used. Indicates that a i m , b i m 、c i m represents the normal vector of the mirror plane corresponding to the i-th mirror, d i m is the plane intercept, i represents the mirror ID (identification) number, and m represents that it is a mirror plane.

[0049] Step S130: rasterize the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area.

[0050] Due to the limitations of the laser radar's illumination angle (such as the horizontal field of view angle, the vertical field of view angle) and the measurement distance, the determined laser radar's illumination range is limited. Correspondingly, its perception area can be determined based on the horizontal field of view angle and the vertical field of view angle. In this regard, the corresponding surface can be selected as the perception area for rasterization. It can be imagined that in the spherical coordinate system centered on the laser radar, according to the horizontal field of view angle and the vertical field of view angle of the laser radar, combined with the measurement distance, a target surface can be determined in the spherical coordinate system as the perception area, and then rasterized. Of course, it is also possible to select any length within the measurement distance to determine another surface for rasterization.

[0051] In the process of rasterization, the size of the grid area can be the same, and the specific area size can be set according to the actual application, such as the area defined by the horizontal field of view and the vertical field of view of the unit angle (such as 1°). Each grid area can also be a larger area formed by merging the above-mentioned areas of the same size, resulting in inconsistent final area sizes.

[0052] In one embodiment, if Figure 4 As shown, Figure 4 A schematic diagram of rasterization of a perception area provided in an embodiment of the present application, in Figure 4 In the figure, point O represents the position of the laser radar, and the target distance (such as the measurement distance) is selected for the laser radar's irradiation range. For example, the laser radar's irradiation range is 360° in the horizontal direction and 180° in the vertical direction. Based on the horizontal field of view and the vertical field of view of the laser radar, a target surface can be determined, and then the surface is used as the sensing area and rasterized. The sensing area is as follows: Figure 4 The hemispherical curved area 401 is shown.

[0053] According to the horizontal and vertical field of view angles, the sensing area of ​​the laser radar can be divided into multiple grid areas. For example, a grid area is determined for every 1° interval in the horizontal and vertical field of view angles, that is, the span of the grid area in the horizontal and vertical directions is 1°, and then the entire sensing area is divided into multiple grid areas to achieve gridding, thereby obtaining the corresponding two-dimensional grid map. In this regard, in order to avoid showing multiple grid areas on the map and making the attached figure unclear, Figure 4 The line area 402 in the figure briefly shows one of the grid areas, and the other grid areas are similarly divided according to the areas defined by the corresponding horizontal field of view angles and vertical field of view angles.

[0054] Therefore, through the two-dimensional grid map, the device can know the grid area corresponding to each point, so as to quickly locate whether there is reflection at the laser point and the corresponding mirror surface when there is reflection, which helps to improve efficiency.

[0055] It should be noted that, in some embodiments, the scheme for dividing the grid areas in rasterization can also be to divide the corresponding grid areas according to the sizes of reflectors and other objects, thereby dividing the perception area of ​​the lidar into multiple grid areas of different sizes to obtain a corresponding two-dimensional grid map.

[0056] Step S140: determining a target grid area corresponding to the reflector on the two-dimensional grid map, and associating the mirror plane parameters with the target grid area.

[0057] For the target grid area corresponding to the reflector, the position corresponding to the reflector can be determined in the two-dimensional grid map according to the angle of the reflector in the horizontal and vertical directions within the irradiation range of the laser radar, and then the occupied area is divided into the target grid area. In addition, the target grid area is associated with the mirror plane parameters of the corresponding reflector, so that the mirror plane parameters are associated with the target grid area, for example, a corresponding mapping relationship is established, such as setting a corresponding area ID for the target grid area and binding the mirror plane parameters to the area ID, so that the two are associated. Accordingly, the mapping relationship can be stored in the storage location of the device in the form of an array, text, etc., so that it is convenient to query and determine the corresponding mirror plane parameters when the position of the laser point is determined later.

[0058] Step S150: When it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, the real coordinates of the laser point are determined based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area.

[0059] After the device receives the three-dimensional coordinates of the laser point, according to the grid area corresponding to the three-dimensional coordinates of the laser point, the three-dimensional coordinates of the laser point can be converted into coordinates in a spherical coordinate system centered on the laser radar, so as to determine the angles in the horizontal and vertical directions, and then determine the grid area corresponding to it on the two-dimensional grid map. When it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, it can be determined that the laser point is a mirror point. In this regard, the three-dimensional coordinates of the laser point are converted according to the mirror plane parameters, such as converting the three-dimensional coordinates according to the corresponding calculation formula, so as to obtain the real coordinates of the laser point, that is, to obtain the three-dimensional coordinates of the landing point reflected on the surface of the object.

[0060] From the above scheme, it can be seen that the equipment can use the strong reflective ability of the mirror to the laser beam to reflect the laser beam to the blind spot. By calibrating the mirror plane parameters and rasterizing the perception area corresponding to the laser radar, a two-dimensional grid map associating the grid area with the mirror plane parameters is established. This is to facilitate determining the true coordinates of the laser beam reflected by the reflector, thereby filling the blind spot and effectively improving the perception range of the equipment when using the laser radar for perception. While achieving accurate detection, there is no need to deploy a large number of laser radars, which helps to reduce the cost of commercial equipment.

[0061] In one embodiment, the unmanned driving device sets a corresponding area of ​​interest, such as the area of ​​interest includes the area to be detected by the laser positioning device, for example, the area on one side of the forward direction, the blind spot in the forward direction, etc. The unmanned driving device performs laser scanning on the area of ​​interest through the laser radar on it. The laser radar emits a laser beam and receives the reflected signal. After the laser beam is irradiated to the surface of the object in the space environment, the laser radar can receive the reflected signal after reflection, thereby obtaining the corresponding point cloud data. It is conceivable that in some embodiments, in order to better improve the precision and accuracy of the point cloud data, the posture correction can also be performed by sensor-assisted methods, such as calculating the laser radar posture through auxiliary methods such as inertial measurement unit (IMU) and wheel odometer (Odometry, Odom) for correction, thereby achieving the effect of improving the precision and accuracy of the point cloud data.

[0062] The characteristics of mirror reflection include that when originally adjacent points are reflected by the same mirror, their image points are still adjacent; when originally adjacent points are reflected by different mirrors, their image points are not adjacent; and the mirror image of a plane is still a plane.

[0063] In this regard, after extracting the corresponding point cloud data for the region of interest corresponding to the laser radar, that is, after obtaining the ROI point cloud data, the ROI point cloud data model is clustered according to the point cloud distance and the plane normal vector to divide several groups of clustered point cloud data. It can be understood that after the reflector is installed in the device and adjusted to a suitable position, the surface of the object on which the device is traveling is used as a reference plane (such as the ground, etc.), and the plane parameters corresponding to the plane can be determined, such as calculating the corresponding plane parameters after obtaining the corresponding point cloud data through the laser radar. In this regard, according to the point cloud distance, the device can determine the data belonging to the same cluster through the above-mentioned clustering algorithm, such as the clustered point cloud data corresponding to different reflectors. Similarly, according to the plane parameters, the device can determine the corresponding clustered point cloud data through the corresponding clustering algorithm, such as the clustered point cloud data corresponding to the reference plane.

[0064] Therefore, the ROI point cloud data is clustered to segment several groups of clustered point cloud data, so that the required point cloud data can be separated therefrom, which helps to better fit the corresponding plane to the point cloud data and make the corresponding plane parameters more accurate.

[0065] In some embodiments, if there is noise and / or clutter in the acquired ROI point cloud data, filtering processing is required before clustering the ROI data, that is, filtering the extracted ROI point cloud data to remove the noise data in the ROI point cloud data. For example, in one embodiment, through a statistical outlier removal algorithm (SOR), the unmanned driving device can filter out the noise in the ROI point cloud data accordingly and remove the noise to improve the accuracy of the point cloud data, such as Figure 5 As shown, Figure 5 The flowchart of the filtering process provided in one embodiment of the present application includes steps S210-220, which are as follows:

[0066] Step S210: Based on a statistical filtering algorithm and according to a preset distance threshold, outliers outside the area corresponding to the distance threshold are screened out.

[0067] Step 220: Use the three-dimensional coordinates corresponding to the outlier points as noise data, and delete the noise data from the ROI point cloud data.

[0068] It can be understood that the statistical filtering algorithm is based on the distribution of the distance from the point in the point cloud data to its neighboring points. It can be imagined that the entire point cloud satisfies the Gaussian distribution, that is, its distribution satisfies the corresponding mean μ and standard deviation σ. Correspondingly, the preset distance threshold is μ+σ×std mul , where std mul is the preset standard deviation multiple. Therefore, by calculating the distance from the midpoint of the point cloud data to its neighboring points, and then filtering according to the distance threshold, the point whose distance from the point to its neighboring points is greater than the distance threshold can be considered as an outlier outside the area corresponding to the distance threshold. All outliers can be filtered out by continuously traversing the point cloud data. The three-dimensional coordinates corresponding to the outliers are used as noise data, and the noise data is extracted from the ROI point cloud data in turn, thereby improving the accuracy of the ROI point cloud data, which is helpful for better clustering of the ROI point cloud data in the future to improve the fitting effect.

[0069] It should be noted that, in one embodiment, the noise in the ROI point cloud data can also be removed by using methods such as mean filtering and median filtering to improve the accuracy of the point cloud data, so that more accurate data can be used for subsequent clustering segmentation, which helps to obtain better fitting effects.

[0070] In some embodiments, after acquiring the clustered point cloud data, the unmanned driving device performs plane fitting on different groups of clustered point cloud data to determine the fitted plane and the corresponding plane parameters, and then calculates the mirror plane parameters of the corresponding reflector, such as Figure 6 As shown, Figure 6 The flowchart of the steps for determining the mirror plane parameters provided in an embodiment of the present application includes steps S310-330, which are specifically as follows:

[0071] Step S310: performing plane fitting on different groups of clustered point cloud data to determine first plane parameters corresponding to each group of clustered point cloud data.

[0072] Step S320, determining target plane parameters corresponding to the mirror plane in the first plane parameters, the mirror plane is a plane formed by the reference plane being reflected in the reflector, and the reference plane is a plane where the laser beam lands after being reflected by the reflector.

[0073] Step S330: determining the mirror plane parameters corresponding to the reflector based on the target plane parameters and the second plane parameters, where the second plane parameters are the plane parameters of the reference plane.

[0074] It is understandable that after clustering and segmenting the ROI point cloud data, the different groups of clustered point cloud data determined can be fitted with fitting planes accordingly, such as the plane corresponding to the mirror plane and the plane of the object surface. In this regard, the device can perform plane fitting on different groups of clustered point cloud data, that is, according to the corresponding plane fitting algorithm, such as the Chebyshev plane fitting algorithm, the plane fitting algorithm based on singular value decomposition (SVD), etc., so as to determine the first plane parameters corresponding to each group of clustered point cloud data. It can be imagined that the plane parameters include the normal vectors and intercepts corresponding to the X, Y, and Z planes, so that the corresponding plane can be uniquely determined by the plane parameters in the three-dimensional coordinate system.

[0075] In practical applications, when the blind spot to be filled is fixed, the inclination angle of the reflector is also fixed. The plane where the unmanned driving equipment falls after reflection through the reflector and the plane illuminated by the laser radar are both reference planes. In this regard, after fitting the reference plane by the above-mentioned plane fitting method, the corresponding plane parameters can also be determined. Since the mirror plane and the reference plane are mirror-symmetrical relative to each other, based on this feature, after determining the target plane parameters of the corresponding mirror plane, the plane parameters of each mirror can be calibrated by combining the target plane parameters and the plane parameters of the reference plane. For example, if the reflector is placed horizontally, the mirror plane and the reference plane are parallel planes, and the plane parameters of the mirror can be determined based on the distance between the two planes; if the reflector is placed at an angle, the intersection of the two planes can be calculated to determine a point on the intersection, and the normal vector of the mirror plane can be determined by the vector method, and then the plane equation of the mirror can be constructed by the point method to determine the plane parameters of the mirror.

[0076] Therefore, by utilizing the mirror plane and the reference plane, after plane fitting, the device can quickly determine the plane parameters of the mirror plane and record the mirror plane parameters to complete calibration, so that blind spots can be quickly and accurately filled.

[0077] In one embodiment, after completing the calibration of the two-dimensional grid map, that is, determining the grid area on the two-dimensional grid map and the corresponding plane parameters, the unmanned driving device can calculate the three-dimensional coordinates of the acquired laser point based on the two-dimensional grid map. It can be imagined that if the laser point is not reflected by a reflector, the obtained three-dimensional coordinates are the real coordinates of the laser point, that is, no additional calculation processing is required. For the case where the laser point is reflected by a reflector, such as Figure 7 As shown, Figure 7The flowchart of the steps for calculating the real coordinates of the laser point provided in an embodiment of the present application, after the device obtains the three-dimensional coordinates of the laser point, when it determines that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, it performs corresponding calculations on the three-dimensional coordinates of the laser point, and the calculation process includes steps S410-430, which are specifically as follows:

[0078] Step S410: Determine a first distance from the corresponding point to the mirror surface according to the three-dimensional coordinates of the laser point and the mirror plane parameters.

[0079] Step S420, extracting the normal vector from the mirror plane parameters, and using the product of the first matrix corresponding to the normal vector and the second distance as the spacing matrix, where the second distance is twice the first distance.

[0080] Step S430: Calculate the sum of the spacing matrix and the second matrix corresponding to the three-dimensional coordinates of the laser point to determine the real coordinates of the laser point.

[0081] It can be understood that after obtaining the three-dimensional coordinates of the laser point, the corresponding horizontal field of view angle and vertical field of view angle can be determined by coordinate conversion, and then the grid area corresponding to the laser point can be determined on the two-dimensional grid map. If the laser point is located in the target grid area, since the target grid area is associated with the corresponding mirror plane parameters, the device can determine the mirror plane parameters corresponding to the target grid area according to the target grid area corresponding to the laser point. Accordingly, the first distance from the point to the mirror is calculated according to the point-to-plane distance formula. The corresponding calculation formula is as follows:

[0082]

[0083] Where, dist is the distance from the mirror point to its corresponding mirror surface, (x j m ,y j m 、z j m ) is the three-dimensional coordinate of the laser point obtained, [a i , b i , c i , d i ] is the plane parameter of the mirror plane, m indicates that the point is a mirror point, and i indicates the mirror ID number corresponding to the mirror plane. It can be imagined that when the plane normal vector is a unit normal vector, the denominator in the above formula is 1, and there is no need to calculate the modulus length.

[0084] In addition, after determining the first distance from the point to the corresponding mirror surface, the corresponding normal vector is extracted from the mirror plane parameters, that is, the normal vector a corresponding to the X, Y, and Z axes is obtained. i , b i 、c i, and accordingly calculate the product of the first matrix corresponding to the normal vector and the second distance, thereby obtaining the spacing matrix, wherein the second distance is twice dist, that is, twice the first distance. Similarly, add the second matrix corresponding to the three-dimensional coordinates of the laser point to the above spacing matrix, and use the sum of the two to determine the true coordinates of the laser point. Correspondingly, the specific formula is as follows:

[0085]

[0086] Among them, (x j ,y j , z j ) is the real coordinate of the laser point.

[0087] Therefore, through the reflection of the laser beam by the reflector, for the laser point corresponding to the laser beam landing in the blind spot, the device obtains the coordinates of its mirror point and calculates the coordinates of the real point based on the mirror parameters of the reflector, thereby effectively filling the blind spot.

[0088] It can be imagined that, in one embodiment, when calculating the real coordinates of the laser point, the device can construct a plane equation based on the mirror plane parameters, and then based on the principle of equal point-to-plane distance, that is, the distance between the mirror point and the mirror plane and the distance between the real point and the mirror plane are equal, calculate the corresponding point-to-plane distance, and then determine the plane equation translated to the real point; and based on the relationship that the line between the mirror point and the real point is perpendicular to the mirror plane and the coordinates of the mirror point, construct the corresponding straight line equation, thereby analyzing the coordinates of the intersection of the line and the plane to determine the real coordinates of the laser point.

[0089] Figure 8 This is a schematic diagram of the structure of a laser radar blind spot filling device provided in an embodiment of the present application. The device can be applied to the above-mentioned laser positioning device and is used to execute the laser radar blind spot filling method provided in the above-mentioned embodiment, and has the corresponding functional modules and beneficial effects of the execution method. As shown in the figure, the device includes a point cloud data acquisition module 501, a plane parameter acquisition module 502, a gridding module 503, a parameter association module 504 and a coordinate output module 505.

[0090] The point cloud data acquisition module 501 is configured to extract the ROI point cloud data acquired by the laser radar in the region of interest, and cluster the ROI point cloud data to determine a number of groups of clustered point cloud data;

[0091] The plane parameter acquisition module 502 is configured to perform plane fitting on the clustered point cloud data, and determine the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting;

[0092] The rasterization module 503 is configured to rasterize the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area;

[0093] The parameter association module 504 is configured to determine a target grid area corresponding to the reflector on the two-dimensional grid map, and associate the mirror plane parameters with the target grid area;

[0094] The coordinate output module 505 is configured to determine the real coordinates of the laser point based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area when determining that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map.

[0095] In some embodiments, the point cloud data acquisition module 501 is specifically configured as follows:

[0096] The three-dimensional coordinate information after the laser radar performs laser scanning on the area of ​​interest is used as ROI point cloud data, and the area of ​​interest includes the detection area of ​​the laser positioning device;

[0097] According to the point cloud distance and the plane normal vector, the ROI point cloud data is clustered to divide several groups of clustered point cloud data.

[0098] In some embodiments, the point cloud data acquisition module 501 is further specifically configured as follows:

[0099] The extracted ROI point cloud data is filtered to remove noise data in the ROI point cloud data.

[0100] In some embodiments, the point cloud data acquisition module 501 is further specifically configured as follows:

[0101] Based on the statistical filtering algorithm, according to the preset distance threshold, outliers outside the area corresponding to the distance threshold are screened out;

[0102] The 3D coordinates corresponding to the outliers are taken as noise data, and the noise data is deleted from the ROI point cloud data.

[0103] In some embodiments, the plane parameter acquisition module 502 is specifically configured as follows:

[0104] Performing plane fitting on different groups of clustered point cloud data to determine first plane parameters corresponding to each group of clustered point cloud data;

[0105] Determine the target plane parameters corresponding to the mirror plane in the first plane parameters, the mirror plane is the plane formed by the reference plane reflected in the reflector, and the reference plane is the plane where the laser beam falls after being reflected by the reflector;

[0106] Based on the target plane parameters and the second plane parameters, the mirror plane parameters corresponding to the reflector are determined, and the second plane parameters are the plane parameters of the reference plane.

[0107] In some embodiments, the rasterization module 503 is specifically configured as follows:

[0108] Based on the laser radar's irradiation range, determine the laser radar's sensing area on the target surface. The laser radar's irradiation range is determined according to the laser radar's measurement distance, horizontal field of view angle, and vertical field of view angle;

[0109] According to the horizontal field of view angle and the vertical field of view angle, the sensing area of ​​the lidar is divided into multiple grid areas as grid areas on a two-dimensional grid map.

[0110] In some embodiments, the coordinate output module 505 is specifically configured as follows:

[0111] Determine a first distance from the corresponding point to the mirror surface according to the three-dimensional coordinates of the laser point and the mirror plane parameters;

[0112] Extract the normal vector from the mirror plane parameters, and use the product of the first matrix corresponding to the normal vector and the second distance as the spacing matrix, where the second distance is twice the first distance;

[0113] The sum of the spacing matrix and the second matrix corresponding to the three-dimensional coordinates of the laser point is calculated to determine the real coordinates of the laser point.

[0114] It is worth noting that in the embodiment of the above-mentioned device, the modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the modules are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of the present application.

[0115] Fig. 9 A schematic structural diagram of a laser positioning device provided in one embodiment of the present application, the device includes a laser radar and a reflector, the reflector is arranged on the radiation path of the laser radar, the reflector is used to reflect the laser beam to an area outside the irradiation range of the laser radar, and the device is used to execute the laser radar blind spot filling method provided in the above embodiment, and has functional modules and beneficial effects corresponding to the execution method.

[0116] As shown in the figure, the laser positioning device also includes a processor 601, a memory 602, an input device 603 and an output device 604. The number of processors 601 can be one or more, and one processor 601 is taken as an example in the figure; the processor 601, the memory 602, the input device 603 and the output device 604 can be connected via a bus or other means, and the figure takes the connection via a bus as an example. The memory 602, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the program instructions / modules corresponding to the laser radar blind spot filling method in the embodiment of the present application. The processor 601 executes various corresponding functional applications and data processing by running the software programs, instructions and modules stored in the memory 602, that is, realizing the above-mentioned laser radar blind spot filling method.

[0117] The memory 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data recorded or created during use, etc. In addition, the memory 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 602 may further include a memory remotely arranged relative to the processor 601, and these remotely arranged memories may be connected to the device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0118] The input device 603 can be used to input corresponding digital or character information to the processor 601, and to generate key signal input related to the user settings and function control of the device; the output device 604 can be used to send or display key signal output related to the user settings and function control of the device.

[0119] An embodiment of the present application also provides a storage medium storing computer executable instructions, which, when executed by a processor, are used to perform relevant operations in the laser radar blind spot filling method provided in any embodiment of the present application.

[0120] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device.

[0121] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0122] Note that the above are only preferred embodiments of the present application and the technical principles used. Those skilled in the art will understand that the present application is not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the scope of protection of the present application. Therefore, although the present application is described in more detail through the above embodiments, the present application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the present application, and the scope of the present application is determined by the scope of the appended claims.

Claims

1. A laser radar blind spot filling method, characterized in that: Applied to a laser positioning device, the laser positioning device includes a laser radar and a reflector, the reflector is arranged on the radiation path of the laser radar, and the reflector is used to reflect the laser beam to an area outside the irradiation range of the laser radar. The laser radar blind spot filling method includes: Extracting ROI point cloud data acquired by the laser radar in a region of interest, and clustering the ROI point cloud data to determine a plurality of groups of clustered point cloud data; Performing plane fitting on the clustered point cloud data, and determining mirror plane parameters corresponding to the reflector according to plane parameters obtained after plane fitting; Rasterizing the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area; Determining a target grid area corresponding to a reflector on the two-dimensional grid map, and associating the mirror plane parameters with the target grid area; When it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, the real coordinates of the laser point are determined based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area.

2. The laser radar blind spot filling method according to claim 1, characterized in that: The extracting the ROI point cloud data acquired by the laser radar in the region of interest, and clustering the ROI point cloud data to determine a plurality of groups of clustered point cloud data, comprises: The three-dimensional coordinate information after the laser radar performs laser scanning on the region of interest is used as ROI point cloud data, and the region of interest includes the detection area of ​​the laser positioning device; The ROI point cloud data is clustered according to the point cloud distance and the plane normal vector to divide into a plurality of groups of clustered point cloud data.

3. The laser radar blind spot filling method according to claim 1, characterized in that: Before clustering the ROI point cloud data based on the point cloud distance and the plane normal vector to divide the ROI point cloud data into a plurality of groups of clustered point cloud data, the laser radar blind spot filling method further includes: The extracted ROI point cloud data is filtered to remove noise data in the ROI point cloud data.

4. The laser radar blind spot filling method according to claim 3, characterized in that: The filtering the extracted ROI point cloud data to remove noise data in the ROI point cloud data includes: Based on the statistical filtering algorithm, according to the preset distance threshold, outliers located outside the area corresponding to the distance threshold are screened out; The three-dimensional coordinates corresponding to the outlier points are used as the noise point data, and the noise point data are deleted from the ROI point cloud data.

5. The laser radar blind spot filling method according to claim 1, characterized in that: The performing plane fitting on the clustered point cloud data and determining the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting, comprises: Performing plane fitting on different groups of clustered point cloud data to determine first plane parameters corresponding to each group of clustered point cloud data; Determine a target plane parameter corresponding to a mirror plane in the first plane parameter, wherein the mirror plane is a plane formed by the reference plane being reflected in the reflector, and the reference plane is a plane where the laser beam falls after being reflected by the reflector; The mirror plane parameters corresponding to the reflector are determined based on the target plane parameters and the second plane parameters, where the second plane parameters are the plane parameters of the reference plane.

6. The laser radar blind spot filling method according to claim 1, characterized in that: The rasterizing the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area includes: Based on the illumination range of the laser radar, determining the sensing area of ​​the laser radar on the target surface, wherein the illumination range of the laser radar is determined according to the measurement distance, horizontal field of view angle and vertical field of view angle of the laser radar; According to the horizontal field of view angle and the vertical field of view angle, the sensing area of ​​the laser radar is divided into a plurality of grid areas to serve as the grid areas on the two-dimensional grid map.

7. The laser radar blind spot filling method according to claim 1, 5 or 6, characterized in that: In the case where it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map, determining the real coordinates of the laser point based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area includes: Determine a first distance from a corresponding point to a mirror surface according to the three-dimensional coordinates of the laser point and the mirror surface plane parameters; Extracting a normal vector from the mirror plane parameters, and using the product of a first matrix corresponding to the normal vector and a second distance as a spacing matrix, wherein the second distance is twice the first distance; The sum of the spacing matrix and the second matrix corresponding to the three-dimensional coordinates of the laser point is calculated to determine the real coordinates of the laser point.

8. A laser radar blind spot compensation device, characterized in that: Applied to a laser positioning device, the laser positioning device includes a laser radar and a reflector, the reflector is arranged on the radiation path of the laser radar, and the reflector is used to reflect the laser beam to an area outside the irradiation range of the laser radar. The laser radar blind spot compensation device includes: A point cloud data acquisition module is configured to extract ROI point cloud data acquired by the laser radar in a region of interest, and cluster the ROI point cloud data to determine a plurality of groups of clustered point cloud data; A plane parameter acquisition module is configured to perform plane fitting on the clustered point cloud data, and determine the mirror plane parameters corresponding to the reflector according to the plane parameters obtained after the plane fitting; A rasterization module configured to rasterize the sensing area of ​​the laser radar to determine a two-dimensional grid map corresponding to the sensing area; a parameter association module configured to determine a target grid area corresponding to the reflector on the two-dimensional grid map, and associate the mirror plane parameters with the target grid area; The coordinate output module is configured to determine the real coordinates of the laser point based on the three-dimensional coordinates of the laser point and the mirror plane parameters associated with the target grid area when it is determined that the grid area corresponding to the laser point is the target grid area on the two-dimensional grid map.

9. A laser positioning device, characterized in that: include: one or more processors; A storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the laser radar blind spot filling method as described in any one of claims 1-7.

10. A storage medium storing computer executable instructions, characterized in that: When executed by a processor, the computer executable instructions are used to execute the laser radar blind spot filling method as described in any one of claims 1 to 7.

Images