Expansion point cloud removal method, intelligent device and storage medium

Abstract

The invention relates to the technical field of automatic driving, particularly provides an expansion point cloud removal method, intelligent equipment and a storage medium, and aims to solve the technical problem of abnormal conditions of object recognition and size detection caused by light spot dispersion when a window of an existing laser radar has smudginess or water drops. In order to achieve the purpose, the expansion point cloud removal method comprises the steps of obtaining an original three-dimensional point cloud collected by point cloud collection equipment; acquiring a corresponding two-dimensional feature map based on the original three-dimensional point cloud; obtaining a target object from the two-dimensional feature map, wherein the target object is an object of which the reflection parameter is higher than a first preset threshold value; obtaining target object three-dimensional point cloud data corresponding to the target object; based on the three-dimensional point cloud data of the target object, the expansion point cloud around the target object is removed from the original three-dimensional point cloud, and the reflection parameter of the expansion point cloud is lower than that of the target object. In this way, the low-reflectivity or low-reflection-intensity point cloud close to the target object in the three-dimensional space is filtered, and the detection precision of the object is improved.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving technology, and specifically provides an expansion point cloud removal method, an intelligent device and a storage medium. Background Art

[0002] LiDAR plays an important role in assisted driving and autonomous driving. Compared with passive vision sensors, LiDAR has important advantages such as being less affected by light and being able to provide accurate three-dimensional information about the environment.

[0003] However, for highly reflective objects, such as road signs and traffic cones, when the lidar window is dirty or has water droplets, the light spot is diffused, causing abnormalities in object recognition and size detection. Summary of the invention

[0004] In order to overcome the above defects, the present application is proposed to provide a solution or at least partially solve the technical problem that the spot diffusion when the window of the existing laser radar is dirty or has water droplets causes abnormal object recognition and size detection. The present application provides a method for removing the expanded point cloud, an intelligent device and a storage medium.

[0005] In a first aspect, the present application provides a method for removing an expanded point cloud, the method comprising:

[0006] Obtaining the original three-dimensional point cloud collected by the point cloud collection device;

[0007] Acquire a corresponding two-dimensional feature map based on the original three-dimensional point cloud;

[0008] Acquire a target object from the two-dimensional feature map, the target object being an object having a reflection parameter higher than a first preset threshold;

[0009] Acquire three-dimensional point cloud data of the target object corresponding to the target object;

[0010] Based on the three-dimensional point cloud data of the target object, an expanded point cloud around the target object is removed from the original three-dimensional point cloud, wherein a reflection parameter of the expanded point cloud is lower than a reflection parameter of the target object.

[0011] In one embodiment of the present application, obtaining a corresponding two-dimensional feature map based on the three-dimensional point cloud includes:

[0012] Determine the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning method of the point cloud acquisition device;

[0013] The two-dimensional feature map is drawn based on the horizontal axis, the vertical axis, and the scanning parameters and reflection parameters of each point cloud.

[0014] In one embodiment of the present application, the scanning mode of the point cloud acquisition device includes a one-dimensional scanning mode and a two-dimensional scanning mode;

[0015] The determining of the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning mode of the point cloud acquisition device includes: when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, determining that the horizontal axis of the two-dimensional feature map is the horizontal azimuth of the scan, and the vertical axis of the two-dimensional feature map is the vertical azimuth of the scan;

[0016] When the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the scanning line number, and the vertical axis of the two-dimensional feature map is the scanning point number on the scanning line, wherein different combinations of scanning line numbers and scanning point numbers correspond to different point clouds in the original three-dimensional point cloud.

[0017] In one embodiment of the present application, the removing of the expansion point cloud around the target object in the original three-dimensional point cloud based on the three-dimensional point cloud data of the target object includes:

[0018] Acquire an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object;

[0019] determining an expansion point cloud in the expansion point cloud elimination area;

[0020] The expanded point cloud is removed from the expanded point cloud removal area.

[0021] In one embodiment of the present application, the step of obtaining an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object includes:

[0022] Acquire the spatial size of the target object based on the three-dimensional point cloud data of the target object;

[0023] Determining the spatial size of the expanded point cloud elimination area based on the spatial size of the target object and a preset proportional coefficient, wherein the preset proportional coefficient is set based on the distance between the target object and the point cloud acquisition device and is proportional to the distance;

[0024] The center position corresponding to the target object in the original three-dimensional point cloud is used as the center position of the expanded point cloud elimination area, and the expanded point cloud elimination area is obtained according to the spatial size of the expanded point cloud elimination area.

[0025] In one embodiment of the present application, the reflection parameter includes reflectivity and / or reflection intensity, and the first preset threshold includes a first reflectivity threshold and / or a first reflection intensity threshold;

[0026] The determining of the expanded point cloud in the expanded point cloud elimination area includes: taking a point cloud in the expanded point cloud elimination area whose reflectivity is lower than a second reflectivity threshold and / or whose reflection intensity is lower than a second reflection intensity threshold as the expanded point cloud, wherein the first reflectivity threshold is greater than the second reflectivity threshold, and the first reflection intensity threshold is greater than the second reflection intensity threshold.

[0027] In one embodiment of the present application, acquiring the target object from the two-dimensional feature map includes:

[0028] Binarize the two-dimensional feature map based on the reflection parameter to obtain a binary map;

[0029] A connected area of ​​pixels of a preset color in the binary image is obtained, and the target object is obtained based on the connected area.

[0030] In one embodiment of the present application, before obtaining the connected region of pixels of preset color in the binary image, the method further includes: optimizing the binary image using a mathematical morphology method.

[0031] In one embodiment of the present application, the method further includes:

[0032] Deleting the connected regions of the preset color pixels, the number of points of which is less than a third preset threshold; and / or

[0033] Slide a preset window in each of the connected areas to calculate the mean and variance of the depth information in each preset window;

[0034] In a preset window corresponding to a variance smaller than a preset variance threshold, points whose depth information has a difference with the mean value greater than a fourth preset threshold are deleted.

[0035] In a second aspect, a smart device is provided, comprising:

[0036] at least one processor;

[0037] and, a memory communicatively coupled to the at least one processor;

[0038] The memory stores a computer program, and when the computer program is executed by the at least one processor, the aforementioned expansion point cloud removal method is implemented.

[0039] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute any of the aforementioned methods for removing an expanded point cloud.

[0040] Solution 1. A method for removing an expanded point cloud, characterized in that the method comprises:

[0041] Obtaining the original three-dimensional point cloud collected by the point cloud collection device;

[0042] Acquire a corresponding two-dimensional feature map based on the original three-dimensional point cloud;

[0043] Acquire a target object from the two-dimensional feature map, the target object being an object having a reflection parameter higher than a first preset threshold;

[0044] Acquire three-dimensional point cloud data of the target object corresponding to the target object;

[0045] Based on the three-dimensional point cloud data of the target object, an expanded point cloud around the target object is removed from the original three-dimensional point cloud, wherein a reflection parameter of the expanded point cloud is lower than a reflection parameter of the target object.

[0046] Solution 2. The method for removing the expanded point cloud according to Solution 1 is characterized in that the corresponding two-dimensional feature map is obtained based on the three-dimensional point cloud, comprising:

[0047] Determine the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning method of the point cloud acquisition device;

[0048] The two-dimensional feature map is drawn based on the horizontal axis, the vertical axis, and the scanning parameters and reflection parameters of each point cloud.

[0049] Solution 3. The method for removing the expanded point cloud according to Solution 1 is characterized in that the scanning mode of the point cloud acquisition device includes a one-dimensional scanning mode and a two-dimensional scanning mode;

[0050] The determining of the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning mode of the point cloud acquisition device includes: when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, determining that the horizontal axis of the two-dimensional feature map is the horizontal azimuth of the scan, and the vertical axis of the two-dimensional feature map is the vertical azimuth of the scan;

[0051] When the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the scanning line number, and the vertical axis of the two-dimensional feature map is the scanning point number on the scanning line, wherein different combinations of scanning line numbers and scanning point numbers correspond to different point clouds in the original three-dimensional point cloud.

[0052] Solution 4. The method for removing the expanded point cloud according to Solution 1 is characterized in that, based on the three-dimensional point cloud data of the target object, removing the expanded point cloud around the target object in the original three-dimensional point cloud comprises:

[0053] Acquire an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object;

[0054] determining an expansion point cloud in the expansion point cloud elimination area;

[0055] The expanded point cloud is removed from the expanded point cloud removal area.

[0056] Solution 5. The method for removing an expanded point cloud according to Solution 4 is characterized in that the step of obtaining an expanded point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object comprises:

[0057] Acquire the spatial size of the target object based on the three-dimensional point cloud data of the target object;

[0058] Determining the spatial size of the expanded point cloud elimination area based on the spatial size of the target object and a preset proportional coefficient, wherein the preset proportional coefficient is set based on the distance between the target object and the point cloud acquisition device and is proportional to the distance;

[0059] The center position corresponding to the target object in the original three-dimensional point cloud is used as the center position of the expanded point cloud elimination area, and the expanded point cloud elimination area is obtained according to the spatial size of the expanded point cloud elimination area.

[0060] Solution 6. The method for removing an expanded point cloud according to Solution 4, characterized in that the reflection parameter includes reflectivity and / or reflection intensity, and the first preset threshold includes a first reflectivity threshold and / or a first reflection intensity threshold;

[0061] The determining of the expanded point cloud in the expanded point cloud elimination area includes: taking a point cloud in the expanded point cloud elimination area whose reflectivity is lower than a second reflectivity threshold and / or whose reflection intensity is lower than a second reflection intensity threshold as the expanded point cloud, wherein the first reflectivity threshold is greater than the second reflectivity threshold, and the first reflection intensity threshold is greater than the second reflection intensity threshold.

[0062] Solution 7. The method for removing the expanded point cloud according to Solution 1 is characterized in that the step of obtaining the target object from the two-dimensional feature map comprises:

[0063] Binarize the two-dimensional feature map based on the reflection parameter to obtain a binary map;

[0064] A connected area of ​​pixels of a preset color in the binary image is obtained, and the target object is obtained based on the connected area.

[0065] Solution 8. The method for removing the expanded point cloud according to Solution 7 is characterized in that before obtaining the connected area of ​​the preset color pixels in the binary image, the method also includes: optimizing the binary image using a mathematical morphology method.

[0066] Solution 9. The method for removing the expanded point cloud according to Solution 7, characterized in that the method further comprises:

[0067] Deleting the connected regions of the preset color pixels, the number of points of which is less than a third preset threshold; and / or

[0068] Slide a preset window in each of the connected areas to calculate the mean and variance of the depth information in each preset window;

[0069] In a preset window corresponding to a variance smaller than a preset variance threshold, points whose depth information has a difference with the mean value greater than a fourth preset threshold are deleted.

[0070] Solution 10. A smart device, comprising:

[0071] at least one processor;

[0072] and, a memory communicatively coupled to the at least one processor;

[0073] The memory stores a computer program, and when the computer program is executed by the at least one processor, the method for removing the expanded point cloud described in any one of Schemes 1 to 9 is implemented.

[0074] Solution 11. A computer-readable storage medium storing a plurality of program codes, wherein the program codes are suitable for being loaded and run by a processor to execute the method for removing an expanded point cloud described in any one of Solutions 1 to 9.

[0075] The above one or more technical solutions of this application have at least one or more of the following Beneficial effects:

[0076] The expansion point cloud removal method in the present application specifically includes: obtaining the original point cloud information collected by the point cloud acquisition device; obtaining a two-dimensional feature map based on the original point cloud information; obtaining a target object from the two-dimensional feature map, the target object being a road object with a reflectivity higher than a first preset threshold; filtering out the expansion point cloud around the target object. In this way, the high-reflectivity target object is identified based on the two-dimensional feature map, and the point cloud with low reflectivity or low reflection intensity adjacent to the target object in three-dimensional space is filtered out, avoiding the misdetection of the size of the high-reflectivity target object or misidentification of a false target, thereby improving the detection accuracy of the object. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:

[0078] Figure 1 This is a schematic diagram of the main process of the expansion point cloud removal method in one embodiment of the present application;

[0079] Figure 2 is a schematic diagram of an expanded point cloud in one embodiment of the present application;

[0080] Figure 3 It is a schematic diagram of converting an original three-dimensional point cloud into a two-dimensional feature map in one embodiment of the present application;

[0081] Figure 4 It is a schematic diagram of a two-dimensional characteristic map of a laser radar composed of a polygonal scanning mirror and a galvanometer mirror in one embodiment of the present application;

[0082] Figure 5 is a schematic diagram of obtaining a target object from a two-dimensional feature map in one embodiment of the present application;

[0083] Figure 6 is a schematic diagram of a target object obtained in one embodiment of the present application;

[0084] Figure 7 This is a schematic diagram of the effect of the expansion point cloud filtering method in one embodiment of the present application;

[0085] Figure 8 It is a schematic diagram of the main structure of an expansion point cloud removal device in one embodiment of the present application;

[0086] Fig. 9 It is a schematic diagram of the structure of a smart device in one embodiment of the present application. DETAILED DESCRIPTION

[0087] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0088] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0089] At present, for high reflectivity objects, such as road signs and traffic cones, when the laser radar window is dirty or has water droplets, the light spot is diffused, which will cause abnormal object recognition and size detection. To this end, the present application proposes a method for removing expansion point clouds, an intelligent device and a storage medium.

[0090] See attached Figure 1 , Figure 1 It is a flowchart of the main steps of a method for removing an expanded point cloud according to an embodiment of the present application.

[0091] like Figure 1 As shown, the expansion point cloud removal method in the embodiment of the present application mainly includes the following steps S10 to S50.

[0092] Step S10: Acquire the original three-dimensional point cloud collected by the point cloud collection device.

[0093] Point cloud acquisition equipment refers to a device for collecting point cloud information in three-dimensional space. For example, a laser radar can be used as an example of the point cloud acquisition equipment.

[0094] The original 3D point cloud refers to the unprocessed 3D point cloud data directly obtained from the point cloud acquisition device. The original 3D point cloud may include data such as spatial coordinates (x, y, z), reflectivity, reflection intensity, and scanning parameters, where the scanning parameters correspond to the scanning mode of the point cloud acquisition device. For example, when the scanning mode is a one-dimensional scanning mode, the scanning parameters include the horizontal azimuth and vertical azimuth of the scan; when the scanning mode is a two-dimensional scanning mode, the scanning parameters include the scanning line number and the scanning point number.

[0095] Step S20: Obtain a corresponding two-dimensional feature map based on the original three-dimensional point cloud.

[0096] The two-dimensional feature map is a two-dimensional feature representation map obtained based on the original three-dimensional point cloud. It can retain important feature information of the point cloud, such as reflectivity or reflection intensity.

[0097] Step S30: Acquire a target object from the two-dimensional feature map, where the target object is an object having a reflection parameter higher than a first preset threshold.

[0098] The target object refers to an object with specific properties (such as high reflectivity parameters), which can be distinguished and extracted by setting a threshold. For example, road signs, traffic cones, etc. on the road can be used as examples of the target object.

[0099] Reflection parameters refer to the measure of the degree of reflection of electromagnetic waves by a point cloud. Reflection parameters include reflectivity and reflection intensity.

[0100] The first preset threshold value may be a preset standard value, which may be adaptively adjusted according to the actual scene. The first preset threshold value is used to select points with sufficiently high reflection parameters as part of the target object.

[0101] Step S40: Acquire the target object's three-dimensional point cloud data corresponding to the target object.

[0102] The target object three-dimensional point cloud data refers to the three-dimensional point cloud data corresponding to the target object, which includes the specific shape and position information of the target object.

[0103] Step S50: Based on the three-dimensional point cloud data of the target object, the expanded point cloud around the target object is removed from the original three-dimensional point cloud, wherein the reflection parameter of the expanded point cloud is lower than the reflection parameter of the target object.

[0104] The expansion point cloud refers to the abnormal or inaccurate point cloud data around the target object caused by the diffusion of light spots when the laser radar window is dirty or water droplets. For example, Figure 2 The purple point cloud in the figure is the real point cloud of the road sign, and the yellow point cloud and red point cloud are both expanded point clouds.

[0105] Based on the above steps S10 to S50, first obtain the original three-dimensional point cloud collected by the point cloud acquisition device; obtain the corresponding two-dimensional feature map based on the original three-dimensional point cloud; obtain the target object from the two-dimensional feature map, the target object is an object with a reflection parameter higher than the first preset threshold; obtain the target object three-dimensional point cloud data corresponding to the target object; based on the target object three-dimensional point cloud data, remove the expansion point cloud around the target object in the original three-dimensional point cloud, the reflection parameter of the expansion point cloud is lower than the reflection parameter of the target object. In this way, the recognition of high-reflection target objects is performed on the basis of the two-dimensional feature map, and the point clouds with low reflectivity or low reflection intensity that are close to the target object in three-dimensional space are filtered out, so as to avoid the misdetection of the size of the high-reflectivity target object or the misidentification of the false target, thereby improving the detection accuracy of the object.

[0106] The above steps S20 to S50 are further explained below.

[0107] The above step S20 can be implemented through the following steps S201 to S202.

[0108] Step S201: Determine the horizontal axis and vertical axis of the two-dimensional feature map according to the scanning method of the point cloud acquisition device.

[0109] Point cloud acquisition equipment usually collects three-dimensional information in the environment according to a certain scanning method. Different scanning methods will result in different distribution characteristics of the acquired data.

[0110] In a specific embodiment of the present application, the scanning mode of the point cloud acquisition device includes a one-dimensional scanning mode and a two-dimensional scanning mode; the determining of the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning mode of the point cloud acquisition device includes: when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, determining that the horizontal axis of the two-dimensional feature map is the horizontal azimuth of the scan, and the vertical axis of the two-dimensional feature map is the vertical azimuth of the scan; when the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the scan line number, and the vertical axis of the two-dimensional feature map is the scan point number on the scan line, wherein different combinations of scan line numbers and scan point numbers correspond to different point clouds in the original three-dimensional point cloud.

[0111] Specifically, when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, the specific scanning position can be determined by the horizontal azimuth and the vertical azimuth. When the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the specific scanning position can be determined by the line number and the point number of the scanning line, wherein different combinations of the line number of the scanning line and the scanning point number correspond to different point clouds in the original three-dimensional point cloud. Therefore, when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the horizontal azimuth of the scan, and the vertical axis is the vertical azimuth of the scan. When the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the scanning line number, and the vertical axis is the scanning point number on the scanning line.

[0112] For example, for a mechanical one-dimensional rotating scanning laser radar, the horizontal azimuth and the vertical azimuth can be used as the horizontal axis and the vertical axis of the two-dimensional feature map, respectively. For a two-dimensional scanning laser radar, such as a dual polygon scanning mirror laser radar, a laser radar with a polygon scanning mirror and a galvanometer, a dual galvanometer laser radar, a micro-electromechanical system galvanometer laser radar, etc., the scanning line number and the scanning point number are selected as the horizontal axis and the vertical axis of the two-dimensional feature map, respectively.

[0113] Step S202: Draw a two-dimensional feature map based on the horizontal axis, the vertical axis, and the scanning parameters and reflection parameters of each point cloud.

[0114] Scanning parameters refer to the specific information recorded by the point cloud acquisition device when acquiring data in different scanning modes. This information is used to describe the position of each point in three-dimensional space. For example, the horizontal azimuth and vertical azimuth corresponding to the one-dimensional scanning mode; the line number and point number corresponding to the two-dimensional scanning mode, etc.

[0115] Specifically, the position of each point in the two-dimensional feature map is determined by its corresponding horizontal and vertical axis values. The color or brightness of each point can be set according to the reflection parameters (such as reflectivity or reflection intensity) to intuitively display the characteristics of different object surfaces. For example, a strong reflection area can be represented by a bright color, while a weak reflection area can be represented by a dark color.

[0116] For example, Figure 3 It can be used as an example of converting the original 3D point cloud into a 2D feature map. The original 3D point cloud may include information such as x-coordinate, y-coordinate, z-coordinate, reflectivity, reflection intensity, scan line number, and scan point number. The 2D feature map represents the original 3D point cloud with scan line number and scan point number, but each pixel in the 2D feature map also carries the corresponding x-coordinate, y-coordinate, z-coordinate, reflectivity, reflection intensity, and other information.

[0117] For example, Figure 4This is an example of a two-dimensional feature map of a laser radar that is a combination of a polygonal scanning mirror and a galvanometer, where the horizontal axis of the two-dimensional feature map is the scanning line number and the vertical axis is the scanning point number. The yellow area in the figure represents a strong reflection area, and other color areas represent non-strong reflection areas. The yellow area may correspond to a high-reflectivity object or a high-reflection intensity object in the scene, and other color areas may correspond to a low-reflectivity object or a low-reflection intensity object in the scene.

[0118] The above is a further description of step S20 , and the following is a further description of step S30 .

[0119] The above step S30 can be implemented through the following steps S301 and S302.

[0120] Step S301: binarizing the two-dimensional feature map based on the reflection parameter to obtain a binarized map.

[0121] Specifically, first, a preset threshold corresponding to the reflection parameter can be set. Exemplarily, when the reflection parameter is reflectivity, the preset threshold can be a reflectivity threshold; when the reflection parameter is reflection intensity, the preset threshold can be a first reflection intensity threshold. Then, a binarization technique is used to assign a value of 1 to pixels in the two-dimensional feature map that are higher than the first preset threshold as a high reflection area, and a value of 0 to pixels in the two-dimensional feature map that are lower than the first preset threshold as a low reflection area. After the two-dimensional feature map is binarized, pixels assigned a value of 1 are displayed as white, and pixels assigned a value of 0 are displayed as black.

[0122] Step S302: obtaining a connected region of pixels of a preset color in the binary image, and obtaining a target object based on the connected region.

[0123] The preset color pixel may be a pixel of a preset color.

[0124] A connected region refers to a region composed of connected pixels in a binary image. The connected region includes four-connected and eight-connected regions. For example, the four-connected region can be directly used to connect the white blocks that have a connection relationship with each other to obtain a connected region, thereby obtaining the initial target object.

[0125] In a specific implementation of the present application, before obtaining the preset color pixel connected area in the binary image, the method further includes: optimizing the binary image using a mathematical morphology method.

[0126] Mathematical morphology methods include closing operations and opening operations, where the closing operation includes a dilation operation and an erosion operation performed in sequence, and the opening operation includes an erosion operation and a dilation operation performed in sequence. Dilation is an operation that expands the foreground object (usually white pixels) in an image by increasing the size of the foreground object to fill small holes inside it or connect adjacent objects. Erosion is an operation that shrinks the foreground object in an image by reducing the size of the foreground object to eliminate small protrusions or separate closely contacting objects.

[0127] Specifically, the binary image can be closed first and then opened. The closing operation means filling the small holes or gaps in the image first and then smoothing the edges; the opening operation is the opposite, smoothing the edges first and then filling the small holes or gaps in the image. The ultimate goal is to eliminate noise and smooth small holes in the binary image. Closing first and then opening will better restore the outer lines of the white area and reduce the loss of the original shape of the image due to smoothing.

[0128] In a specific embodiment of the present application, the method further includes: deleting the connected area of ​​the preset color pixel whose number of points is less than a third preset threshold; and / or

[0129] Slide a preset window in each of the connected areas to calculate the mean and variance of the depth information in each preset window; in the preset windows corresponding to the variances less than the preset threshold, delete the points where the difference between the depth information and the mean is greater than a fourth preset threshold.

[0130] The third preset threshold may be a pre-set value, which may be adaptively adjusted according to actual scenarios, and is not specifically limited thereto.

[0131] Specifically, the size of each connected area found can be filtered, specifically, the connected areas containing pixels greater than or equal to the third preset threshold are retained, and the connected areas containing pixels less than the third preset threshold are deleted. In this way, small-scale false positive connected areas caused by noise or other reasons are excluded.

[0132] The preset window may be a preset sliding window, and the preset window may include a preset number of adjacent pixels.

[0133] Specifically, in each connected area, a sliding window is used to move one by one to cover the entire area. For each pixel in the sliding window, the variance and mean of the vertical coordinate of each pixel are calculated. In the window where the variance is less than the preset threshold, the pixel points with a vertical coordinate less than the mean are deleted, and the points in the window have good flatness. The vertical coordinate of each pixel is the Z coordinate in the original three-dimensional point cloud. In this way, those areas with small changes in Z-axis coordinates and high flatness can be effectively identified, thereby reducing the possibility of misjudgment.

[0134] For example, Figure 5 This can be used as an example to obtain the target object from the two-dimensional feature map. Figure 5 As shown, the step of acquiring the target object from the two-dimensional feature map can be implemented through the following steps S100-S600.

[0135] Step S100: binarizing the two-dimensional feature map to obtain a binarized map.

[0136] Step S200: performing dilation and erosion operations on the binary image in sequence.

[0137] Step S300: performing an erosion operation and a dilation operation on the binary image in sequence.

[0138] Step S400: obtaining connected areas in the binary graph by using a connected component search method.

[0139] Step S500: retaining the connected regions whose number of pixels is greater than or equal to a preset threshold value, and deleting the connected regions whose number of pixels is less than the preset threshold value.

[0140] Step S600: Slide a preset window in each retained connected area to calculate the mean and variance of the depth information in each preset window. In the window where the variance is less than the preset threshold, the pixel points whose vertical coordinates are less than the mean are deleted. At this time, the points in the window have good flatness, so as to obtain a high-reflectivity object with high confidence.

[0141] For example, Figure 6 The method described in the above steps S100-S600 can be used to Figure 4 An example of a target object obtained from a two-dimensional feature map. Figure 6 The horizontal axis is the scan line number, and the vertical axis is the scan point number. Figure 6 The background is black, and the white area is the connected area, which corresponds to the high reflectivity objects or high reflection intensity objects in the scene.

[0142] The above is a further description of step S30 , and the following is a further description of step S40 .

[0143] After obtaining the target object from the two-dimensional feature map, the three-dimensional point cloud data of the target object corresponding to the target object can be further obtained from the original three-dimensional point cloud.

[0144] The above is a further description of step S40 , and the following is a further description of step S50 .

[0145] The above step S40 can be implemented through the following steps S401 to S403.

[0146] Step S401: obtaining an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object.

[0147] The expansion point cloud elimination area refers to a specific spatial range defined to remove the expansion point cloud around the target object. The area is set around the target object and contains possible expansion point clouds.

[0148] The above step S401 can be implemented through the following steps S4011 to S4013.

[0149] Step S4011: Acquire the spatial size of the target object based on the three-dimensional point cloud data of the target object.

[0150] Specifically, the spatial size of the target object is obtained according to the maximum and minimum values ​​of the x-coordinate, y-coordinate, and z-coordinate of the three-dimensional point cloud data of the target object.

[0151] Step S4012: Determine the spatial size of the expanded point cloud elimination area based on the spatial size of the target object and a preset proportional coefficient. The preset proportional coefficient is set based on the distance between the target object and the point cloud acquisition device and is proportional to the distance.

[0152] The preset proportionality coefficient may be a pre-set coefficient. Since the laser radar beam has a divergent characteristic, in order to improve robustness, the distance factor may be considered when expanding the point cloud to eliminate the area, that is, the preset proportionality coefficient may be proportional to the distance of the laser radar. Exemplarily, 1.2, 2, 2.5, 3, etc. may all be used as examples of the preset proportionality coefficient.

[0153] Specifically, with respect to the spatial size of the target object, the products of the length, width and height of the target object and the preset proportional coefficient are respectively used as the length, width and height of the expanded point cloud elimination area, thereby obtaining the spatial size of the expanded point cloud elimination area.

[0154] Step S4013: Taking the center position corresponding to the target object in the original three-dimensional point cloud as the center position of the expanded point cloud elimination area, the expanded point cloud elimination area is obtained according to the spatial size of the expanded point cloud elimination area.

[0155] Specifically, after obtaining the spatial size of the expanded point cloud elimination area, the center position corresponding to the target object in the original three-dimensional point cloud is used as the center position of the expanded point cloud elimination area, and the expanded point cloud elimination area can be obtained according to the spatial size of the expanded point cloud elimination area.

[0156] Step S402: determining the expansion point cloud in the expansion point cloud elimination area.

[0157] In a specific embodiment of the present application, the reflection parameter includes reflectivity and / or reflection intensity, and the first preset threshold includes a first reflectivity threshold and / or a first reflection intensity threshold; the determining of the expanded point cloud in the expanded point cloud elimination area includes: taking the point cloud in the expanded point cloud elimination area whose reflectivity is lower than a second reflectivity threshold, and / or whose reflection intensity is lower than a second reflection intensity threshold as the expanded point cloud, wherein the first reflectivity threshold is greater than the second reflectivity threshold, and the first reflection intensity threshold is greater than the second reflection intensity threshold.

[0158] Specifically, in the expanded point cloud elimination area, point clouds whose reflectivity is lower than a second reflectivity threshold and / or whose reflection intensity is lower than a second reflection intensity threshold in the expanded point cloud elimination area are taken as expanded point clouds.

[0159] Step S403: removing the expanded point cloud from the expanded point cloud elimination area.

[0160] By accurately identifying and removing the expanded point cloud, the expanded point cloud can be effectively removed, making the point cloud data more accurate and reliable, which is beneficial to enhancing the recognition accuracy of the target object.

[0161] like Figure 7 As shown in the figure, the red point cloud is the real point cloud of the highly reflective object, the white point cloud and the green point cloud are the expanded point clouds filtered out by the method of the present application, the yellow point cloud is the point cloud that is mistakenly deleted when the expanded point cloud is removed by the existing method, the blue point cloud is the point cloud corresponding to the road surface or the curb, and the red box is the detection box. Figure 7 It can be seen that compared with the existing method, the present application better retains the real point cloud and effectively eliminates the expanded point cloud, which is beneficial to improving the detection accuracy of the target object.

[0162] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present application.

[0163] Furthermore, the present application also provides an expansion point cloud removal device.

[0164] See attached Figure 8 , Figure 8 It is a main structural block diagram of an expansion point cloud removal device according to an embodiment of the present application.

[0165] like Figure 8As shown, the expansion point cloud removal device in the embodiment of the present application mainly includes a first acquisition module 11, a second acquisition module 12, a third acquisition module 13, a fourth acquisition module and a removal module 15. In some embodiments, one or more of the first acquisition module 11, the second acquisition module 12, the third acquisition module 13, the fourth acquisition module and the removal module 15 can be combined into one module.

[0166] In some embodiments, the first acquisition module 11 may be configured to acquire an original three-dimensional point cloud acquired by a point cloud acquisition device.

[0167] The second acquisition module 12 may be configured to acquire a corresponding two-dimensional feature map based on the original three-dimensional point cloud.

[0168] The third acquisition module 13 may be configured to acquire a target object from the two-dimensional feature map, where the target object is an object having a reflection parameter higher than a first preset threshold.

[0169] The fourth acquisition module 14 may be configured to acquire target object three-dimensional point cloud data corresponding to the target object.

[0170] The removal module 15 may be configured to remove the expanded point cloud around the target object in the original three-dimensional point cloud based on the target object three-dimensional point cloud data, wherein the reflection parameter of the expanded point cloud is lower than the reflection parameter of the target object.

[0171] In one implementation, the description of the specific implementation functions can refer to steps S10-S50.

[0172] The above-mentioned expansion point cloud removal device is used to perform Figure 1 The embodiment of the expanded point cloud removal method shown in the figure has similar technical principles, technical problems solved and technical effects produced. Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the expanded point cloud removal device can refer to the contents described in the embodiment of the expanded point cloud removal method, which will not be repeated here.

[0173] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0174] It is understood by those skilled in the art that each module in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present application.

[0175] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0176] Furthermore, the present application also provides an intelligent device, which may include at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program, and when the computer program is executed by the at least one processor, the method for removing the expanded point cloud described in any of the above embodiments is implemented. Fig. 9 As shown, Fig. 9 exemplarily shows the structure of a smart device, which includes a processor 100 and a memory 200.

[0177] In some embodiments of the present application, the smart device further includes at least one sensor, which is used to sense information. The sensor is communicatively connected to any type of processor mentioned in the present application. Optionally, the smart device further includes an autonomous driving system, which is used to guide the smart device to drive itself or assist in driving. The processor communicates with the sensor and / or the autonomous driving system to complete the expansion point cloud removal method described in any of the above embodiments.

[0178] Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium can be configured to store a program for executing the expansion point cloud removal method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned expansion point cloud removal method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium can be a memory device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0179] The relevant user personal information that may be involved in the various embodiments of this application is strictly in accordance with the requirements of laws and regulations, following the principles of legality, legitimacy and necessity, based on the reasonable purposes of business scenarios, to process the personal information that users actively provide during the use of products / services or generated due to the use of products / services, as well as the personal information obtained with the user's authorization.

[0180] The user personal information processed by this application will vary depending on the specific product / service scenario, and will be based on the specific scenario in which the user uses the product / service. It may involve the user's account information, device information, driving information, vehicle information or other related information. The applicant will treat the user's personal information and its processing with a high degree of diligence.

[0181] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.

[0182] So far, the technical solutions of the present application have been described in conjunction with the specific implementations shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific implementations. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. A method for removing an expanded point cloud, characterized in that: The method comprises: Obtaining the original three-dimensional point cloud collected by the point cloud collection device; Acquire a corresponding two-dimensional feature map based on the original three-dimensional point cloud; Acquire a target object from the two-dimensional feature map, the target object being an object having a reflection parameter higher than a first preset threshold; Acquire three-dimensional point cloud data of the target object corresponding to the target object; Based on the three-dimensional point cloud data of the target object, an expanded point cloud around the target object is removed from the original three-dimensional point cloud, wherein a reflection parameter of the expanded point cloud is lower than a reflection parameter of the target object.

2. The method for removing the expanded point cloud according to claim 1, characterized in that: Acquiring a corresponding two-dimensional feature map based on the three-dimensional point cloud includes: Determine the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning method of the point cloud acquisition device; The two-dimensional feature map is drawn based on the horizontal axis, the vertical axis, and the scanning parameters and reflection parameters of each point cloud.

3. The method for removing the expanded point cloud according to claim 1, characterized in that: The scanning mode of the point cloud acquisition device includes a one-dimensional scanning mode and a two-dimensional scanning mode; The determining of the horizontal axis and the vertical axis of the two-dimensional feature map according to the scanning mode of the point cloud acquisition device includes: when the scanning mode of the point cloud acquisition device is a one-dimensional scanning mode, determining that the horizontal axis of the two-dimensional feature map is the horizontal azimuth of the scan, and the vertical axis of the two-dimensional feature map is the vertical azimuth of the scan; When the scanning mode of the point cloud acquisition device is a two-dimensional scanning mode, the horizontal axis of the two-dimensional feature map is the scanning line number, and the vertical axis of the two-dimensional feature map is the scanning point number on the scanning line, wherein different combinations of scanning line numbers and scanning point numbers correspond to different point clouds in the original three-dimensional point cloud.

4. The method for removing the expanded point cloud according to claim 1, characterized in that: The removing the expanded point cloud around the target object in the original three-dimensional point cloud based on the three-dimensional point cloud data of the target object comprises: Acquire an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object; determining an expansion point cloud in the expansion point cloud elimination area; The expanded point cloud is removed from the expanded point cloud removal area.

5. The method for removing the expanded point cloud according to claim 4, characterized in that: The step of obtaining an expansion point cloud elimination area in the three-dimensional original point cloud based on the three-dimensional point cloud data of the target object includes: Acquire the spatial size of the target object based on the three-dimensional point cloud data of the target object; Determining the spatial size of the expanded point cloud elimination area based on the spatial size of the target object and a preset proportional coefficient, wherein the preset proportional coefficient is set based on the distance between the target object and the point cloud acquisition device and is proportional to the distance; The center position corresponding to the target object in the original three-dimensional point cloud is used as the center position of the expanded point cloud elimination area, and the expanded point cloud elimination area is obtained according to the spatial size of the expanded point cloud elimination area.

6. The method for removing the expanded point cloud according to claim 4, characterized in that: The reflection parameter includes reflectivity and / or reflection intensity, and the first preset threshold includes a first reflectivity threshold and / or a first reflection intensity threshold; The determining of the expanded point cloud in the expanded point cloud elimination area includes: taking a point cloud in the expanded point cloud elimination area whose reflectivity is lower than a second reflectivity threshold and / or whose reflection intensity is lower than a second reflection intensity threshold as the expanded point cloud, wherein the first reflectivity threshold is greater than the second reflectivity threshold, and the first reflection intensity threshold is greater than the second reflection intensity threshold.

7. The method for removing the expanded point cloud according to claim 1, characterized in that: The acquiring the target object from the two-dimensional feature map comprises: Binarize the two-dimensional feature map based on the reflection parameter to obtain a binary map; A connected area of ​​pixels of a preset color in the binary image is obtained, and the target object is obtained based on the connected area.

8. The method for removing the expanded point cloud according to claim 7, characterized in that: Before obtaining the connected area of ​​the preset color pixels in the binary image, the method further includes: optimizing the binary image by using a mathematical morphology method.

9. The method for removing the expanded point cloud according to claim 7, characterized in that: The method further comprises: Deleting the connected regions of the preset color pixels, the number of points of which is less than a third preset threshold; and / or Slide a preset window in each of the connected areas to calculate the mean and variance of the depth information in each preset window; In a preset window corresponding to a variance smaller than a preset variance threshold, points whose depth information has a difference with the mean value greater than a fourth preset threshold are deleted.

10. A smart device, characterized in that: include: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the method for removing the expanded point cloud according to any one of claims 1 to 9 is implemented.

Images