Point cloud processing method, electronic equipment, storage medium and vehicle

Abstract

The invention discloses a point cloud processing method, electronic equipment, a storage medium and a vehicle, relates to the technical field of signal processing, and aims to solve the problem of poor quality of point cloud data caused by false point cloud data in the point cloud data of a laser radar in related technologies. The point cloud processing method comprises the following steps: acquiring original point cloud data, wherein the original point cloud data comprises information of a plurality of points; determining an object point and a ground point from the plurality of points based on the information of the plurality of points; screening out information of invalid points from the original point cloud data to obtain processed point cloud data; the invalid points are other points except the object point and the ground point in the plurality of points.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, and in particular to a point cloud processing method, electronic device, storage medium, and vehicle. Background Technology

[0002] With the continuous development of technology, lidar has been widely used in the field of detection due to its advantages of high resolution, good concealment, and strong anti-interference capabilities, particularly in intelligent robots and autonomous vehicles. However, during the detection of real objects, the lidar's signal transmission and reception are affected by ambient light, spatial debris, and other factors. This results in spurious point cloud data, or noise, appearing in the lidar's output point cloud data, leading to poor data quality and inaccurate processing results. Therefore, improving the quality of point cloud data is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] The purpose of this application is to provide a point cloud processing method, electronic device, storage medium, and vehicle, aiming to solve the problem in the related art that there are false point cloud data in the point cloud data of lidar, resulting in poor quality of point cloud data.

[0004] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0005] In a first aspect, embodiments of this application provide a point cloud processing method, the method comprising:

[0006] Acquire raw point cloud data, which includes information about multiple points;

[0007] Based on information from multiple points, the object point and the ground point are determined from the multiple points;

[0008] The invalid point cloud data is obtained by filtering out invalid points from the original point cloud data; invalid points are points other than object points and ground points.

[0009] The point cloud processing method provided in this application determines the object points and ground points of the real object from the original point cloud data based on the information of multiple points in the original point cloud data, and filters out invalid points other than object points and ground points in the original point cloud data, thereby avoiding the existence of false point cloud data in the processed point cloud data and improving the quality of point cloud data.

[0010] In some embodiments, determining an object point and a ground point from multiple points based on information from multiple points includes: determining an object point from multiple points based on information from multiple points; and determining a ground point from non-object points based on information from multiple points, wherein non-object points are points other than the object point among the multiple points.

[0011] In some embodiments, determining an object point from multiple points based on information from multiple points includes: determining a target point that satisfies the local consistency condition as an object point based on the location information of the target point and the location information of the target point's neighboring points, wherein the target point is any one of the multiple points.

[0012] In some embodiments, the local consistency condition includes: the distance between any one or more of the object point's top, bottom, left, and right neighboring points and the object point is less than or equal to a first preset distance threshold.

[0013] In some embodiments, the local consistency condition includes: the number of valid points in the neighborhood of the object point with a preset size is greater than or equal to a preset number; wherein, a valid point is a neighborhood point in the neighborhood of the object point with a preset size whose distance from the object point is less than a second preset distance threshold.

[0014] In some embodiments, determining ground points from non-object points based on information from multiple points includes: determining non-object points that satisfy the ground coplanarity condition and the distance consistency condition as ground points based on the location information of the non-object points and the location information of the neighboring points of the non-object points.

[0015] In some embodiments, the ground coplanarity condition includes at least one of the following: ground line distance condition, tilt angle condition, and slope angle condition.

[0016] In some embodiments, the ground line distance condition includes: the distances between neighboring points in the preset longitudinal neighborhood of the non-object point and the reference point are all greater than the blind zone distance and less than the upper limit distance of the ground line; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0017] In some embodiments, the tilt angle condition includes: both the first tilt angle and the second tilt angle are less than the first preset angle, and the absolute value of the difference between the first tilt angle and the second tilt angle is less than the second preset angle; wherein, the first tilt angle is the tilt angle between the non-object point and the first neighboring point in the longitudinal direction, and the second tilt angle is the tilt angle between the non-object point and the second neighboring point in the longitudinal direction.

[0018] In some embodiments, the slope angle condition includes: the slope angle of a non-object point relative to a reference point is less than a third preset angle, and the absolute value of the difference between the slope angle and the first tilt angle is less than a fourth preset angle; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0019] In some embodiments, the method further includes: determining the ground height of the current frame based on the position information of the first row of points in the original point cloud data of the current frame; and determining the slope angle of the non-object points relative to the reference point based on the position information of the non-object points and the ground height of the current frame.

[0020] In some embodiments, the location information includes height values; determining the ground height of the current frame based on the location information of the first row of points in the original point cloud data of the current frame includes: if the average height value of the first row of points in the original point cloud data of the current frame is less than a preset height, determining the average height value of the first row of points in the original point cloud data of the current frame as the ground height of the current frame.

[0021] In some embodiments, the location information includes height values; determining the ground height of the current frame based on the location information of the first row of points in the original point cloud data of the current frame includes: if the average height value of the first row of points in the original point cloud data of the current frame is greater than or equal to a preset height, determining the ground height of the previous frame or the preset ground height as the ground height of the current frame.

[0022] In some embodiments, the distance consistency condition includes: the distances between the neighboring points in the preset lateral neighborhood of the non-object point and the reference point are all within a preset distance range; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0023] Secondly, embodiments of this application provide a point cloud processing apparatus, the apparatus comprising:

[0024] The acquisition module is used to acquire raw point cloud data, which includes information about multiple points.

[0025] The determination module is used to determine object points and ground points from multiple points based on information from multiple points;

[0026] The processing module is used to filter out invalid point information from the original point cloud data to obtain processed point cloud data; invalid points are points other than object points and ground points among multiple points.

[0027] In some embodiments, the determining module is specifically used to determine an object point from multiple points based on information from multiple points; and to determine a ground point from non-object points based on information from multiple points, wherein non-object points are other points from the multiple points excluding the object point.

[0028] In some embodiments, the determining module is specifically used to determine a target point that satisfies the local consistency condition as an object point based on the location information of the target point and the location information of the neighboring points of the target point. The target point is any one of multiple points.

[0029] In some embodiments, the local consistency condition includes: the distance between any one or more of the object point's top, bottom, left, and right neighboring points and the object point is less than or equal to a first preset distance threshold.

[0030] In some embodiments, the local consistency condition includes: the number of valid points in the neighborhood of the object point with a preset size is greater than or equal to a preset number; wherein, a valid point is a neighborhood point in the neighborhood of the object point with a preset size whose distance from the object point is less than a second preset distance threshold.

[0031] In some embodiments, the determining module is specifically used to determine non-object points that satisfy the ground coplanarity condition and the distance consistency condition as ground points based on the location information of the non-object points and the location information of the neighboring points of the non-object points.

[0032] In some embodiments, the ground coplanarity condition includes at least one of the following: ground line distance condition, tilt angle condition, and slope angle condition.

[0033] In some embodiments, the ground line distance condition includes: the distances between neighboring points in the preset longitudinal neighborhood of the non-object point and the reference point are all greater than the blind zone distance and less than the upper limit distance of the ground line; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0034] In some embodiments, the tilt angle condition includes: both the first tilt angle and the second tilt angle are less than the first preset angle, and the absolute value of the difference between the first tilt angle and the second tilt angle is less than the second preset angle; wherein, the first tilt angle is the tilt angle between the non-object point and the first neighboring point in the longitudinal direction, and the second tilt angle is the tilt angle between the non-object point and the second neighboring point in the longitudinal direction.

[0035] In some embodiments, the slope angle condition includes: the slope angle of a non-object point relative to a reference point is less than a third preset angle, and the absolute value of the difference between the slope angle and the first tilt angle is less than a fourth preset angle; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0036] In some embodiments, the determining module is further configured to determine the ground height of the current frame based on the position information of the first row of points in the original point cloud data of the current frame; and to determine the slope angle of the non-object points relative to the reference point based on the position information of the non-object points and the ground height of the current frame.

[0037] In some embodiments, the location information includes height values; the determining module is specifically used to determine the average height of the first row of points in the original point cloud data of the current frame as the ground height of the current frame when the average height of the first row of points in the original point cloud data of the current frame is less than a preset height.

[0038] In some embodiments, the location information includes height values; the determining module is specifically used to determine the ground height of the previous frame or the preset ground height as the ground height of the current frame when the average height value of the first row of points in the original point cloud data of the current frame is greater than or equal to a preset height.

[0039] In some embodiments, the distance consistency condition includes: the distances between the neighboring points in the preset lateral neighborhood of the non-object point and the reference point are all within a preset distance range; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0040] Thirdly, an electronic device is provided, comprising: a processor and a memory for storing processor-executable instructions; wherein the processor is configured to execute instructions to implement the point cloud processing method described above.

[0041] Fourthly, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the point cloud processing method described above.

[0042] Fifthly, a computer-readable storage medium is provided, on which computer program instructions are stored, which, when executed by a processor, implement the point cloud processing method of any of the above embodiments.

[0043] Sixthly, a lidar system is provided, which includes the electronic device described above, or the computer-readable storage medium described above.

[0044] In a seventh aspect, a vehicle is provided, comprising: the electronic device described above, or the computer-readable storage medium described above, or the lidar system described above.

[0045] Eighthly, a computer program product is provided, the computer program product including computer program instructions, which, when executed by a processor, implement the point cloud processing method of any of the above embodiments.

[0046] For a detailed description of aspects two through eight and their various implementations in this application, please refer to the detailed description in aspect one and its various implementations; and for a detailed description of the beneficial effects of aspects two through eight and their various implementations, please refer to the beneficial effect analysis in aspect one and its various implementations, which will not be repeated here. Attached Figure Description

[0047] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1A flowchart illustrating a point cloud processing method provided in this application embodiment;

[0049] Figure 2 This is a schematic diagram of point cloud data structure provided in an embodiment of this application;

[0050] Figure 3 A flowchart illustrating another point cloud processing method provided in this application embodiment;

[0051] Figure 4 A flowchart illustrating yet another point cloud processing method provided in this application embodiment;

[0052] Figure 5 A flowchart illustrating yet another point cloud processing method provided in this application embodiment;

[0053] Figure 6 This is a schematic diagram of the structure of a point cloud processing device provided in an embodiment of this application;

[0054] Figure 7 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0055] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0056] In the description of this application, it should be understood that the terms "upper," "lower," "left," "right," "front," "rear," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or relative positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and for simplification, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Unless otherwise specified, the above-mentioned orientational descriptions can be flexibly set in practical applications, provided that the relative positional relationships shown in the accompanying drawings are satisfied.

[0057] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0058] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," "linking," and "communication" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection. They can refer to a direct connection or an indirect connection through an intermediate medium, or a connection within two components. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0059] In embodiments of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, article, or apparatus that includes that element.

[0060] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0061] In the description of this application, unless otherwise stated, " / " means "or". For example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more.

[0062] With the continuous development of technology, lidar has been widely used in the field of detection due to its advantages of high resolution, good concealment, and strong anti-interference capabilities, particularly in intelligent robots and autonomous vehicles. However, during the detection of real objects, the lidar's emitted and received signals are affected by ambient light, spatial debris, and other factors. This results in false point cloud data, or noise, appearing in the lidar's output point cloud data, leading to poor data quality and inaccurate processing results. Therefore, it is necessary to filter out noise from the lidar's point cloud data to improve its quality.

[0063] In related technologies, isolated noise points in the original point cloud data are identified and deleted by measuring the distance between each data point and all other data points in the queue containing that data point. However, this method is prone to mistakenly deleting data points representing the edges of real objects and distant ground lines. This results in some loss of real objects and distant ground lines when isolated noise points are filtered out, leading to poor point cloud data quality. Therefore, how to filter point cloud data to improve its quality is a pressing technical problem that needs to be solved.

[0064] To address the aforementioned issues, this application provides a point cloud processing method. Based on information from multiple points in the original point cloud data, noise filtering is performed to identify object points representing real objects. Furthermore, ground points (excluding object points) are identified using ground coplanarity and distance consistency conditions. Invalid points other than object points and ground points are then removed from the original point cloud data. This process preserves the object points corresponding to real objects and the ground points corresponding to ground lines in the original point cloud data, preventing the presence of false point cloud data in the processed data, improving the quality of the point cloud data, and avoiding the loss of real objects and ground lines.

[0065] To facilitate understanding, the point cloud processing method provided in this application will be described in detail below with reference to the accompanying drawings.

[0066] In some embodiments, the execution subject of the point cloud processing method provided in this application may be the lidar itself, the lidar control device, the vehicle controller, or a lidar point cloud data processing device, etc., which can process the point cloud data of the lidar. This application does not limit this.

[0067] Figure 1 A point cloud processing method provided in this application embodiment, such as Figure 1 As shown, the method includes the following steps S101-S103:

[0068] S101. Obtain raw point cloud data.

[0069] The raw point cloud data includes information about multiple points.

[0070] In some embodiments, a one-dimensional rotating mirror lidar can acquire raw point cloud data using a line scanning method. Each column of point cloud data can be divided into four line scanning regions, each containing 24 data points. Once all four scanning regions for a column are scanned, that column of point cloud data is obtained. The process continues with the next column, and so on, until a complete frame of raw point cloud data from the lidar is obtained.

[0071] It should be noted that in the acquired raw point cloud data, the left and right neighboring points of each data point are data points that are 96 positions apart in the point cloud data, and the upper and lower neighboring points of each data point are data points that are 1 position apart in the point cloud data.

[0072] S102. Based on information from multiple points, determine the object point and the ground point from the multiple points.

[0073] Among them, object points and ground points are the data points corresponding to real objects and ground in the environment scanned by the lidar in the original point cloud data.

[0074] In some embodiments, the surfaces of real objects in the environment scanned by the LiDAR are typically continuous and smooth within local areas. Therefore, the distances between data points corresponding to real objects and their neighboring data points in the original point cloud data vary gradually. Thus, noise in the point cloud data can be filtered out based on the positional information of multiple points, determining the object point corresponding to the real object from among the multiple points, and ground points can be determined from non-object points other than the object point based on the characteristics of ground points in the point cloud data.

[0075] S103. Remove invalid point information from the original point cloud data to obtain the processed point cloud data.

[0076] Invalid points are those other than object points and ground points among multiple points.

[0077] It should be understood that after noise filtering of the original point cloud data, object points are obtained. Then, based on the characteristics of the ground points, a secondary judgment is performed to replenish the ground points that are not object points, resulting in processed point cloud data. This avoids the accidental deletion of ground points and improves the quality of the processed point cloud data.

[0078] Figure 2 This is a schematic diagram of point cloud data composition provided in an embodiment of this application, such as... Figure 2 As shown, each column of data in the point cloud is divided into four line scan areas. Figure 2 The diagram shows line scan areas 1, 2, 3, and 4. Each line scan area contains 24 data points. After the first column of data points is scanned, the next column is scanned, and so on, to obtain the spatial logical relationships between the data points in the point cloud data.

[0079] In some embodiments, such as Figure 3 As shown, step S102 above can be specifically implemented through the following steps S1021-S1022:

[0080] S1021. Based on information from multiple points, determine the object point from the multiple points.

[0081] In some embodiments, based on the continuity of the surface of a real object, for each of a plurality of points, the position information of that point can be compared with the position information of points in its neighborhood to determine the object point among the plurality of points. Specifically, different preset distance thresholds can be set for different neighborhoods of a point based on the position information of different points among the plurality of points to determine whether that point is an object point.

[0082] S1022. Based on information from multiple points, determine the ground points from non-object points.

[0083] Among them, non-object points are the points other than object points among multiple points.

[0084] In some embodiments, ground points in the point cloud data of a lidar system exhibit significant planar continuity. Therefore, the fluctuations in the positional information of ground points in the point cloud data are relatively slow and continuous, allowing ground points to be determined from non-object points based on the positional information of multiple points.

[0085] In some embodiments, step S1021 above can be implemented through the following steps:

[0086] Based on the location information of the target point and the location information of its neighboring points, the target point that satisfies the local consistency condition is determined as the object point.

[0087] The target point is any one of multiple points.

[0088] It should be noted that a target point satisfying the local consistency condition means that the target point and its neighboring points have continuity with the surface of a real object. Therefore, a target point that satisfies the local consistency condition is an object point.

[0089] As one way to implement the local consistency condition: the local consistency condition includes: the distance between any one or more of the object point's top, bottom, left, and right neighboring points and the object point is less than or equal to a first preset distance threshold.

[0090] In some embodiments, for any target point among multiple points, the distance between the target point and its neighboring points (up, down, left, and right) can be determined based on the target point's location information and the location information of its neighboring points (up, down, left, and right). If one or more of the distances between the target point and its neighboring points are less than or equal to a first preset distance threshold, it indicates that the target point is continuous with one or more of its neighboring points, and therefore the target point can be identified as an object point.

[0091] If there is no distance less than or equal to the first preset distance threshold between the target point and its neighboring points in the upper, lower, left, and right directions, the target point can be determined as a non-object point.

[0092] The first preset distance threshold is determined based on the location information of the target point and the surface features of the real object in the area scanned by the lidar, and this application embodiment does not limit it.

[0093] As another way to implement the local consistency condition, the local consistency condition includes: the number of valid points in the neighborhood of the object point with a preset size is greater than or equal to the preset number; wherein, the valid points are the neighborhood points in the neighborhood of the object point with a preset size whose distance from the object point is less than a second preset distance threshold.

[0094] For example, for any target point among multiple points, a 3x5 neighborhood of the target point can be selected. Based on the location information of the target point and the location information of the neighboring points in the 3x5 neighborhood, a distance matrix is ​​determined between the target point and its neighboring points. The number of distances in the distance matrix that are less than a second preset distance threshold is then counted to determine the number of valid points in the 3x5 neighborhood. If the number of valid points in the 3x5 neighborhood is greater than or equal to the preset number, it indicates that there is continuity between the target point and the neighboring points in the 3x5 neighborhood, and therefore the target point can be identified as an object point.

[0095] If the number of valid points in the 3x5 neighborhood is less than the preset number, the target point can be determined as a non-object point.

[0096] The second preset distance threshold and the preset number are determined based on the location information of the target point and the surface features of the real objects in the area scanned by the lidar. This application embodiment does not limit these.

[0097] It should be understood that, for multiple points in the original point cloud data, segmentation and regional processing are performed based on the spatial regional characteristics of the data points. Furthermore, dynamic preset distance thresholds are set for different neighborhoods of different data points, more accurately identifying object points among multiple points and improving the quality of the point cloud data.

[0098] In some embodiments, step S1022 above can be implemented through the following steps:

[0099] Based on the location information of non-object points and the location information of their neighboring points, non-object points that satisfy the ground coplanarity condition and the distance consistency condition are identified as ground points.

[0100] Among them, the ground coplanarity condition includes at least one of the following: ground line distance condition, tilt angle condition, and slope angle condition.

[0101] In some embodiments, the ground line distance condition includes: the distances between neighboring points in the preset longitudinal neighborhood of the non-object point and the reference point are all greater than the blind zone distance and less than the upper limit distance of the ground line.

[0102] The reference point is a reference point for the location information of multiple points in the original point cloud data, such as the location of a lidar.

[0103] For example, a 3x1 vertical neighborhood of a non-object point is selected as the preset vertical neighborhood of the non-object point. The 3x1 vertical neighborhood includes the left neighbor point and the right neighbor point of the non-object point. If the distance between the non-object point and its left neighbor point and the distance between the non-object point and its right neighbor point are both greater than the blind zone distance and less than the upper limit distance of the ground line, then the non-object point satisfies the ground line distance condition.

[0104] The blind zone distance is determined based on the configuration and equipment parameters of the lidar, and the upper limit distance of the ground line is determined based on the vertical distance of the lidar from the ground and the scanning area of ​​the lidar. This application embodiment does not limit these aspects.

[0105] In some embodiments, the tilt angle condition includes: both the first tilt angle and the second tilt angle are less than the first preset angle, and the absolute value of the difference between the first tilt angle and the second tilt angle is less than the second preset angle.

[0106] Wherein, the first tilt angle is the tilt angle between the non-object point and the first neighboring point in the vertical direction, and the second tilt angle is the tilt angle between the non-object point and the second neighboring point in the vertical direction.

[0107] For example, in the 3x1 vertical neighborhood of a non-object point, the coordinates of the left neighbor point are (x1, y1, z1), and the coordinates of the right neighbor point are (x3, y3, z3). The coordinates of the non-object point are (x2, y2, z2). The first and second tilt angles can then be determined based on the following expressions (1) and (2):

[0108] α1=tan -1 ((z2-z1) / (x2-x1)) (1)

[0109] α2=tan -1 ((z3-z2) / (x3-x2)) (2)

[0110] Where α1 is the first tilt angle and α2 is the second tilt angle.

[0111] When |α1|<9°, |α2|<9°, and |α1-α2|<2°, the non-object point satisfies the tilt angle condition.

[0112] The first preset angle and the second preset angle are determined according to the environment of the LiDAR scanning area. For example, the first preset angle is 9° and the second preset angle is 2°. This application embodiment does not limit this.

[0113] In some embodiments, the slope angle condition includes: the slope angle of the non-object point relative to the reference point is less than a third preset angle, and the absolute value of the difference between the slope angle and the first tilt angle is less than a fourth preset angle.

[0114] The reference point is a reference point for the location information of multiple points in the original point cloud data.

[0115] For example, the slope angle of a non-object point relative to a reference point can be determined based on the following expression (3):

[0116] β=sin -1 ((z2-h) / d2) (3)

[0117] Where β is the slope angle of the non-object point relative to the reference point. z2 is the height value of the non-object point. h is the ground height in the original point cloud data of the current frame. d2 is the distance between the non-object point and the reference point.

[0118] When β < 9° and |β-α1| < 2°, the non-object point satisfies the slope angle condition.

[0119] The third and fourth preset angles are determined based on the environment of the LiDAR scanning area and the road planning requirements within that environment. For example, the third preset angle is 9° and the fourth preset angle is 2°. This application embodiment does not limit this.

[0120] In some embodiments, the distance consistency condition includes: the distances between the neighboring points in the preset lateral neighborhood of the non-object point and the reference point are all within a preset distance range.

[0121] The reference point is a reference point for the location information of multiple points in the original point cloud data.

[0122] For example, a 1x5 horizontal neighborhood of a non-object point is selected as the preset horizontal neighborhood of the non-object point. This 1x5 horizontal neighborhood includes four neighboring points: A, B, C, and D. The distances between the four neighboring points A, B, C, and D and the reference point are a, b, c, and d, respectively. The distance between the non-object point and the reference point is d². The mean of a, b, c, d, and d² is γ. If a, b, c, d, and d² are all within the preset redundancy distance range of γ, then the non-object point satisfies the distance consistency condition.

[0123] The preset redundancy distance range is set by relevant technical personnel and can be ±0.1m, ±0.2m, or ±0.3m; this embodiment does not limit this range. The preset distance range is determined based on the average distance between neighboring points in the preset lateral neighborhood of the non-object point and the reference point; this embodiment does not limit this range.

[0124] In some embodiments, such as Figure 4 As shown, the point cloud processing method provided in this application embodiment further includes the following steps S401-S402:

[0125] S401. Determine the ground height of the current frame based on the position information of the first row of points in the original point cloud data of the current frame.

[0126] S402. Based on the position information of the non-object point and the ground height of the current frame, determine the slope angle of the non-object point relative to the reference point.

[0127] The ground height in the current frame is the ground height relative to the reference point. Location information includes the height value.

[0128] In some embodiments, the first row of points in the raw point cloud data typically corresponds to the nearest ground region within the LiDAR scanning range. Therefore, the ground height of the current frame can be determined based on the bit height value of the first row of points in the raw point cloud data of the current frame.

[0129] As one implementation of S401: if the average height of the first row of points in the original point cloud data of the current frame is less than the preset height, the average height of the first row of points in the original point cloud data of the current frame is determined as the ground height of the current frame.

[0130] Since the LiDAR is typically positioned above the ground when scanning its surroundings, the ground height in the current frame should be less than a preset height. Therefore, if the average height of the first row of points in the raw point cloud data of the current frame is less than the preset height, the average height of the first row of points can be directly determined as the ground height of the current frame.

[0131] The preset height is determined based on the position of the lidar and can be zero; however, this embodiment does not limit this.

[0132] As another implementation of S401: if the average height of the first row of points in the raw point cloud data of the current frame is greater than or equal to the preset height, the ground height of the previous frame or the preset ground height is determined as the ground height of the current frame.

[0133] If the average height of the first row of points in the raw point cloud data of the current frame is greater than or equal to a preset height, the average height of the first row of points cannot be used as the ground height of the current frame. Since the point cloud data of two consecutive LiDAR frames are not significantly different, the ground height of the previous frame can be used as the ground height of the current frame. Alternatively, the height of a vehicle-mounted LiDAR is generally a fixed height, so a preset ground height can be determined based on the height of the vehicle-mounted LiDAR and set as the ground height of the current frame.

[0134] It should be understood that by judging each point in the original point cloud data, identifying object points and ground points, and filtering out invalid points, the quality of the point cloud data is improved. Furthermore, the point cloud processing methods in this application all employ low-computational-complexity judgment methods, enabling real-time optimization of point cloud data at the radar chip level with a frame rate of 10Hz. This achieves a dual guarantee of timeliness optimization and quality improvement.

[0135] Figure 5 A flowchart of another point cloud processing method provided in the embodiments of this application is shown below. Figure 5 As shown, the method includes the following steps:

[0136] S501. Obtain the raw point cloud data from the lidar.

[0137] S502. For each point in the original point cloud data, based on the location information of each point, set a preset distance threshold for different neighboring points of each point.

[0138] The preset distance threshold is determined based on the location information of each point and the size of the neighborhood of that point.

[0139] S503. Select the neighborhood of the current point with a preset size.

[0140] S504. Obtain the distance between the current point and the neighboring points within the preset size neighborhood.

[0141] S505. Determine whether the distance between the neighboring point and the current point is less than the second preset distance threshold.

[0142] If the distance between the neighboring point and the current point is not less than the second preset distance threshold, return to step S502 to judge the next data point.

[0143] If the distance between a neighboring point and the current point is less than a second preset distance threshold, proceed to step S506.

[0144] S506. Count the number of valid points in the neighborhood of the preset size.

[0145] S507. Determine whether the number of valid points in the neighborhood of the preset size is greater than the preset number.

[0146] S508. Determine whether there are any neighboring points in the current point's upper, lower, left, and right neighboring points whose distance from the current point is less than a first preset distance threshold.

[0147] If the number of valid points in the neighborhood of the preset size is greater than the preset number, or if there are neighboring points in the current point's upper, lower, left, and right neighborhoods that are less than the first preset distance threshold, then step S509 is executed.

[0148] If the number of valid points in the neighborhood of the preset size is not greater than the preset number, and there are no neighboring points in the current point's upper, lower, left, and right neighboring points whose distance to the current point is less than the first preset distance threshold, return to step S502 to judge the next data point.

[0149] S509. Determine the current point as an object point.

[0150] S510. For non-object points, determine the ground height of the current frame based on the position information of the first row of data points in the original point cloud data of the current frame.

[0151] S511. Determine whether the current point meets the ground line distance condition.

[0152] If the ground line distance condition is met at the current point, proceed to step S512.

[0153] If the current point does not meet the ground line distance condition, return to step S510 to judge the next non-object point.

[0154] S512. Determine whether the current point meets the tilt angle condition.

[0155] If the tilt angle condition is met at the current point, proceed to step S513.

[0156] If the tilt angle condition is not met at the current point, return to step S510 to judge the next non-object point.

[0157] S513. Determine whether the current point meets the slope angle condition.

[0158] If the slope angle condition is met at the current point, proceed to step S514.

[0159] If the slope angle condition is not met at the current point, return to step S510 to judge the next non-object point.

[0160] S514. Determine whether the current point satisfies the distance consistency condition.

[0161] If the distance consistency condition is met at the current point, proceed to step S515.

[0162] If the distance consistency condition is not met at the current point, return to step S510 to judge the next non-object point.

[0163] S515. Determine the current point as the ground point.

[0164] S516. Remove invalid point information from the original point cloud data to obtain the processed point cloud data.

[0165] The foregoing mainly describes the solutions of the embodiments of this application from a methodological perspective. It is understood that, in order to achieve the above functions, the point cloud processing device includes at least one of the hardware structures and software modules corresponding to each function. Those skilled in the art should readily recognize that, in conjunction with the units and algorithm steps of the various examples described in the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0166] This application embodiment can divide the point cloud processing device into functional modules according to the above method embodiment. For example, each function can be divided into a separate functional module, or two or more functions can be integrated into one functional module. The integrated module can be implemented in hardware or software. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division. In actual implementation, there may be other division methods. The following description uses the example of dividing each function into a separate functional module.

[0167] Figure 6 This is a schematic diagram of a point cloud processing apparatus provided in an embodiment of this application. The point cloud processing apparatus can execute the point cloud processing method provided in the above-described method embodiments. Figure 6 As shown, the point cloud processing device 600 includes: an acquisition module 601, a determination module 602, and a processing module 603.

[0168] The acquisition module 601 is used to acquire raw point cloud data, which includes information about multiple points.

[0169] The determination module 602 is used to determine the object point and the ground point from multiple points based on information from multiple points;

[0170] The processing module 603 is used to filter out invalid point information from the original point cloud data to obtain processed point cloud data; invalid points are points other than object points and ground points among multiple points.

[0171] In some embodiments, the determining module 602 is specifically used to determine an object point from multiple points based on information from multiple points; and to determine a ground point from non-object points based on information from multiple points, wherein non-object points are other points from the multiple points excluding the object point.

[0172] In some embodiments, the determining module 602 is specifically used to determine, for any one of the multiple points, the target point that satisfies the local consistency condition as the object point based on the location information of the target point and the location information of the neighboring points of the target point.

[0173] In some embodiments, the local consistency condition includes: the distance between any one or more of the object point's top, bottom, left, and right neighboring points and the object point is less than or equal to a first preset distance threshold.

[0174] In some embodiments, the local consistency condition includes: the number of valid points in the neighborhood of the object point with a preset size is greater than or equal to a preset number; wherein, a valid point is a neighborhood point in the neighborhood of the object point with a preset size whose distance from the object point is less than a second preset distance threshold.

[0175] In some embodiments, the determining module 602 is specifically used to determine, for any non-object point, a non-object point that satisfies the ground coplanarity condition and the distance consistency condition as a ground point based on the location information of the non-object point and the location information of the neighboring points of the non-object point.

[0176] In some embodiments, the ground coplanarity condition includes: ground line distance condition, tilt angle condition, and slope angle condition.

[0177] In some embodiments, the ground line distance condition includes: the distances between neighboring points in the preset longitudinal neighborhood of the non-object point and the reference point are all greater than the blind zone distance and less than the upper limit distance of the ground line; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0178] In some embodiments, the tilt angle condition includes: both the first tilt angle and the second tilt angle are less than the first preset angle, and the absolute value of the difference between the first tilt angle and the second tilt angle is less than the second preset angle; wherein, the first tilt angle is the tilt angle between the non-object point and the first neighboring point in the longitudinal direction, and the second tilt angle is the tilt angle between the non-object point and the second neighboring point in the longitudinal direction.

[0179] In some embodiments, the slope angle condition includes: the slope angle of a non-object point relative to a reference point is less than a third preset angle, and the absolute value of the difference between the slope angle and the first tilt angle is less than a fourth preset angle; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0180] In some embodiments, the determining module 602 is further configured to determine the ground height of the current frame based on the position information of the first row of points in the original point cloud data of the current frame; and to determine the slope angle of the non-object points relative to the reference point based on the position information of the non-object points and the ground height of the current frame.

[0181] In some embodiments, the location information includes height values; the determining module 602 is specifically used to determine the average height of the first row of points in the original point cloud data of the current frame as the ground height of the current frame when the average height of the first row of points in the original point cloud data of the current frame is less than a preset height.

[0182] In some embodiments, the location information includes height values; the determining module 602 is specifically used to determine the ground height of the previous frame or the preset ground height as the ground height of the current frame when the average height value of the first row of points in the original point cloud data of the current frame is greater than or equal to a preset height.

[0183] In some embodiments, the distance consistency condition includes: the distances between the neighboring points in the preset lateral neighborhood of the non-object point and the reference point are all within a preset distance range; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

[0184] When implementing the functions of the integrated modules described above in hardware, this application provides a possible structure for the electronic device involved in the above embodiments. For example... Figure 7 As shown, the electronic device 700 includes: a processor 702 and a bus 704. Optionally, the electronic device 700 may also include a memory 701; optionally, the electronic device 700 may also include a communication interface 703.

[0185] Processor 702 may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this application. Processor 702 may be a central processing unit, a general-purpose processor, a digital signal processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with embodiments of this application. Processor 702 may also be a combination that implements computing functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0186] The communication interface 703 is used to connect to other devices via a communication network. This communication network can be Ethernet, wireless access network, wireless local area network (WLAN), etc.

[0187] The memory 701 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), disk storage medium or other magnetic storage device, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but is not limited thereto.

[0188] As one possible implementation, the memory 701 can exist independently of the processor 702. The memory 701 can be connected to the processor 702 via a bus 704 and is used to store instructions or program code. When the processor 702 calls and executes the instructions or program code stored in the memory 701, it can implement the point cloud processing method provided in the embodiments of this application.

[0189] In another possible implementation, the memory 701 can also be integrated with the processor 702.

[0190] The 704 bus can be an extended industry standard architecture (EISA) bus, etc. The 704 bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 7 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0191] Some embodiments of this application provide a computer-readable storage medium (e.g., a non-transitory computer-readable storage medium) storing computer program instructions that, when executed on a computer, cause the computer to perform a point cloud processing method as described in any of the above embodiments.

[0192] For example, the aforementioned computer-readable storage media may include, but are not limited to: magnetic storage devices (e.g., hard disks, floppy disks, or magnetic tapes), optical discs (e.g., compact disks (CDs), digital versatile disks (DVDs), etc.), smart cards, and flash memory devices (e.g., erasable programmable read-only memory (EPROMs), cards, sticks, or key drives, etc.). The various computer-readable storage media described in this application may represent one or more devices and / or other machine-readable storage media for storing information. The term "machine-readable storage medium" may include, but is not limited to, wireless channels and various other media capable of storing, containing, and / or carrying instructions and / or data.

[0193] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to execute the point cloud processing method of any of the above embodiments.

[0194] In the description of this specification, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.

[0195] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A point cloud processing method, characterized in that, The method includes: Acquire raw point cloud data, which includes information about multiple points; Based on the information from the multiple points, object points and ground points are determined from the multiple points; The invalid point data is obtained by filtering out the invalid point information from the original point cloud data; the invalid point is any point other than the object point and the ground point among the multiple points.

2. The method according to claim 1, characterized in that, The step of determining the object point and the ground point from the multiple points based on the information from the multiple points includes: Based on the information from the plurality of points, the object point is determined from the plurality of points; Based on the information from the plurality of points, the ground point is determined from the non-object points, wherein the non-object points are the points other than the object points among the plurality of points.

3. The method according to claim 2, characterized in that, The step of determining the object point from the plurality of points based on the information from the plurality of points includes: Based on the location information of the target point and the location information of the neighboring points of the target point, the target point that satisfies the local consistency condition is determined as the object point, and the target point is any one of the multiple points.

4. The method according to claim 3, characterized in that, The local consistency condition includes: the distance between any one or more of the object point's top, bottom, left, and right neighboring points and the object point is less than or equal to a first preset distance threshold.

5. The method according to claim 3, characterized in that, The local consistency condition includes: the number of valid points in the neighborhood of the object point with a preset size is greater than or equal to a preset number; wherein, the valid points are neighborhood points in the neighborhood of the object point with a preset size whose distance from the object point is less than a second preset distance threshold.

6. The method according to claim 2, characterized in that, The step of determining the ground point from non-object points based on the information of the plurality of points includes: Based on the location information of the non-object points and the location information of the neighboring points of the non-object points, non-object points that satisfy the ground coplanarity condition and the distance consistency condition are determined as ground points.

7. The method according to claim 6, characterized in that, The ground coplanar conditions include at least one of the following: ground line distance condition, tilt angle condition, and slope angle condition.

8. The method according to claim 7, characterized in that, The ground line distance conditions include: the distances between the neighboring points in the preset longitudinal neighborhood of the non-object point and the distance between the non-object point and the reference point are both greater than the blind zone distance and less than the upper limit distance of the ground line; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

9. The method according to claim 7, characterized in that, The tilt angle conditions include: both the first tilt angle and the second tilt angle are less than the first preset angle, and the absolute value of the difference between the first tilt angle and the second tilt angle is less than the second preset angle; wherein, the first tilt angle is the tilt angle between the non-object point and the first neighboring point in the longitudinal direction, and the second tilt angle is the tilt angle between the non-object point and the second neighboring point in the longitudinal direction.

10. The method according to claim 7, characterized in that, The slope angle conditions include: the slope angle of the non-object point relative to the reference point is less than a third preset angle, and the absolute value of the difference between the slope angle and the first tilt angle is less than a fourth preset angle; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

11. The method according to claim 10, characterized in that, The method further includes: Based on the position information of the first row of points in the original point cloud data of the current frame, determine the ground height of the current frame; Based on the location information of the non-object point and the ground height of the current frame, the slope angle of the non-object point relative to the reference point is determined.

12. The method according to claim 11, characterized in that, The location information includes an altitude value; determining the ground altitude of the current frame based on the location information of the first row of points in the original point cloud data of the current frame includes: If the average height of the first row of points in the original point cloud data of the current frame is less than a preset height, the average height of the first row of points in the original point cloud data of the current frame is determined as the ground height of the current frame.

13. The method according to claim 11, characterized in that, The location information includes an altitude value; determining the ground altitude of the current frame based on the location information of the first row of points in the original point cloud data of the current frame includes: If the average height of the first row of points in the original point cloud data of the current frame is greater than or equal to a preset height, the ground height of the previous frame or the preset ground height is determined as the ground height of the current frame.

14. The method according to claim 6, characterized in that, The distance consistency condition includes: the distances between the neighboring points in the preset horizontal neighborhood of the non-object point and the distances between the non-object point and the reference point are all within a preset distance range; wherein, the reference point is a reference point for the position information of multiple points in the original point cloud data.

15. An electronic device, characterized in that, include: A processor and a memory for storing processor-executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 14.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1 to 14.

17. A lidar system, characterized in that, include: The electronic device of claim 15, or the computer-readable storage medium of claim 16.

18. A vehicle, characterized in that, include: The electronic device of claim 15, or the computer-readable storage medium of claim 16, or the lidar system of claim 17.

19. A computer program product, characterized in that, The computer program product includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1 to 14.

Images