A point cloud sequence ground segmentation method and device, electronic equipment and storage medium

Abstract

The application discloses a point cloud sequence ground segmentation method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring a point cloud sequence and a pose sequence corresponding to the point cloud sequence in the driving process of an unmanned vehicle; extracting ground point clouds in the point cloud sequence to obtain ground scene point clouds; segmenting a motion trajectory according to the pose sequence and a preset trajectory segmentation length to obtain a segmented line segment set; wherein the motion trajectory is obtained by splicing the pose sequence; using the segmented line segment set to perform regional blocking on the point clouds in the ground scene point clouds to obtain a segmented region set; and performing ground fitting on the segmented region set to obtain a ground point cloud segmentation result. The technical scheme can realize the effects of ground segmentation and fitting of a scene in the driving process of an unmanned vehicle, filters out point cloud information of obstacles of the autonomous vehicle in the ground point cloud segmentation result, avoids ground point cloud noise interference, and restores the real ground conditions.

Description

Technical Field

[0001] This invention relates to the field of unmanned vehicle trajectory segmentation technology, and in particular to a point cloud sequence ground segmentation method, device, electronic device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, autonomous driving technology is becoming increasingly mature. Autonomous driving technology can be broadly categorized into perception, prediction, localization, decision-making, planning, and control. Perception typically includes static scene perception and dynamic scene perception. Ground segmentation in offline scene reconstruction tasks is a fundamental task in static scene perception, providing raw input for related tasks such as 3D lane line recognition and ground traffic sign recognition. It is also crucial for navigation and obstacle recognition.

[0003] However, the ground is often not a flat plane because it has steep slopes, bumpy roads, or objects that interfere with its movement, such as curbs, flower beds, vehicles, and pedestrians. Current technologies directly use planar fitting methods to segment ground point clouds.

[0004] Directly using planar fitting methods for ground segmentation of point clouds can result in insufficient segmentation, leading to a lack of ground information. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for ground segmentation of point cloud sequences, which can achieve the effect of ground segmentation and fitting of a scene, avoid interference from point cloud noise on the ground, as well as changes in ground slope, and restore the real ground situation.

[0006] According to one aspect of the present invention, a method for ground segmentation of a point cloud sequence is provided, the method comprising:

[0007] Obtain the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process;

[0008] The ground point cloud is extracted from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain the ground scene point cloud;

[0009] The motion trajectory is segmented according to the pose sequence and the preset trajectory segment length to obtain a set of segmented line segments; wherein, the motion trajectory is obtained by splicing the pose sequence;

[0010] Using the segmented line segment set, the point cloud in the ground scene point cloud is divided into regions to obtain a set of segmented regions;

[0011] The ground is fitted to the block region set using the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0012] According to another aspect of the present invention, a point cloud sequence ground segmentation apparatus is provided, the apparatus comprising:

[0013] The point cloud sequence acquisition module is used to acquire the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process.

[0014] The ground scene point cloud acquisition module is used to extract ground point clouds from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain ground scene point clouds;

[0015] The segmented line segment set acquisition module is used to segment the motion trajectory according to the pose sequence and the preset trajectory segment length to obtain a segmented line segment set; wherein, the motion trajectory is obtained by splicing the pose sequence;

[0016] The module for obtaining the block region set is used to divide the point cloud in the ground scene point cloud into regions using the segmented line segment set, thereby obtaining the block region set.

[0017] The ground point cloud segmentation result acquisition module is used to perform ground fitting on the block region set based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0018] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform a point cloud sequence ground segmentation method according to any embodiment of the present invention.

[0022] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement a point cloud sequence ground segmentation method according to any embodiment of the present invention.

[0023] The technical solution of this invention obtains a point cloud sequence and a corresponding pose sequence during the driving process of an autonomous vehicle. Based on the patchwork++ ground segmentation algorithm, it extracts ground point clouds from the point cloud sequence to obtain a ground scene point cloud. Then, it segments the motion trajectory according to the pose sequence and a preset trajectory segment length, obtaining a set of segmented line segments. Using this set of segmented line segments, it further divides the point cloud in the ground scene into regions, obtaining a set of segmented regions. Finally, it performs ground fitting on the set of segmented regions using the RANSAC plane fitting method to obtain the ground point cloud segmentation result. This technical solution can achieve effective ground segmentation and fitting of a scene, avoiding interference from ground point cloud noise and ground slope changes, and restoring the true ground conditions.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a flowchart of a point cloud sequence ground segmentation method provided in Embodiment 1 of the present invention;

[0027] Figure 2 This is a schematic diagram of a point cloud sequence ground segmentation process provided in Embodiment 1 of this application;

[0028] Figure 3 This is a trajectory segmentation result diagram provided in Embodiment 1 of this application;

[0029] Figure 4 This is a schematic diagram of the ground fitting process provided in Embodiment 1 of this application;

[0030] Figure 5 This is a ground point cloud segmentation result image provided in Embodiment 1 of this application;

[0031] Figure 6 This is a flowchart of the motion trajectory segmentation process provided in Embodiment 2 of the present invention;

[0032] Figure 7 This is a flowchart of another motion trajectory segmentation process provided in Embodiment 2 of this application;

[0033] Figure 8This is a flowchart of the process for determining the set of segmented regions provided in Embodiment 3 of the present invention;

[0034] Figure 9 This is a flowchart of another process for determining a set of segmented regions provided in Embodiment 3 of this application;

[0035] Figure 10 This is a schematic diagram of constructing a set of segmented regions provided in Embodiment 3 of this application;

[0036] Figure 11 This is a 3D lane line annotation result diagram provided in the embodiments of this application;

[0037] Figure 12 This is a schematic diagram of the structure of a point cloud sequence ground segmentation device provided in Embodiment 4 of the present invention;

[0038] Figure 13 This is a schematic diagram of the structure of an electronic device that implements a point cloud sequence ground segmentation method according to an embodiment of the present invention. Detailed Implementation

[0039] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0040] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0041] Example 1

[0042] Figure 1This is a flowchart of a point cloud sequence ground segmentation method according to Embodiment 1 of the present invention. This embodiment is applicable to the segmentation of ground point clouds. The method can be executed by a point cloud sequence ground segmentation device, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0043] S110. Obtain the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the driving process of the unmanned vehicle.

[0044] In this embodiment, the point cloud sequence can be represented as {PC1, PC2, ..., PC...} N The pose sequence corresponding to the point cloud sequence can be represented as {P1, P2, ..., P}. N The symbol} represents the point cloud sequence. Here, N in the subscript represents the total number of frames in the point cloud, and also the number of poses in the pose sequence. Point cloud sequence PC i The number of points (i = 1, 2, ..., N) is denoted as n. i (i = 1, 2, ..., N), point cloud sequence PC i The coordinates of a point p in the equation are denoted as p = (x p ,y p ,z p Pose sequence P i The translation vector q in (i = 1, 2, ..., N) is denoted as q = (x q ,y q ,z q ).

[0045] In this solution, the point cloud sequence and the corresponding pose sequence of the point cloud can be obtained based on the radar and IMU (Inertial Measurement Unit, autonomous driving hardware system) installed on the autonomous vehicle.

[0046] In this embodiment, Figure 2 This is a schematic diagram of a point cloud sequence ground segmentation process provided in Embodiment 1 of this application, as shown below. Figure 2 As shown, the point cloud sequence {PC1, PC2, ..., PC} during the autonomous vehicle's driving process is determined. N} and the pose sequence corresponding to the point cloud sequence {P1,P2,…,P} N}

[0047] S120. Based on the patchwork++ ground segmentation algorithm, extract the ground point cloud from the point cloud sequence to obtain the ground scene point cloud.

[0048] Among them, ground scene point clouds can be represented by F. PC express.

[0049] In this embodiment, the point cloud sequence may contain various types of point clouds such as ground point clouds, building point clouds, and tree point clouds. By extracting the ground point clouds from the point cloud sequence, a point cloud containing only the ground scene can be obtained.

[0050] Optionally, the ground point cloud in the point cloud sequence is extracted to obtain the ground scene point cloud, including step A1:

[0051] Step A1: Extract the ground point cloud from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain the ground scene point cloud.

[0052] In this plan, such as Figure 2 As shown, for the point cloud sequence {PC1,PC2,…,PC…} N The ground point cloud in each frame of the point cloud sequence can be extracted using the patchwork++ ground segmentation algorithm to obtain a new point cloud T. i (i = 1, 2, ..., N), and the point cloud T i The ground scene point cloud F is obtained by stitching together (i = 1, 2, ..., N). PC .

[0053] By extracting ground point clouds from point cloud sequences, ground information during vehicle movement can be taken into account, enabling the filtering out of obvious obstacles on the ground.

[0054] S130. The motion trajectory is segmented according to the pose sequence and the preset trajectory segment length to obtain a set of segmented line segments; wherein, the motion trajectory is obtained by splicing the pose sequence.

[0055] The preset trajectory segment length can be set according to the segmentation requirements of the autonomous vehicle's motion trajectory. The length of the trajectory segment can be represented by S.

[0056] In this scheme, the set of segmented line segments can be represented as {SL1,SL2,…,SL3}. M The symbol} represents the total number of line segments. The subscript M indicates the total number of segments.

[0057] In this embodiment, as Figure 2 As shown, when the pose sequence {P1, P2, ..., P} is determined, N}, and after determining the trajectory segment length S, we can base it on the pose sequence {P1,P2,…,P} N The trajectory is divided into segments by the length S of the segments to obtain a set of segmented line segments {SL1,SL2,…,SL3}. M}

[0058] Specifically, Figure 3This is a trajectory segmentation result diagram provided in Embodiment 1 of this application. Figure 3 (a) shows the trajectory segmentation results for a straight-line scene; Figure 3 (b) shows the trajectory segmentation results for a straight-line scene. For example... Figure 3 As shown in (a) and (b), the motion trajectory of straight and curved scenes can be segmented according to the pose sequence and the preset trajectory segment length.

[0059] S140. Using the segmented line segment set, the point cloud in the ground scene point cloud is divided into regions to obtain a segmented region set.

[0060] The set of partitioned regions can be represented as {SEG1, SEG2, ..., SEG}. L The} indicates that the L in the subscript represents the number of regions corresponding to the segment.

[0061] In this embodiment, as Figure 2 As shown, we can determine the segmented line segment set {SL1,SL2,…,SL3} based on the segmented line segment set {SL1,SL2,…,SL3}. M}, Ground scene point cloud F PC Based on the Euclidean distance from a point to a line segment, the ground scene point cloud F PC The point cloud is divided into regions to obtain each segment line segment {SL1,SL2,…,SL3}. M The corresponding set of partitioned regions {SEG1, SEG2, ..., SEG} L}

[0062] S150. Based on the RANSAC plane fitting method, perform ground fitting on the block region set to obtain the ground point cloud segmentation result.

[0063] In this embodiment, as Figure 2 As shown, the RANSAC plane fitting method (RAndom SAmpleConsensus, random sampling consensus) can be used to fit the block region set {SEG1, SEG2, ..., SEG}. L Ground fitting is performed on the point cloud in the data to obtain optimized ground point cloud segmentation results.

[0064] Optionally, ground fitting is performed on the block region set based on the RANSAC plane fitting method to obtain ground point cloud segmentation results, including:

[0065] The point cloud in the block region set is successively fitted to the ground using the RANSAC plane fitting method, and the remaining point cloud after fitting is stored in the point cloud set.

[0066] After performing ground fitting on all the segmented region sets, the point cloud in the point cloud set is then fitted with ground based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0067] Specifically, performing ground fitting on the aforementioned block region set to obtain the ground point cloud segmentation result can be achieved using the following steps B1-B5:

[0068] Step B1: Determine the sixth variable; the sixth variable is used to characterize the segment index in the segmented line segment set; the initial value of the sixth variable is 1;

[0069] The sixth variable can be represented by sl_idx, and the initial value of sl_idx is 1.

[0070] Step B2: Determine the seventh variable; the seventh variable is used to characterize the index of the block region in the block region set; the initial value of the seventh variable is 1;

[0071] The seventh variable can be represented by seg_idx, and the initial value of seg_idx is 1.

[0072] In this plan, Figure 4 This is a schematic diagram of the ground fitting process provided in Embodiment 1 of this application, where each segment SL... i The set of partitioned regions corresponding to (i = 1, 2, ..., M) {SEG1, SEG2, ..., SEG} L}, where M represents the number of segmented lines and L represents the number of block regions. The set of block regions is {SEG1, SEG2, ..., SEG}. L The midpoint coordinates p are denoted as p = (x p ,y p ,z p ).like Figure 4 As shown, a variable sl_idx is defined as the segment index in a set of segmented line segments; and a variable seg_idx is defined as the block region index in a set of block regions.

[0073] Step B3: Perform planar fitting on all point clouds in the block region corresponding to the seventh variable, and store the remaining point clouds after fitting into the point cloud set;

[0074] Point cloud collections can be used with a PC. set express.

[0075] In this embodiment, the block region corresponding to the seventh variable can be represented as SEG. seg_idx ,like Figure 4 As shown, the RANSAC algorithm can be used to analyze the block region SEG corresponding to the seventh variable. seg_idxAll point clouds are fitted to a plane, then outliers are filtered out, and the remaining points p are recorded in the point cloud set PC. set middle.

[0076] Step B4: Increment the seventh variable by one. If the incremented seventh variable is greater than the number of block regions in the block region set, then perform plane fitting on all point clouds in the point cloud set to obtain the ground point cloud segmentation result.

[0077] In this scheme, the seventh variable after accumulation can be calculated using the following formula: seg_idx = seg_idx + 1;

[0078] In this embodiment, as Figure 4 As shown, after obtaining the accumulated seventh variable seg_idx, it is determined whether the accumulated seventh variable seg_idx is greater than L. If so, the RANSAC algorithm is used to process the point cloud set PC. set All point clouds in the dataset are subjected to planar fitting to obtain the ground point cloud segmentation result. After planar fitting, the point cloud set PC is... set There may still be some anomalies, such as clearing the point cloud collection PC. set This is so that the remaining point cloud data can be stored in the point cloud collection PC in the next loop. set middle.

[0079] Furthermore, if the summed seventh variable seg_idx is not greater than L, then continue executing steps B3-B4 until the summed seventh variable seg_idx is greater than L.

[0080] Step B5: Increment the sixth variable by one. If the incremented sixth variable is greater than the number of segments in the segmented line segment set, then end the ground fitting of the segmented region set, thereby obtaining all ground point cloud segmentation results.

[0081] The sixth variable after accumulation can be calculated using the following formula: sl_idx = sl_idx + 1.

[0082] In this plan, such as Figure 4 As shown, after performing planar fitting on all point clouds in the point cloud set to obtain the ground point cloud segmentation result, the sixth variable is incremented by one to obtain the incremented sixth variable sl_idx. It is then determined whether the incremented sixth variable sl_idx is greater than M. If so, the ground fitting of the block region set is terminated, thus obtaining all ground point cloud segmentation results.

[0083] Furthermore, if the accumulated sixth variable sl_idx is not greater than M, then continue to execute steps B2-B5 until the accumulated sixth variable sl_idx is greater than M, and end the ground fitting of the block region set.

[0084] In this embodiment, Figure 5 This is a ground point cloud segmentation result image provided in Embodiment 1 of this application. Figure 5 (a) shows the original point cloud scene. Figure 5 (b) shows the results of ground point cloud extraction. Figure 5 (c) is a top-down view, such as Figure 5 As shown in (a), (b), and (c), ground segmentation of the point cloud sequence can avoid noise interference from the ground point cloud and restore the true ground situation.

[0085] The technical solution of this invention obtains a point cloud sequence and a corresponding pose sequence during the autonomous vehicle's driving process. Based on the patchwork++ ground segmentation algorithm, it extracts ground point clouds from the point cloud sequence to obtain a ground scene point cloud. Then, it segments the motion trajectory according to the pose sequence and a preset trajectory segment length, obtaining a set of segmented line segments. Using this set of segmented line segments, it further divides the point cloud in the ground scene into regions, obtaining a set of segmented regions. Finally, it performs ground fitting on the set of segmented regions using the RANSAC plane fitting method to obtain the ground point cloud segmentation result. By implementing this technical solution, the ground segmentation and fitting effect of the scene during autonomous vehicle driving can be achieved. The ground point cloud segmentation result filters out point cloud information of obstacles of the autonomous vehicle, avoiding noise interference from ground point clouds and changes in ground slope, thus restoring the true ground condition.

[0086] Example 2

[0087] Figure 6 This is a flowchart of the motion trajectory segmentation process provided in Embodiment 2 of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the motion trajectory segmentation process. Figure 6 As shown, the method includes:

[0088] S610. Obtain the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process.

[0089] S620. Based on the patchwork++ ground segmentation algorithm, extract the ground point cloud from the point cloud sequence to obtain the ground scene point cloud.

[0090] S630. Calculate the Euclidean distance between two adjacent poses in the pose sequence in sequence to obtain the distance value.

[0091] S640. If the distance value is less than or equal to the preset trajectory segment length, then determine the segment set based on two adjacent poses in the pose sequence.

[0092] S650. If the distance value is greater than the preset trajectory segment length, then based on the two adjacent poses in the pose sequence and the trajectory segment length, determine the target pose from between the two adjacent poses, and based on the previous pose and the target pose among the two adjacent poses, determine the segment set.

[0093] Specifically, the set of segmented line segments can be determined using the following steps 1-8:

[0094] Step 1: Determine the first variable; the first variable is used to characterize the pose coordinates in the pose sequence; the initial value of the first variable is the first pose coordinate in the pose sequence.

[0095] The first variable can be represented by PRE_PT.

[0096] Step 2: Determine the second variable; the second variable is used to characterize the pose coordinate index in the pose sequence; the initial value of the second variable is 2.

[0097] In this embodiment, the second variable can be represented by p_idx, and the initial value of p_idx is 2.

[0098] Step 3: Determine the third variable; the third variable is used to characterize the pose coordinates corresponding to the second variable.

[0099] In this scheme, the third variable can be represented by CUR_PT, where CUR_PT = PT p_idx .

[0100] in, Figure 7 This is a flowchart of another motion trajectory segmentation process provided in Embodiment 2 of this application, with pose sequence P. i The translation vector q in (i = 1, 2, ..., N) is denoted as q = (x q ,y q ,z q Take the x and y vectors, and obtain PT. Where PT = (x... q ,y q That is, the PT sequence {PT1,PT2,…,PT} can be obtained. N}

[0101] Furthermore, PT1 is used as the starting position PT of the trajectory. b PT b =PT1; and PT N The endpoint PT of the trajectory e PT e =PT N .

[0102] In this embodiment, as Figure 7As shown, a variable PRE_PT is defined to represent the pose coordinates in a pose sequence, a variable p_idx is defined to represent the pose coordinate index in a pose sequence, and a variable CUR_PT is defined to represent the pose coordinates corresponding to a second variable.

[0103] Step 4: Calculate the Euclidean distance between the first variable and the third variable to obtain the distance value.

[0104] The distance value can be represented by D. PT express.

[0105] Specifically, such as Figure 7 As shown, for the first variable PRE_PT = (x1, y1) and the third variable CUR_PT = (x2, y2), the distance between the first variable PRE_PT = (x1, y1) and the third variable CUR_PT = (x2, y2) can be calculated using the following formula:

[0106]

[0107] Step 5: If the distance value is greater than the preset trajectory segment length, then determine the target point cloud based on the first variable, the third variable, and the trajectory segment length; wherein, the Euclidean distance between the first variable and the target point cloud is equal to the trajectory segment length.

[0108] The target point cloud can be represented by TMP_PT.

[0109] In this embodiment, as Figure 7 As shown, after obtaining the distance value D PT Then, determine D. PT If the value is greater than S, then the target point cloud is determined using the first variable, the third variable, and the trajectory segment length.

[0110] Specifically, the target point cloud is calculated using the following formula:

[0111]

[0112] Optionally, after obtaining the distance value, the method further includes steps C1-C2:

[0113] Step C1: If the distance value is less than or equal to the preset trajectory segment length, then update the first variable according to the third variable; and increment the second variable by one, and update the third variable according to the incremented second variable.

[0114] Specifically, such as Figure 7 As shown, if D PTIf the value is not greater than S, then update the first variable based on the third variable, i.e., determine the first variable PRE_PT = CUR_PT using the following formula; and calculate the accumulated second variable p_idx = p_idx + 1 using the following formula, and update the third variable CUR_PT = PT based on the accumulated second variable p_idx. p_idx .

[0115] Step C2: Continue to construct segmented lines based on the updated first and third variables until the accumulated second variable is greater than the number of poses in the pose sequence, and construct a segmented line set based on all the segmented lines.

[0116] In this embodiment, as Figure 7 As shown, after obtaining the updated first variable PRE_PT and the third variable CUR_PT, the segmented line segments are constructed based on the updated first variable PRE_PT and the third variable CUR_PT until the accumulated second variable is greater than the number of poses in the pose sequence.

[0117] By updating the first and third variables, the updated first and third variables can be used to construct segmented line segments, thereby achieving segmentation of the motion trajectory.

[0118] Step 6: Construct segmented line segments based on the first variable and the target point cloud.

[0119] Among them, the segmented line segment can be represented by SL.

[0120] In this embodiment, a segmented line segment SL = (PRE_PT, TMP_PT) is constructed based on the first variable PRE_PT and the target point cloud TMP_PT.

[0121] Step 7: Update the first variable according to the pose coordinates of the target point cloud; and increment the second variable by one, and update the third variable according to the incremented second variable.

[0122] Specifically, such as Figure 7 As shown, the first variable is updated based on the pose coordinates of the target point cloud, i.e., the first variable PRE_PT = TMP_PT is determined using the following formula; and the accumulated second variable p_idx = p_idx + 1 is calculated using the following formula, and the third variable CUR_PT = PT is updated based on the accumulated second variable p_idx. p_idx .

[0123] Step 8: Continue to construct segmented lines based on the updated first and third variables until the accumulated second variable is greater than the number of poses in the pose sequence, and construct a segmented line set based on all the segmented lines.

[0124] Furthermore, determine whether the accumulated second variable p_idx is greater than N. If yes, end the construction of the segmented line segment; otherwise, continue to execute steps 3-8 until the accumulated second variable is greater than the number of poses in the pose sequence.

[0125] S660. Using the segmented line segment set, the point cloud in the ground scene point cloud is divided into regions to obtain a segmented region set.

[0126] S670. Ground fitting is performed on the block region set based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0127] The technical solution of this invention obtains a point cloud sequence and a corresponding pose sequence during the autonomous vehicle's driving process. Based on the patchwork++ ground segmentation algorithm, it extracts ground point clouds from the point cloud sequence to obtain a ground scene point cloud. Then, it calculates the Euclidean distance between two adjacent poses in the pose sequence, obtains the distance value, and segments the motion trajectory according to the distance value and a preset trajectory segment length to obtain a set of segmented line segments. Using this set of segmented line segments, it divides the point cloud in the ground scene into regions, obtaining a set of segmented regions. Finally, it performs ground fitting on the set of segmented regions based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result. By implementing this technical solution and constructing a set of segmented line segments, it is possible to achieve ground segmentation and fitting of the scene, avoiding interference from ground point cloud noise and ground slope changes, and restoring the true ground conditions.

[0128] Example 3

[0129] Figure 8 This is a flowchart of the process for determining the set of segmented regions provided in Embodiment 3 of the present invention. The relationship between this embodiment and the above embodiments is a detailed description of the process for determining the set of segmented regions. Figure 8 As shown, the method includes:

[0130] S810: Obtain the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process.

[0131] S820. Based on the patchwork++ ground segmentation algorithm, the ground point cloud in the point cloud sequence is extracted to obtain the ground scene point cloud.

[0132] S830. The motion trajectory is segmented according to the pose sequence and the preset trajectory segment length to obtain a set of segmented line segments; wherein, the motion trajectory is obtained by splicing the pose sequence.

[0133] S840. Calculate the Euclidean distance between each point cloud in the ground scene point cloud and each segmented line segment; determine the minimum Euclidean distance for each point cloud, and determine the segmented line segment associated with the minimum Euclidean distance.

[0134] S850. Based on the segmented line segments associated with the minimum Euclidean distance, determine a set of segmented regions from a preset set of regions to be stored, and store the point cloud coordinates into the set of segmented regions. Specifically, the set of segmented regions can be determined using the following steps 11-18:

[0135] Step 11: Determine the fourth variable; the fourth variable is used to characterize the point cloud coordinate index in the ground scene point cloud; the initial value of the fourth variable is 1.

[0136] The fourth variable can be represented by pt_idx, and the initial value of pt_idx is 1.

[0137] Step 12: Determine the fifth variable; the fifth variable is used to characterize the segment index in the segmented line segment set; the initial value of the fifth variable is 1.

[0138] In this embodiment, the fifth variable can be represented by sl_idx, and the initial value of sl_idx is 1.

[0139] In this plan, Figure 9 This is a flowchart of another process for determining a set of segmented regions provided in Embodiment 3 of this application, where the set of segmented lines {SL1,SL2,…,SL3} is defined. M The starting position of the trajectory is PT. b The endpoint of the trajectory is PT. e and ground scene point cloud F PC Define the ground scene point cloud F PC The number of points in the midpoint cloud is K, F PC The coordinates of a point p in the equation are denoted as p = (x p ,y p ,z p ), taking the x and y coordinates to obtain PT. Where PT = (x... q ,y q That is, the PT sequence {PT1,PT2,…,PT} can be obtained. K}.like Figure 9 As shown, a variable pt_idx is defined as the point cloud coordinate index in a ground scene point cloud, and a variable sl_idx is defined as the segment index in a set of segmented line segments.

[0140] Step 13: Calculate the Euclidean distance between the point cloud coordinates corresponding to the fourth variable and the segmented line segment corresponding to the fifth variable to obtain the target distance value.

[0141] The target distance value can be represented by D. PL express.

[0142] Specifically, such as Figure 9 As shown, the segmented line SL corresponding to the fifth variable sl_idx =(PT) a ,PT b ), where PT a For segmented line segment SL sl_idx Starting position, PT a =(x a ,y a );PT b For segmented line segment SL sl_idx Termination position, PT b =(x b ,y b ).

[0143] Furthermore, for the point cloud coordinates PT corresponding to the fourth variable... pt_idx = (x1, y1) and the segment SL corresponding to the fifth variable sl_idx =(PT) a ,PT b The Euclidean distance between the two points can be calculated using the following formula:

[0144] D pa =D PT (PT ptidx ,PT a );

[0145] D pb =D PT (PT ptidx ,PT b );

[0146] D ab =D PT (PT a ,PT b );

[0147]

[0148]

[0149]

[0150] Step 14: Construct a distance array based on the target distance value and the fifth variable.

[0151] The distance array can be represented by (D) PL,sl_idx) indicates that a distance set D can be constructed based on multiple distance arrays. set .

[0152] In this plan, such as Figure 9 As shown, the target distance value and the fifth variable can be associated and stored in a distance set. That is, the fifth variable associated with the target distance value can be found in the distance set.

[0153] Step 15: Increment the fifth variable by one until the incremented fifth variable is greater than the number of segments in the segmented line segment set, and construct a distance set based on all distance arrays.

[0154] The fifth variable, sl_idx, after accumulation can be calculated using the following formula: sl_idx = sl_idx + 1.

[0155] In this embodiment, as Figure 9 As shown, after obtaining the fifth variable sl_idx after accumulation, it is determined whether the fifth variable sl_idx after accumulation is greater than M. If yes, then steps 16-17 are executed; otherwise, step 18 is executed.

[0156] Step 16: Determine the minimum target distance value from the distance set, and determine the fifth variable associated with the minimum target distance value.

[0157] Specifically, such as Figure 9 As shown, from the distance set D set Find the smallest D in PL And determine the minimum D PL The associated sl_idx.

[0158] Step 17: Determine a set of block regions from a preset set of block regions to be stored based on the fifth variable associated with the minimum target distance value, and store the point cloud coordinates corresponding to the fourth variable into the set of block regions.

[0159] Among them, the preset set of block regions to be stored is {SEG1, SEG2, ..., SEG}. L If the content of} is empty, the point cloud coordinates corresponding to the fourth variable can be stored in the block region set.

[0160] Furthermore, such as Figure 9 As shown, in determining the minimum D PL After associating sl_idx, sl_idx is denoted as S_idx. The point cloud coordinates PT corresponding to the fourth variable are then set. pt_idx Store to SL S_idx The corresponding set of partitioned regions {SEG1, SEG2, ..., SEG} L In}, and clear the distance set D.set .

[0161] Specifically, based on the smallest D PL Search and SL S_idx The corresponding set of partitioned regions {SEG1, SEG2, ..., SEG} L Furthermore, it is possible to search for the region matching SL from the set of regions to be stored. S_idx The distance between them is equal to the minimum D PL A set of partitioned regions.

[0162] Step 18: If the accumulated fifth variable is less than or equal to the number of segmented lines in the segmented line segment set, then construct a distance array based on the point cloud coordinates corresponding to the fourth variable and the segmented lines corresponding to the accumulated fifth variable, until the accumulated fifth variable is greater than the number of segmented lines in the segmented line segment set.

[0163] Specifically, if the fifth variable sl_idx after accumulation is not greater than M, continue to execute steps 12-15 until the fifth variable sl_idx after accumulation is greater than M.

[0164] In this embodiment, Figure 10 This is a schematic diagram of constructing a set of segmented regions provided in Embodiment 3 of this application. Figure 10 (a) is a schematic diagram of the block region set in a straight road scene. Figure 10 (b) is a schematic diagram of the segmented regions in a straight-line scene. For example... Figure 10 As shown in (a) and (b), the point cloud in the ground scene can be divided into regions using the segmented line segment set to obtain the segmented region set of the straight road scene and the curved road scene.

[0165] S860. Ground fitting is performed on the block region set based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0166] The technical solution of this invention involves acquiring a point cloud sequence and a corresponding pose sequence during the driving process of an unmanned vehicle. Ground point clouds are extracted from the point cloud sequence using the patchwork++ ground segmentation algorithm to obtain a ground scene point cloud. Then, the motion trajectory is segmented according to the pose sequence and a preset trajectory segment length to obtain a set of segmented lines. The Euclidean distance between each point cloud in the ground scene point cloud and each of the segmented lines is calculated. The minimum Euclidean distance for each point cloud is determined, and the segmented line associated with the minimum Euclidean distance is identified. Based on the segmented line associated with the minimum Euclidean distance, a set of block regions is determined from a preset set of regions to be stored, and the point cloud coordinates are stored in the block region set. Finally, ground fitting is performed on the block region set using the RANSAC plane fitting method to obtain the ground point cloud segmentation result. By implementing this technical solution, by constructing a set of segmented regions, ground fitting is performed on the segmented point cloud to filter out abnormal point clouds, thus achieving the effect of ground segmentation and fitting of the scene. This can avoid interference from point cloud noise on the ground and changes in ground slope, and restore the real ground situation.

[0167] In this embodiment, Figure 11 This is a 3D lane line annotation result diagram provided in the embodiments of this application, such as... Figure 11 As shown, the ground point cloud segmentation effect can be referenced in the 3D lane line annotation application. After filtering out the abnormal points of the ground segmentation of the point cloud sequence, the point cloud information of vehicle obstacles is filtered out in the ground point cloud segmentation result. Furthermore, the lane lines annotated on the ground point cloud segmentation result can be correctly reprojected into the image taken by the camera installed around the vehicle, achieving efficient annotation.

[0168] Example 4

[0169] Figure 12 This is a schematic diagram of a point cloud sequence ground segmentation device provided in Embodiment 4 of the present invention. Figure 12 As shown, the device includes:

[0170] The point cloud sequence acquisition module 1210 is used to acquire the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the driving process of the unmanned vehicle.

[0171] The ground scene point cloud acquisition module 1220 is used to extract ground point clouds from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain ground scene point clouds.

[0172] The segmented line segment set acquisition module 1230 is used to segment the motion trajectory according to the pose sequence and the preset trajectory segment length to obtain a segmented line segment set; wherein, the motion trajectory is obtained by splicing the pose sequence;

[0173] The block region set acquisition module 1240 is used to divide the point cloud in the ground scene point cloud into regions using the segmented line segment set to obtain the block region set.

[0174] The ground point cloud segmentation result acquisition module 1250 is used to perform ground fitting on the block region set based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0175] Optionally, the segmented line segment set yields module 1230, which is specifically used for:

[0176] Calculate the Euclidean distance between two adjacent poses in the pose sequence sequentially to obtain the distance value;

[0177] If the distance value is less than or equal to the preset trajectory segment length, then the segment set is determined based on two adjacent poses in the pose sequence.

[0178] Optionally, the segmented line segment set obtained by module 1230 is also used for:

[0179] If the distance value is greater than the preset trajectory segment length, then based on the two adjacent poses in the pose sequence and the trajectory segment length, the target pose is determined from the two adjacent poses, and based on the previous pose and the target pose among the two adjacent poses, the segmented line segment set is determined.

[0180] Optionally, the set of segmented regions yields module 1240, which is specifically used for:

[0181] Calculate the Euclidean distance between each point cloud in the ground scene point cloud and each segmented line segment; determine the minimum Euclidean distance for each point cloud, and determine the segmented line segment associated with the minimum Euclidean distance;

[0182] Based on the segmented line segments associated with the minimum Euclidean distance, a set of block regions is determined from a preset set of block regions to be stored, and the point cloud coordinates are stored in the set of block regions.

[0183] Optionally, the block region set obtained by module 1240 is also used for:

[0184] Based on the minimum Euclidean distance, a search is performed from the preset set of block regions to be stored to obtain the set of block regions corresponding to the segmented line segment; wherein, the distance between the segmented line segment and the set of block regions is equal to the minimum Euclidean distance.

[0185] Optionally, the ground point cloud segmentation result is obtained from module 1250, which is specifically used for:

[0186] The point cloud in the block region set is successively fitted to the ground using the RANSAC plane fitting method, and the remaining point cloud after fitting is stored in the point cloud set.

[0187] After performing ground fitting on all the segmented region sets, the point cloud in the point cloud set is then fitted with ground based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

[0188] The point cloud sequence ground segmentation device provided in this embodiment of the invention can execute the point cloud sequence ground segmentation method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0189] Example 5

[0190] Figure 13 A schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0191] like Figure 13 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0192] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0193] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a point cloud sequence ground segmentation method.

[0194] In some embodiments, a point cloud sequence ground segmentation method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or mounted on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the point cloud sequence ground segmentation method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a point cloud sequence ground segmentation method by any other suitable means (e.g., by means of firmware).

[0195] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0196] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0197] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0198] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0199] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0200] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0201] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0202] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for ground segmentation of a point cloud sequence, characterized in that, include: Obtain the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process; The ground point cloud is extracted from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain the ground scene point cloud; The motion trajectory is segmented according to the pose sequence and the preset trajectory segment length to obtain a set of segmented line segments; wherein, the motion trajectory is obtained by splicing the pose sequence; Using the segmented line segment set, the point cloud in the ground scene point cloud is divided into regions to obtain a set of segmented regions; The ground point cloud segmentation result is obtained by performing ground fitting on the block region set based on the RANSAC plane fitting method. Specifically, the motion trajectory is segmented according to the pose sequence and the preset trajectory segment length to obtain a set of segmented line segments, including: Calculate the Euclidean distance between two adjacent poses in the pose sequence sequentially to obtain the distance value; If the distance value is less than or equal to the preset trajectory segment length, then the segment set is determined based on two adjacent poses in the pose sequence; If the distance value is greater than the preset trajectory segment length, then based on the two adjacent poses in the pose sequence and the trajectory segment length, the target pose is determined from the two adjacent poses, and based on the previous pose and the target pose, the segmented line segment set is determined. Specifically, the point cloud in the ground scene is divided into regions using the segmented line segment set to obtain a set of segmented regions, including: Calculate the Euclidean distance between each point cloud in the ground scene point cloud and each segmented line segment; determine the minimum Euclidean distance for each point cloud, and determine the segmented line segment associated with the minimum Euclidean distance; Based on the segmented line segments associated with the minimum Euclidean distance, a set of block regions is determined from a preset set of block regions to be stored, and the point cloud coordinates are stored in the set of block regions.

2. The method according to claim 1, characterized in that, Based on the segmented line segments associated with the minimum Euclidean distance, a set of block regions is determined from a preset set of regions to be stored, including: Based on the minimum Euclidean distance, a search is performed from the preset set of block regions to be stored to obtain the set of block regions corresponding to the segmented line segment; wherein, the distance between the segmented line segment and the set of block regions is equal to the minimum Euclidean distance.

3. The method according to claim 1, characterized in that, The ground is fitted to the block region set using the RANSAC plane fitting method to obtain the ground point cloud segmentation results, including: The point cloud in the block region set is successively fitted to the ground using the RANSAC plane fitting method, and the remaining point cloud after fitting is stored in the point cloud set. After performing ground fitting on all the segmented region sets, the point cloud in the point cloud set is then fitted with ground based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result.

4. A point cloud sequence ground segmentation device, characterized in that, include: The point cloud sequence acquisition module is used to acquire the point cloud sequence and the pose sequence corresponding to the point cloud sequence during the autonomous vehicle's driving process. The ground scene point cloud acquisition module is used to extract ground point clouds from the point cloud sequence based on the patchwork++ ground segmentation algorithm to obtain ground scene point clouds; The segmented line segment set acquisition module is used to segment the motion trajectory according to the pose sequence and the preset trajectory segment length to obtain a segmented line segment set; wherein, the motion trajectory is obtained by splicing the pose sequence; The module for obtaining the block region set is used to divide the point cloud in the ground scene point cloud into regions using the segmented line segment set, thereby obtaining the block region set. The ground point cloud segmentation result acquisition module is used to perform ground fitting on the block region set based on the RANSAC plane fitting method to obtain the ground point cloud segmentation result. The module obtained from the segmented line segment set is specifically used for: Calculate the Euclidean distance between two adjacent poses in the pose sequence sequentially to obtain the distance value; If the distance value is less than or equal to the preset trajectory segment length, then the segment set is determined based on two adjacent poses in the pose sequence; If the distance value is greater than the preset trajectory segment length, then based on the two adjacent poses in the pose sequence and the trajectory segment length, the target pose is determined from the two adjacent poses, and based on the previous pose and the target pose, the segmented line segment set is determined. The module obtained by dividing the region into blocks is specifically used for: Calculate the Euclidean distance between each point cloud in the ground scene point cloud and each segmented line segment; determine the minimum Euclidean distance for each point cloud, and determine the segmented line segment associated with the minimum Euclidean distance; Based on the segmented line segments associated with the minimum Euclidean distance, a set of block regions is determined from a preset set of block regions to be stored, and the point cloud coordinates are stored in the set of block regions.

5. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform a point cloud sequence ground segmentation method according to any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute a point cloud sequence ground segmentation method according to any one of claims 1-3.

Images