Obstacle segmentation methods, devices, electronic equipment and storage media

Abstract

This invention provides an obstacle segmentation method, apparatus, electronic device, and storage medium, relating to the field of target recognition technology. The method includes: acquiring translational and rotational extrinsic parameters of multiple sensing devices in a device coordinate system, wherein the multiple sensing devices are positioned on a device under test corresponding to the device coordinate system; the translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system; calculating a global point cloud based on the translational and rotational extrinsic parameters, wherein the global point cloud is a stitched result of the point clouds of multiple sensing devices in the device coordinate system; performing ground segmentation on the global point cloud, removing the ground point cloud to obtain an obstacle point cloud; projecting the obstacle point cloud into a two-dimensional grid map, and determining the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map. This application can meet the stable perception requirements of autonomous driving systems in complex environments.

Description

Technical Field

[0001] This invention relates to the field of target recognition technology, and more specifically to an obstacle segmentation method, apparatus, electronic device, and storage medium. Background Technology

[0002] In the field of target recognition and perception, sensing devices, such as lidar, have become core sensors due to their high-precision 3D modeling capabilities, while point cloud obstacle segmentation, as a key link in environmental understanding, directly determines the safety of system decision-making based on its reliability.

[0003] However, existing point cloud obstacle segmentation methods still have significant shortcomings: On the one hand, the perception range of a single LiDAR is limited, and to achieve more comprehensive environmental perception, it is often necessary to stitch together the perception fields of multiple radars. Traditional multi-radar calibration relies on extrinsic parameter registration using calibration objects, such as checkerboard patterns or corner reflectors. Radars detect these calibration objects to obtain corresponding measurement data, which is then used to calculate the extrinsic parameters between the radars. Traditional methods are complex and have limited accuracy, easily leading to problems such as ghosting and misalignment in point cloud stitching, causing deviations between the global point cloud input to the segmentation model and the real environment, thus affecting the accuracy of the segmentation results from the source. On the other hand, existing segmentation methods themselves have obvious limitations, typically facing bottlenecks such as high computational complexity and large processing latency. For example, the difficulty in extracting unstructured features due to the sparsity of point clouds, or the random computational overhead of iterative segmentation algorithms, makes it difficult for traditional segmentation methods to meet real-time requirements in complex urban scenarios. Especially in high-speed driving scenarios, delayed obstacle detection can pose serious safety hazards.

[0004] Therefore, the input data deviation caused by the above-mentioned inaccurate calibration, along with the problems of insufficient real-time performance, poor adaptability, and reliance on a large amount of pre-trained data in the segmentation method, result in the low reliability of existing point cloud obstacle segmentation methods, making it difficult to meet the stable perception requirements of autonomous driving systems for complex environments. Summary of the Invention

[0005] The purpose of this application is to provide an obstacle segmentation method, apparatus, electronic device, and storage medium to solve the problem that existing point cloud obstacle segmentation methods are unable to meet the stable perception requirements of autonomous driving systems in complex environments.

[0006] In a first aspect, embodiments of this application provide a method comprising: The translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system are obtained respectively. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. Calculations are performed based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, which is the stitching result of the point clouds of the multiple sensing devices in the device coordinate system. The global point cloud is segmented into ground layers, and the ground point cloud is removed to obtain the obstacle point cloud; The obstacle point cloud is projected into a two-dimensional grid map, and the obstacles corresponding to the obstacle point cloud are determined based on the two-dimensional grid map.

[0007] Optionally, multiple translational extrinsic parameters of the sensing devices in the device coordinate system are obtained, including: The coordinates of multiple sensing devices in the device coordinate system are determined as spatial position parameters, and the translation extrinsic parameters are determined based on the spatial position parameters. The device coordinate system is constructed with a preset feature point on the device under test as the origin, the x-axis direction parallel to the travel direction of the device under test, the y-axis direction parallel to the lateral direction of the device under test, and the z-axis direction perpendicular to the ground. The preset feature point is determined based on the geometric center of the lateral section of the device under test, and the lateral section is parallel to the travel direction.

[0008] Optionally, the attitude tilt angle parameters include a first sub-attitude tilt angle parameter and a second attitude tilt angle parameter; obtaining the rotational extrinsic parameters of multiple sensing devices in the device coordinate system includes: For any one of the plurality of sensing devices, the following steps are performed respectively: In the device coordinate system, the first sub-attitude tilt angle parameter is calculated based on the preset ground reference normal vector and the normal vector of the ground point cloud collected by any of the sensing devices. The first rotation quaternion of any of the sensing devices is determined based on the attitude tilt angle parameter. The first sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the x-axis of the device coordinate system. In the device coordinate system, a second sub-attitude tilt angle parameter is calculated based on a preset ground reference normal vector and the normal vector of the vertical plane collected by any of the sensing devices. The second rotation quaternion of any of the sensing devices is determined based on the attitude tilt angle parameter. The second sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the y-axis of the device coordinate system. The vertical plane is a vertical plane parallel to the y-axis in the device coordinate system. The product of the first rotational quaternion and the second rotational quaternion is determined as the rotational extrinsic parameter of any of the sensing devices.

[0009] Optionally, the calculation based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain the global point cloud includes: For the point cloud of the sensing device collected by any of the plurality of sensing devices, the point cloud of the sensing device is rotated from the sensing device coordinate system of any of the sensing devices to the device coordinate system based on the rotation extrinsic parameter of any of the sensing devices. Based on the translation extrinsic parameters of any sensing device, a translation transformation is performed on the rotated point cloud of the sensing device. The point clouds obtained by merging the point clouds from the multiple sensing devices after rotation and translation transformation are obtained to obtain the global point cloud.

[0010] Optionally, the step of performing ground segmentation on the global point cloud and removing the ground point cloud to obtain the obstacle point cloud includes: The global point cloud is converted into a polar coordinate system, and based on a preset angle and a preset radius, the polar coordinate system is divided into concentric rings to form multiple sector grids; Based on the reference ground normal vector within each of the plurality of sector grids, if the point cloud type in each sector grid is determined to be a ground point cloud, the point cloud is removed, and the remaining point cloud is determined to be the obstacle point cloud.

[0011] Optionally, the step of removing the point cloud and determining the remaining point cloud as the obstacle point cloud based on the reference ground normal vector within each of the plurality of sector grids, when the point cloud type in each sector grid is determined to be a ground point cloud, includes: Obtain the reference ground normal vector within each of the plurality of sector grids; For any point cloud within each of the said sector grids, calculate the local normal vector of any point cloud, which is obtained based on a preset number of neighboring point clouds of any point cloud; If the angle between the local normal vector and the reference ground normal vector of the sector grid is less than or equal to a preset angle threshold, the point cloud is determined to be a ground point cloud and removed, and the sector grid corresponds to the local normal vector; Traverse the point clouds within all sector grids and identify the remaining point clouds as the obstacle point clouds.

[0012] Optionally, calculating the local normal vector of any point cloud within each of the said sector grids includes: Obtain a preset number of neighboring point clouds for any given point cloud; Calculate the centroid of the nearest neighbor point cloud; Based on the deviation of the nearest point cloud relative to the centroid, a covariance matrix is ​​constructed; Solve for multiple sets of eigenvalues ​​and eigenvectors of the covariance matrix, and determine the eigenvector corresponding to the smallest eigenvalue as the local normal vector.

[0013] Optionally, projecting the obstacle point cloud into a two-dimensional grid map includes: Extract the extreme values ​​of the obstacle point cloud in the plane dimension corresponding to the two-dimensional grid map, where the extreme values ​​include the maximum and minimum coordinates in the plane dimension; The size of the two-dimensional raster map is determined based on the difference between the coordinate extreme values ​​and the preset raster resolution. For each point in the obstacle point cloud, the grid coordinates of each point in the two-dimensional grid map are calculated based on the coordinate values ​​of each point in the plane dimension, the extreme values ​​of the coordinates, and the preset grid resolution. Based on the stated dimensions, the grid coordinates of all points are integrated to form the two-dimensional grid map.

[0014] Optionally, determining the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map includes: At least one set of obstacle points in the two-dimensional grid map is obtained through region segmentation processing; Principal component analysis is performed on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points.

[0015] Optionally, at least one set of obstacle points in the two-dimensional grid map is obtained through region segmentation processing, including: The two-dimensional grid map is subjected to connected component analysis to obtain at least one connected component, and each connected component corresponds to a set of obstacle points.

[0016] Optionally, principal component analysis is performed on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points, including: For any set of points in each set of obstacle points, principal component analysis is performed on the set of points based on the mean and covariance matrix of the set of points to obtain the eigenvectors. By inverse transformation of the transition matrix, the set of any point is mapped to a local coordinate system to obtain the target set of points. The local coordinates are constructed based on the feature vectors, and the transition matrix is ​​determined based on the mean and the covariance matrix. Based on the target point set, determine the obstacle bounding box of any point set, and the obstacle bounding box corresponds to an obstacle.

[0017] Optionally, determining the obstacle bounding box of any point set based on the target point set includes: Extract the extreme values ​​of the target point set on each coordinate axis of the local coordinate system, wherein the extreme values ​​include the maximum and minimum coordinate values ​​in each coordinate axis direction; Based on the difference between the extreme values ​​of the coordinates and the preset resolution of the two-dimensional grid map, the length and width of the obstacle bounding box along each coordinate axis of the local coordinate system are calculated; Based on the mean of the set of points and the transition matrix, determine the center coordinates of the obstacle bounding box in the coordinate system of the two-dimensional grid map; Based on the rotation component of the transfer matrix, the rotation angle of the obstacle bounding box relative to the coordinate system of the two-dimensional grid map is determined; By integrating the length, width, center coordinates, and rotation angle, an obstacle bounding box is formed for any set of points.

[0018] Secondly, embodiments of this application also provide an obstacle segmentation device, the device comprising: The acquisition module is used to acquire translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. The calculation module is used to perform calculations based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, which is the stitching result of the point clouds of the multiple sensing devices in the device coordinate system; The segmentation module is used to perform ground segmentation on the global point cloud and remove the ground point cloud to obtain the obstacle point cloud; The projection module is used to project the obstacle point cloud into a two-dimensional grid map, and to determine the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map.

[0019] Thirdly, embodiments of this application also provide an electronic device, the electronic device comprising: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the obstacle segmentation method as described in any one of the first aspects.

[0020] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer program instructions, which, when executed by a processor, implement the obstacle segmentation method as described in any one of the first aspects.

[0021] In the above technical solution, point clouds are first stitched together using the pre-calibrated position and attitude parameters of each sensing device in the device coordinate system to generate a global point cloud. This method is simple to operate and avoids the cumbersome operation and accumulated errors caused by traditional methods that rely on calibration objects for extrinsic parameter estimation. It significantly improves the accuracy and efficiency of point cloud stitching and ensures the quality of input data from the source. Furthermore, by performing ground segmentation and removal on the global point cloud and projecting the remaining obstacle point cloud onto a two-dimensional grid map for recognition, the computational complexity and processing latency of the algorithm are greatly reduced, enhancing the adaptability to complex scenes. It does not require a large amount of pre-training data and effectively improves the real-time performance of obstacle detection and the reliability and safety of the entire autonomous driving system's environmental perception.

[0022] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0023] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 The illustration shows a flowchart of an obstacle segmentation method according to an embodiment of this application; Figure 2 The illustration shows a schematic diagram of the installation of a sensing device according to an embodiment of the present application on a device under test; Figure 3 A schematic diagram of a ground reference according to an embodiment of this application is shown; Figure 4 This illustration schematically shows a calibration plate placement diagram according to an embodiment of the present application; Figure 5(a) schematically shows a top view of a stitched point cloud without rotation correction according to an embodiment of the present application; Figure 5(b) schematically shows a side view of a stitched point cloud without rotation correction according to an embodiment of the present application; Figure 6(a) schematically shows a top view of a rotationally corrected stitched point cloud according to an embodiment of the present application; Figure 6(b) schematically shows a side view of a rotationally corrected stitched point cloud according to an embodiment of the present application; Figure 7 This illustration schematically shows a point cloud mesh partitioning diagram according to an embodiment of this application; Figure 8(a) schematically shows the effect of point cloud projection before projection according to an embodiment of this application; Figure 8(b) schematically shows the effect of point cloud projection according to an embodiment of this application; Figure 9A schematic diagram of point cloud segmentation according to an embodiment of this application is shown; Figure 10 A schematic diagram of an obstacle enclosure box according to an embodiment of this application is shown; Figure 11 A schematic diagram of an obstacle segmentation device according to an embodiment of this application is shown. Figure 12 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for illustration and explanation of the embodiments of this application and are not intended to limit the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0025] It should be noted that the acquisition, transmission, storage, use, and processing of data in the technical solution of this application all comply with relevant laws and regulations. In the embodiments of this application, certain existing industry solutions such as software, components, and models may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solution of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0026] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0027] Figure 1 The illustration schematically shows a flowchart of an obstacle segmentation method according to an embodiment of this application. Figure 1 As shown in the figure, this application provides an obstacle segmentation method, which includes: Step 101: Obtain translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. In this embodiment, the sensing device refers to any hardware device, hardware combination, or system deployed on or interacting with the device under test, capable of actively or passively acquiring spatial information of the environment surrounding the device under test and generating or providing point cloud data. Its core function is to provide raw three-dimensional spatial data input for the obstacle segmentation method. The sensing device can be a sensor mounted on the device under test itself, a roadside device, or other data source modules capable of providing point cloud data streams.

[0028] In one embodiment, the sensing device includes, but is not limited to, the following types of sensors: lidar, millimeter-wave radar, depth camera, and ultrasonic radar.

[0029] In another embodiment, the sensing device can be a multi-sensor fusion sensing system. For example, a combination system of LiDAR and vision cameras integrated in the same module or different locations in the vehicle.

[0030] For ease of explanation, in the following embodiments, the sensing device will be described as a radar. The device under test can be a vehicle, a working device, a working unit of a device, etc. In the following embodiments, the device under test will be described as a vehicle, and the device coordinate system will be described as the vehicle coordinate system.

[0031] The radar types can include lidar, millimeter-wave radar, infrared radar, or ultrasonic radar, and there can be multiple radars. The specific number can be adjusted according to the radar's field of view so that the field of view of multiple radars can cover the entire perimeter of the vehicle. Figure 2 As shown, Figure 2 This is a schematic diagram of radar installation on a vehicle. A, B, C, and D are the midpoints of the vehicle's tires. Taking four radars as an example, E, F, G, and H are the center points of the four radars. The blue square area represents the vehicle's lateral cross-section.

[0032] The vehicle coordinate system is constructed with a preset feature point on the vehicle as the origin, the x-axis parallel to the vehicle's driving direction, the y-axis parallel to the vehicle's lateral direction, and the z-axis perpendicular to the ground. The preset feature point is determined based on the geometric center of the vehicle's lateral cross-section, which is parallel to the driving direction. Figure 2In this configuration, the geometric center of the transverse section is the intersection of AC and BD, which is also the origin of the vehicle coordinate system. The X-axis points in the direction of vehicle movement, the Y-axis points to the left or right side of the vehicle, and the Z-axis is perpendicular to the ground and pointing upwards. Alternatively, this preset feature point can also be the vehicle's center of mass or the rear axle center, etc., but this embodiment does not impose any limitations.

[0033] Translational extrinsic parameters can be directly determined by obtaining the three-dimensional coordinates of each radar installation center in the vehicle coordinate system using a high-precision radar such as a total station. Rotational extrinsic parameters are determined based on the attitude deviation of the radar relative to the vehicle coordinate system during actual installation. For example, they can be determined by measuring the rotation angles around the X, Y, and Z axes using a level and calibration plate, or expressed directly as quaternions or rotation matrices.

[0034] Step 102: Calculate based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, which is the stitched result of the point clouds of the multiple radars in the vehicle coordinate system; In this embodiment, after acquiring the extrinsic parameters of each radar, the original point clouds collected by each radar are unified into the vehicle coordinate system through coordinate transformation. Specifically, for the point cloud of a certain radar, a rotation transformation is first performed according to the rotation extrinsic parameters, and then a translation transformation is performed according to the translation extrinsic parameters, thereby transforming the radar point cloud into the vehicle coordinate system. The transformed point clouds of all radars are merged to obtain the global point cloud, completing the multi-radar point cloud stitching. This process avoids the inconvenience and errors of traditional methods that rely on calibration objects to fit extrinsic parameters, improving the accuracy of the data source.

[0035] Step 103: Perform ground segmentation on the global point cloud and remove the ground point cloud to obtain the obstacle point cloud; The stitched global point cloud is segmented to distinguish between ground points and non-ground (obstacle) points. In one optional implementation, the point cloud is first transformed into a polar coordinate system centered on the vehicle, and then divided into fan-shaped grid regions according to a certain angle and radial distance. For each grid, a reference ground normal vector is extracted. Then, for each point within the grid, a local normal vector is calculated based on its nearest neighbors, and whether it is a ground point is determined by comparing the angle between the local normal vector and the reference ground normal vector. If the angle is less than a set threshold, it is determined to be a ground point and is removed. After traversing all point clouds, the remaining point cloud is the obstacle point cloud.

[0036] Step 104: Project the obstacle point cloud into a two-dimensional grid map, and determine the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map.

[0037] The obstacle point cloud is projected onto a 2D grid map, essentially flattening the 3D point cloud onto a horizontal plane while ignoring height information. The grid map size and resolution are determined based on the distribution of the point cloud in the XY plane, and each point is placed into its corresponding grid cell according to its X and Y coordinates. Based on this, image processing methods (such as connected component labeling) can be used to analyze the grid map, extracting the set of grid cells belonging to the same obstacle. Then, the position, size, orientation, and other parameters of each obstacle can be calculated to complete obstacle identification.

[0038] In this embodiment, point cloud stitching is first performed directly using the pre-calibrated position and attitude parameters of each radar in the vehicle coordinate system to generate a global point cloud. This method is simple to operate and avoids the cumbersome operation and accumulated errors caused by traditional methods that rely on calibration objects for extrinsic parameter estimation. It significantly improves the accuracy and efficiency of point cloud stitching and ensures the quality of input data from the source. Furthermore, by performing ground segmentation and removal on the global point cloud and projecting the remaining obstacle point cloud onto a two-dimensional grid map for identification, the computational complexity and processing latency of the algorithm are greatly reduced, enhancing the adaptability to complex scenes. It does not require a large amount of pre-training data and effectively improves the real-time performance of obstacle detection and the reliability and safety of the entire autonomous driving system's environmental perception.

[0039] In one embodiment of this application, the translational extrinsic parameters of multiple radars in the vehicle coordinate system are obtained, including: The coordinates of multiple radars in the vehicle coordinate system are determined as spatial position parameters, and the translation extrinsic parameters are determined based on the spatial position parameters.

[0040] In this embodiment, the translation extrinsic parameter refers to the translation vector of each radar installation center point from its own coordinate system to the aforementioned vehicle coordinate system. The spatial position parameter is the three-dimensional coordinate value of the radar in the vehicle coordinate system, which can be directly used as the basis for determining the translation extrinsic parameter. In actual operation, a high-precision measuring instrument, such as a total station, can be used to obtain these coordinate values. Therefore, we can obtain... Figure 2 The radar coordinates at point E relative to the vehicle coordinate system {L} ( Similarly, we can obtain , , .

[0041] In this embodiment, the spatial position parameters of the radar are directly obtained by a precision radar such as a total station to determine the translation extrinsic parameters, which eliminates the need for complex calculation and fitting based on the calibration object, simplifies the operation process, and ensures the accuracy and reliability of the translation parameters.

[0042] In one embodiment of this application, the attitude tilt angle parameters include a first sub-attitude tilt angle parameter and a second attitude tilt angle parameter; obtaining multiple radar rotation extrinsic parameters in the vehicle coordinate system includes: For any one of the plurality of radars, perform the following steps respectively: In the vehicle coordinate system, the first sub-attitude tilt angle parameter is calculated based on the preset ground reference normal vector and the normal vector of the ground point cloud collected by any radar. The first rotation quaternion of any radar is determined based on the attitude tilt angle parameter. The first sub-attitude tilt angle parameter is used to characterize the rotation deviation of the radar around the x-axis of the vehicle coordinate system. In the vehicle coordinate system, a second sub-attitude tilt angle parameter is calculated based on a preset ground reference normal vector and the normal vector of the vertical plane acquired by any radar. The second rotation quaternion of any radar is determined based on the attitude tilt angle parameter. The second sub-attitude tilt angle parameter is used to characterize the rotational deviation of the radar around the y-axis of the vehicle coordinate system. The vertical plane is a vertical plane parallel to the y-axis in the vehicle coordinate system. The product of the first rotated quaternion and the second rotated quaternion is determined as the rotational extrinsic parameter of any radar.

[0043] In this embodiment, the calculation of the first rotation quaternion is specifically as follows: like Figure 3 As shown, Figure 3 This is a ground reference diagram. First, the control radar scans a flat ground area, collecting ground point cloud data. The normal vector n of this ground point cloud is calculated using a plane fitting algorithm. In the vehicle coordinate system, the ground reference normal vector is defined as n. base = [0, 0, 1] T (That is, perpendicular to the horizontal plane and pointing upwards). Next, calculate the measured normal vector n and the reference normal vector n. base The included angle θ1 between them. This included angle θ1 is the first sub-attitude tilt angle parameter, which mainly reflects the pitch angle deviation of the radar caused by installation, that is, the rotation deviation around the X-axis of the vehicle coordinate system.

[0044] Then, according to the axis-angle representation in three-dimensional space, the rotation axis μ can be obtained through the cross product of vectors: μ = n × n base Finally, based on the included angle θ1 and the rotation axis μ, the first rotation quaternion q1 is calculated: ; The function of this quaternion q1 is to correct the elevation angle deviation of the radar, so that the ground point cloud it scans is aligned with the actual horizontal plane.

[0045] The calculation of the second rotation quaternion is as follows: After the pitch angle is corrected, the radar's heading angle also needs to be calibrated. (See attached...) Figure 4 As shown, a level is used to project two mutually perpendicular laser lines onto the ground. One laser line is parallel to the front and rear tires. A calibration plate, a smooth rectangular thin plate, is placed perpendicularly on the other laser line. The control radar scans the calibration plate, collects its point cloud data, and calculates its normal vector m. The reference normal vector of this vertical plane should be parallel to the X-axis of the vehicle coordinate system, i.e., m... base = [1, 0, 0] T Calculate the measured normal vector m and the reference normal vector m. base The included angle θ2 between them.

[0046] 2 = arcos( ); This included angle θ2 is the second sub-attitude tilt angle parameter, which reflects the radar's heading angle installation deviation, i.e., the rotational deviation around the vehicle coordinate system Y-axis.

[0047] Similarly, the axis of rotation φ = m × m base Then, the second rotation quaternion q2 is calculated: ; The function of this quaternion q2 is to correct the radar's heading angle deviation.

[0048] Multiplying the quaternions obtained from the above two steps yields the final rotational extrinsic parameters of the radar: R E = q2 * q1. This synthesized rotational extrinsic parameter includes correction information for both pitch and yaw angles. Applying this extrinsic parameter to the radar point cloud enables a precise rotational transformation of the point cloud from the radar coordinate system to the vehicle coordinate system.

[0049] In this embodiment, the rotation parameters are calculated by directly measuring the geometric relationship between the point cloud normal vector and the physical reference, avoiding the accumulated errors of traditional registration algorithms and improving calibration accuracy. Using ground and vertical calibration plates as stable calibration references minimizes the impact of environmental factors, resulting in stable and reliable results.

[0050] In one embodiment of this application, the calculation based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain the global point cloud includes: For the radar point cloud collected by any one of the plurality of radars, the radar point cloud is rotated from the radar coordinate system of any one radar to the vehicle coordinate system based on the rotation extrinsic parameter of any one radar. Based on the translation extrinsic parameters of any radar, a translation transformation is performed on the rotated radar point cloud; The point clouds obtained by merging the point clouds of the multiple radars after rotation and translation transformation are obtained to obtain the global point cloud.

[0051] In this embodiment, for any radar (e.g., radar E) acquiring the original point cloud Q E (This point cloud data is defined in the radar E's own sensor coordinate system, i.e., in the radar coordinate system.) First, the radar's rotational extrinsic parameter R is applied. E Perform the transformation. Rotate the extrinsic parameter R. E The rotational transformation relationship from the radar coordinate system to the vehicle coordinate system is described.

[0052] The mathematical representation of rotation transformation is as follows: Q E_rotated =R E ·Q E The purpose of this step is to correct the angular deviation caused by the inconsistency between the radar's installation attitude (such as pitch, roll, and yaw angles) and the vehicle's coordinate system. After rotational transformation, the point cloud Q... E_rotated The direction is already aligned with the direction of the vehicle coordinate system.

[0053] After completing the orientation alignment, the next step is to perform position alignment. This involves aligning the point cloud Q after rotation transformation. E_rotated The translation extrinsic parameter t of this radar is applied. E Perform a translation transformation. The translation extrinsic parameter t E It is a three-dimensional vector representing the coordinates of the radar installation center in the vehicle coordinate system, that is, the translation from the origin of the vehicle coordinate system to the origin of the radar coordinate system.

[0054] The mathematical operation of translation transformation is represented as follows: Q E_vehicle =Q E_rotated +t E The purpose of this step is to translate the correctly oriented point cloud from the origin of the radar's own coordinate system to its actual installation position in the vehicle coordinate system. After this transformation, the point cloud Q... E_vehicle It is now completely within a unified vehicle coordinate system.

[0055] The above rotation and translation transformations are repeated for each radar in the system (e.g., radar E, F, G, H...), transforming the point clouds of all radars into the same vehicle coordinate system. Finally, these point cloud data in the unified coordinate system are merged, i.e.: =( + )+ ( + )+ ( + )+ ( + ) in, For global point cloud, For the rotating extrinsic parameters of radar E, For the original point cloud of radar E, For the translation extrinsic parameters of radar E, For the rotating extrinsic parameters of radar F, The original point cloud of radar F, For the translational extrinsic parameters of radar F, For the rotating extrinsic parameters of radar G, For the original point cloud of radar G, For the translational extrinsic parameters of radar G, For the rotating extrinsic parameters of radar H, For the original point cloud of radar H, Let H be the translational extrinsic parameter of radar H.

[0056] The merged global point cloud integrates the perception information from all radars, forming a complete and unified 3D representation of the environment. As shown in Figures 5 and 6, Figure 5 is a schematic diagram of the stitched point cloud without rotation correction; Figure 6 is a schematic diagram of the stitched point cloud after rotation correction. As can be seen from the figures, after correction, the point clouds from different radars can be precisely aligned on the same plane, eliminating the tilt and misalignment phenomena present in the uncorrected state.

[0057] In one embodiment of this application, the step of performing ground segmentation on the global point cloud and removing the ground point cloud to obtain the obstacle point cloud includes: The global point cloud is converted into a polar coordinate system, and based on a preset angle and a preset radius, the polar coordinate system is divided into concentric rings to form multiple sector grids; Based on the reference ground normal vector within each of the plurality of sector grids, if the point cloud type in each sector grid is determined to be a ground point cloud, the point cloud is removed, and the remaining point cloud is determined to be the obstacle point cloud.

[0058] In this embodiment, the stitched global point cloud is first transformed from a Cartesian coordinate system to a polar coordinate system. The polar coordinate system uses the vehicle's position as its origin, and the position of a point is represented by its radial distance ρ and azimuth angle θ. This transformation naturally adapts to the vehicle-centric environmental perception requirements in autonomous driving scenarios.

[0059] Next, based on preset angles (e.g., Δθ = 1°) and preset radii (e.g., Δρ = 1 meter), the polar coordinate system is divided into multiple sector-shaped grids. Δθ determines the granularity of the azimuth direction; a smaller value results in a finer division but also higher computational resource requirements. Δρ determines the granularity of the radial distance division, which can be set according to the principle of requiring high precision in the near field and appropriately reducing precision in the far field in practical applications. (See attached image) Figure 7As shown, the partitioning method is as follows: In the angular dimension, the range of 0-360° is uniformly divided into N = 360 / Δθ sectors; in the radial dimension, starting from the origin, multiple concentric rings are drawn outwards at intervals of ρ. The intersection of the angular and radial partitions forms multiple sector-shaped grid cells. Assuming the maximum radius range is L, the total number of grid cells N = (360 / Δθ) / (ρ ... )*(L / Δ The point clouds within each grid cell have similar distance and azimuth characteristics.

[0060] For each sector of the grid, ground point identification is performed. First, the reference ground normal vector for the current sector is obtained. Specifically, a plane fitting algorithm such as RANSAC (Random Sample Consensus) can be used to fit the ground plane to the point cloud within the grid, and the normal vector of the fitted plane is used as the reference ground normal vector ε. base .

[0061] Next, the type of each point cloud within the grid is determined. If a point is identified as a ground point cloud, it is removed from the point cloud data. This determination is made by comparing the similarity between the point's local normal vector and the grid's reference ground normal vector.

[0062] After traversing all sector grids and completing the judgment and removal of all ground points, the remaining point cloud is the obstacle point cloud.

[0063] In one embodiment, the step of removing the point cloud and determining the remaining point cloud as the obstacle point cloud based on the reference ground normal vector within each of the plurality of sector grids, when the point cloud type in each sector grid is determined to be a ground point cloud, includes: The reference ground normal vector within each of the multiple sector grids will be obtained; For any point cloud within each of the said sector grids, calculate the local normal vector of any point cloud, which is obtained based on a preset number of neighboring point clouds of any point cloud; If the angle between the local normal vector and the reference ground normal vector of the sector grid is less than or equal to a preset angle threshold, the point cloud is determined to be a ground point cloud and removed, and the sector grid corresponds to the local normal vector; Traverse the point clouds within all sector grids and identify the remaining point clouds as the obstacle point clouds.

[0064] In this embodiment, for each sector grid, its reference ground normal vector ε is first obtained. base In practice, the RANSAC algorithm can be used to fit the ground plane of the point cloud within the grid, and the fitted plane normal vector can be used as the reference ground normal vector of the grid.

[0065] In one embodiment, calculating the local normal vector of any point cloud within each of the said sector grids includes: Obtain a preset number of neighboring point clouds for any given point cloud; Calculate the centroid of the nearest neighbor point cloud; Based on the deviation of the nearest point cloud relative to the centroid, a covariance matrix is ​​constructed; Solve for multiple sets of eigenvalues ​​and eigenvectors of the covariance matrix, and determine the eigenvector corresponding to the smallest eigenvalue as the local normal vector.

[0066] In this embodiment, for any point cloud p within the grid i It is necessary to calculate its local normal vector α. The specific implementation method is as follows: With point p i Find a predetermined number (e.g., K=30) of nearest neighbor point clouds centered on a given point. Nearest neighbor search can be implemented using spatial data structures such as KD-tree.

[0067] Calculate the centroids of these nearest neighbor point clouds:

[0068] Where, x i The coordinates of a 3D point in the nearest point cloud.

[0069] Constructing the centroid-free matrix X C And calculate the covariance matrix C: X C =[x1- x2- ,x n - ];C= X C

[0070] Perform eigenvalue decomposition on the covariance matrix C to find its eigenvalues ​​and eigenvectors. The eigenvector corresponding to the smallest eigenvalue is then defined as the local normal vector α at that point.

[0071] Then, calculate the local normal vector α of the point and the reference ground normal vector ε of the corresponding sector grid. base The included angle δ between them: = arcos( ) The included angle δ is compared with a preset angle threshold K (e.g., 5°-15°): if δ ≤ K, the point is determined to be a ground point cloud and removed in subsequent processing; if δ > K, the point is determined to be a non-ground point (i.e., obstacle point cloud) and retained.

[0072] Repeat the above steps for all sector meshes to ensure that all point clouds are processed. After removing all point clouds identified as ground points, the remaining point clouds are determined as the final obstacle point clouds.

[0073] In this embodiment, the problem of sparse point clouds at a distance and dense point clouds at a distance in the Cartesian coordinate system is solved by using grid partitioning in the polar coordinate system, resulting in a relatively uniform point cloud distribution within each grid. Simultaneously, by independently calculating the reference ground normal vector for each grid, the system can adapt to terrain variations such as road slope and undulation, improving segmentation accuracy in complex scenes. Finally, the global segmentation problem is decomposed into local problems across multiple grids, facilitating parallel processing and reducing the number of point clouds processed per iteration, thus improving algorithm efficiency.

[0074] In one embodiment of this application, projecting the obstacle point cloud into a two-dimensional grid map includes: Extract the extreme values ​​of the obstacle point cloud in the plane dimension corresponding to the two-dimensional grid map, where the extreme values ​​include the maximum and minimum coordinates in the plane dimension; The size of the two-dimensional raster map is determined based on the difference between the coordinate extreme values ​​and the preset raster resolution. For each point in the obstacle point cloud, the grid coordinates of each point in the two-dimensional grid map are calculated based on the coordinate values ​​of each point in the plane dimension, the extreme values ​​of the coordinates, and the preset grid resolution. Based on the stated dimensions, the grid coordinates of all points are integrated to form the two-dimensional grid map.

[0075] In this embodiment, firstly, a statistical analysis is performed on the distribution range of the obstacle point cloud on the horizontal plane. Specifically, the extreme values ​​of the point cloud coordinates in the X and Y axes are extracted, including: Minimum value in the X direction X min and maximum value X max ; Minimum value in the Y direction Y min and maximum value Y max .

[0076] These extreme values ​​define the bounding box of the obstacle point cloud on the horizontal plane and are the basis for determining the extent of the raster map.

[0077] Secondly, based on the extracted coordinate extreme values, the span of the point cloud in the X and Y directions is calculated: Span in the X direction: ΔX = X max - X min Span in the Y direction: ΔY = Y max - Y min Based on the preset raster resolution cell, determine the size of the two-dimensional raster map (i.e., the number of rows and columns of the raster). Raster map column count (width): cols = ceil(ΔX / cell) Number of rows (height) in a raster map: rows = ceil(ΔY / cell) The ceil() function can be used to round up, ensuring that all point clouds are included in the raster map.

[0078] Then, for each point p in the obstacle point cloud i Its three-dimensional coordinates are (p i.x , p i.y , p i.z ), calculate its corresponding raster coordinates (u, v) in the two-dimensional raster map according to the following formula: u = floor((p i.x - X min ) / cell) v = floor((p i.y - Y min ) / cell) The `floor()` function can be used to round down the data. This calculation process essentially converts the absolute coordinates of a point into relative coordinates with respect to the map origin, and then discretizes them into raster indices based on the raster resolution.

[0079] Finally, based on the determined raster map size [rows × cols], a two-dimensional array of the corresponding size is created as the raster map. For each raster coordinate (u, v), if at least one point cloud is projected onto that raster, the raster is marked as "occupied"; otherwise, it is marked as "free". As shown in Figure 8, which is a schematic diagram of the effect before and after point cloud projection, after integrating the raster coordinates of all points, a complete two-dimensional raster map is formed.

[0080] In this embodiment, the map extent is determined by dynamically extracting coordinate extrema, which can adapt to changes in point cloud distribution under different scenarios and avoid the limitations of a fixed map extent. Reducing the 3D point cloud to a 2D raster representation significantly reduces the amount of data for subsequent processing, facilitating real-time obstacle segmentation.

[0081] In one embodiment of this application, determining the obstacle corresponding to the obstacle point cloud based on the two-dimensional grid map includes: At least one set of obstacle points in the two-dimensional grid map is obtained through region segmentation processing; Principal component analysis is performed on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points.

[0082] In one embodiment, obtaining at least one set of obstacle points in the two-dimensional grid map through region segmentation processing includes: The two-dimensional grid map is subjected to connected component analysis to obtain at least one connected component, and each connected component corresponds to a set of obstacle points.

[0083] In this embodiment, the two-dimensional grid map is first divided into regions. The purpose is to group the interconnected obstacle grids in the map into different obstacle point sets, each point set corresponding to an independent obstacle instance.

[0084] In practice, this process can be achieved through connected component analysis. Using a two-pass algorithm or a depth-first search-based connected component labeling algorithm, the entire raster map is traversed to divide adjacent "occupied" rasters into the same connected component.

[0085] like Figure 9 As shown, Figure 9 This is a schematic diagram of point cloud segmentation. After connected component analysis, each connected component represents an independent obstacle, and the grid coordinates contained within it constitute an obstacle point set V. For larger connected components, further segmentation can be performed according to actual needs to avoid multiple closely adjacent obstacles being incorrectly merged.

[0086] Then, after obtaining each obstacle point set, principal component analysis is performed on each obstacle point set to determine the specific obstacle parameters corresponding to each obstacle point set.

[0087] The specific implementation process includes: For any set of obstacle points V, first calculate its centroid, i.e., the mean point:

[0088] Among them, v i Let be the two-dimensional coordinates of each point in the point set, and n be the number of points in the point set.

[0089] Next, a decentralized data matrix V is constructed. C And calculate the covariance matrix Z: V C =[v1- v2- ,v n - ] Z= V C

[0090] The covariance matrix Z is decomposed into eigenvalues, yielding two eigenvalues ​​and their corresponding eigenvectors μ1 and μ2. The eigenvector μ1, with the larger eigenvalue, represents the primary direction of the obstacle, while the eigenvector μ2, with the smaller eigenvalue, represents the secondary direction.

[0091] Based on the results of principal component analysis, the following key parameters of the obstacle can be determined: the center position of the obstacle, i.e., the centroid; the principal direction of the obstacle, i.e., the direction of the principal eigenvector μ1; and the size information of the obstacle, which can be determined by the projection range of the point set onto the principal direction.

[0092] In this embodiment, through region partitioning, individual obstacles in the raster map can be automatically identified and separated without manual intervention. Furthermore, principal component analysis (PCA) can adapt to obstacles of various shapes, including pedestrians, vehicles, and trees. Finally, performing PCA in two-dimensional space requires minimal computation, meeting real-time requirements.

[0093] In one embodiment of this application, principal component analysis is performed on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points, including: For any set of points in each set of obstacle points, principal component analysis is performed on the set of points based on the mean and covariance matrix of the set of points to obtain the eigenvectors. By inverse transformation of the transition matrix, the set of any point is mapped to a local coordinate system to obtain the target set of points. The local coordinates are constructed based on the feature vectors, and the transition matrix is ​​determined based on the mean and the covariance matrix. Based on the target point set, determine the obstacle bounding box of any point set, and the obstacle bounding box corresponds to an obstacle.

[0094] In one embodiment, determining the obstacle bounding box of any point set based on the target point set includes: Extract the extreme values ​​of the target point set on each coordinate axis of the local coordinate system, wherein the extreme values ​​include the maximum and minimum coordinate values ​​in each coordinate axis direction; Based on the difference between the extreme values ​​of the coordinates and the preset resolution of the two-dimensional grid map, the length and width of the obstacle bounding box along each coordinate axis of the local coordinate system are calculated; Based on the mean of the set of points and the transition matrix, determine the center coordinates of the obstacle bounding box in the coordinate system of the two-dimensional grid map; Based on the rotation component of the transfer matrix, the rotation angle of the obstacle bounding box relative to the coordinate system of the two-dimensional grid map is determined; By integrating the length, width, center coordinates, and rotation angle, an obstacle bounding box is formed for any set of points.

[0095] In this embodiment, the process of obtaining the feature vector is described in the above embodiment and will not be repeated here.

[0096] After obtaining the eigenvectors, a transition matrix Tv is constructed based on the principal component analysis results: T v =[R v ,t v ] Where the rotation matrix R v =[μ1, μ2], composed of eigenvectors, with translation vector t. v = It is composed of the centroids of a point set.

[0097] By inverse transformation of the transition matrix, the original point set V is mapped to the local coordinate system: V'= V The essence of this transformation is to rotate the point set until its principal direction is aligned with the coordinate axes and translate it to the vicinity of the origin.

[0098] Finally, after obtaining the target point set V' in the local coordinate system, calculate the extreme values ​​of the coordinates of this point set along each coordinate axis: x-direction: min x = min(v'.x), max x = max(v'.x) y-direction: min y = min(v'.y), max y = max(v'.y) Key parameters of the obstacle bounding box are calculated based on these extreme values: Width: width = (max x - min x ) × cell Length: length = (max) y - min y ) × cell Rotation angle: q box = R v (i.e., the main direction) Center position: t box = cell × t v Where cell represents the resolution of the raster map, used to convert raster coordinates back to actual physical dimensions, such as... Figure 10 As shown, Figure 10 This is a schematic diagram of the obstacle bounding box.

[0099] In this embodiment, principal component analysis is used to accurately obtain the natural orientation of obstacles, avoiding directional errors caused by axial alignment of the bounding box. It can accurately handle obstacles of various shapes and orientations, including tilted parked vehicles and pedestrians. Finally, the calculations are performed in two-dimensional space, significantly reducing computational complexity compared to three-dimensional bounding box calculations.

[0100] Figure 11 A schematic diagram of an obstacle segmentation device provided in another embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0101] Reference Figure 11 The obstacle segmentation device 1100 may include: The acquisition module 1101 is used to acquire translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. Calculation module 1102 is used to perform calculations based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, wherein the global point cloud is the stitching result of the point clouds of the multiple sensing devices in the device coordinate system; The segmentation module 1103 is used to perform ground segmentation on the global point cloud and remove the ground point cloud to obtain the obstacle point cloud; The projection module 1104 is used to project the obstacle point cloud into a two-dimensional grid map, and to determine the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map.

[0102] Optionally, module 1101 is used specifically for: The coordinates of multiple sensing devices in the device coordinate system are determined as spatial position parameters, and the translation extrinsic parameters are determined based on the spatial position parameters. The device coordinate system is constructed with a preset feature point on the device under test as the origin, the x-axis direction parallel to the travel direction of the device under test, the y-axis direction parallel to the lateral direction of the device under test, and the z-axis direction perpendicular to the ground. The preset feature point is determined based on the geometric center of the lateral section of the device under test, and the lateral section is parallel to the travel direction.

[0103] Optionally, the acquisition module 1101 includes: For any one of the plurality of sensing devices, the following steps are performed respectively: The first calculation submodule is used to calculate the first sub-attitude tilt angle parameter in the device coordinate system based on the preset ground reference normal vector and the normal vector of the ground point cloud collected by any of the sensing devices, and to determine the first rotation quaternion of any of the sensing devices based on the attitude tilt angle parameter. The first sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the x-axis of the device coordinate system. The second calculation submodule is used to calculate the second sub-attitude tilt angle parameter in the device coordinate system based on the preset ground reference normal vector and the normal vector of the vertical plane collected by any of the sensing devices, and to determine the second rotation quaternion of any of the sensing devices based on the attitude tilt angle parameter. The second sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the y-axis of the device coordinate system. The vertical plane is a vertical plane parallel to the y-axis in the device coordinate system. The first determining submodule is used to determine the product of the first rotation quaternion and the second rotation quaternion as the rotational extrinsic parameter of any of the sensing devices.

[0104] Optionally, the computing module 1102 includes: The rotation submodule is used to rotate the point cloud of the sensing device collected by any of the multiple sensing devices from the sensing device coordinate system to the device coordinate system based on the rotation extrinsic parameters of any of the sensing devices. The translation submodule is used to perform translation transformation on the rotated point cloud of any sensing device based on the translation extrinsic parameters of any sensing device. The merging submodule is used to merge the point clouds of the multiple sensing devices after rotation and translation transformation to obtain the global point cloud.

[0105] Optionally, the segmentation module 1103 includes: The transformation submodule is used to convert the global point cloud into a polar coordinate system, and divide the polar coordinate system into concentric rings based on a preset angle and a preset radius to form multiple sector grids; The removal submodule is used to remove the point cloud based on the reference ground normal vector in each of the plurality of sector grids, and if the point cloud type in each sector grid is determined to be a ground point cloud, and to determine the remaining point cloud as the obstacle point cloud.

[0106] Optionally, remove submodules, including: The first acquisition unit is used to acquire the reference ground normal vector within each of the plurality of sector grids; The first computing unit is used to calculate the local normal vector of any point cloud within each of the said sector grids, the local normal vector being obtained based on a preset number of neighboring point clouds of any point cloud; The determination unit is used to determine that any point cloud is a ground point cloud and remove it when the angle between the local normal vector and the reference ground normal vector of the fan-shaped grid is less than or equal to a preset angle threshold, wherein the fan-shaped grid corresponds to the local normal vector. The traversal unit is used to traverse the point cloud within all sector grids and determine the remaining point cloud as the obstacle point cloud.

[0107] Optionally, the first computing unit includes: A sub-unit is used to acquire a preset number of neighboring point clouds for any given point cloud; The first computational subunit is used to calculate the centroid of the nearest point cloud; Construct sub-units to build a covariance matrix based on the deviation of the nearest point cloud relative to the centroid; The solution sub-unit is used to solve for multiple sets of eigenvalues ​​and eigenvectors of the covariance matrix, and the eigenvector corresponding to the smallest eigenvalue is determined as the local normal vector.

[0108] Optionally, the projection module 1104 includes: The extraction submodule is used to extract the extreme values ​​of the coordinates of the obstacle point cloud in the plane dimension corresponding to the two-dimensional grid map. The extreme values ​​of the coordinates include the maximum and minimum coordinates in the plane dimension. The second determining submodule is used to determine the size of the two-dimensional raster map based on the difference between the coordinate extreme values ​​and the preset raster resolution; The third calculation submodule is used to calculate the grid coordinates of each point in the two-dimensional grid map for each point in the obstacle point cloud, based on the coordinate value of each point in the plane dimension, the extreme value of the coordinates and the preset grid resolution. An integration submodule is used to integrate the grid coordinates of all points based on the stated size to form the two-dimensional grid map.

[0109] Optionally, the projection module 1104 also includes: The acquisition submodule is used to acquire at least one set of obstacle points in the two-dimensional grid map through region division processing; The analysis submodule is used to perform principal component analysis on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points.

[0110] Optionally, obtain submodules, specifically for: The two-dimensional grid map is subjected to connected component analysis to obtain at least one connected component, and each connected component corresponds to a set of obstacle points.

[0111] Optionally, the analysis submodule includes: The analysis unit is used to perform principal component analysis on any set of points in each set of obstacle points, based on the mean and covariance matrix of the set of points, to obtain the eigenvectors. A mapping unit is used to map any set of points to a local coordinate system to obtain a set of target points through the inverse transformation of the transition matrix. The local coordinates are constructed based on the feature vectors, and the transition matrix is ​​determined based on the mean and the covariance matrix. The determining unit is used to determine the obstacle bounding box of any point set based on the target point set, wherein the obstacle bounding box corresponds to an obstacle.

[0112] Optionally, the determined unit includes: An extraction subunit is used to extract the extreme values ​​of the target point set on each coordinate axis of the local coordinate system, wherein the extreme values ​​include the maximum and minimum coordinate values ​​in each coordinate axis direction; The second calculation subunit is used to calculate the length and width of the obstacle bounding box along each coordinate axis of the local coordinate system based on the difference between the coordinate extreme values ​​and the preset resolution of the two-dimensional grid map. The first determining subunit is used to determine the center coordinates of the obstacle bounding box in the coordinate system of the two-dimensional grid map based on the mean of the set of any points and the transition matrix. The second determining subunit is used to determine the rotation angle of the obstacle bounding box relative to the coordinate system of the two-dimensional grid map based on the rotation component of the transfer matrix. An integration subunit is used to integrate the length, width, center coordinates, and rotation angle to form an obstacle bounding box for any set of points.

[0113] Figure 12 A schematic diagram of the hardware structure of the electronic device provided in an embodiment of this application is shown.

[0114] The device may include a processor 1201 and a memory 1202 storing program instructions.

[0115] When processor 1201 executes the program, it implements the steps in any of the above method embodiments.

[0116] For example, the program can be divided into one or more modules / units, one or more of which are stored in memory 1202 and executed by processor 1201 to complete this application. The one or more modules / units can be a series of program instruction segments capable of performing a specific function, which describe the execution process of the program in the device.

[0117] Specifically, the processor 1201 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application.

[0118] Memory 1202 may include mass storage for data or instructions. For example, and not limitingly, memory 1202 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 1202 may include removable or non-removable (or fixed) media. Where appropriate, memory 1202 may be internal or external to the integrated gateway disaster recovery device. In a particular embodiment, memory 1202 is non-volatile solid-state memory.

[0119] Memory may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, and electrical, optical, or other physical / tangible memory storage devices. Therefore, typically, memory includes one or more tangible (non-transitory) readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the methods according to one aspect of this disclosure.

[0120] The processor 1201 implements any of the methods described in the above embodiments by reading and executing program instructions stored in the memory 1202.

[0121] In one example, the electronic device may also include a communication interface 12012 and a bus 1210. The processor 1201, memory 1202, and communication interface 12012 are connected via the bus 1210 and communicate with each other.

[0122] The communication interface 12012 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of this application.

[0123] Bus 1210 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 1210 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, any suitable bus or interconnect is contemplated herein.

[0124] Furthermore, in conjunction with the methods in the above embodiments, this application embodiment can provide a storage medium for implementation. This storage medium stores program instructions; when these program instructions are executed by a processor, they implement any of the methods in the above embodiments.

[0125] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above method embodiments and achieve the same technical effect. To avoid repetition, it will not be described again here.

[0126] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0127] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above method embodiments and achieve the same technical effects. To avoid repetition, it will not be described again here.

[0128] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0129] The functional modules shown in the above block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this application are programs or code segments used to perform the required tasks. Programs or code segments can be stored on machine-readable media or transmitted over a transmission medium or communication link via data signals carried on a carrier wave. "Machine-readable media" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer grids such as the Internet, intranets, etc.

[0130] It should also be noted that the exemplary embodiments mentioned in this application describe methods or systems based on a series of steps or apparatus. However, this application is not limited to the order of the above steps; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0131] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to create a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0132] The above are merely specific embodiments of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. An obstacle segmentation method, characterized in that, The method includes: The translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system are obtained respectively. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. Calculations are performed based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, which is the stitching result of the point clouds of the multiple sensing devices in the device coordinate system. The global point cloud is segmented into ground layers, and the ground point cloud is removed to obtain the obstacle point cloud; The obstacle point cloud is projected into a two-dimensional grid map, and the obstacles corresponding to the obstacle point cloud are determined based on the two-dimensional grid map.

2. The obstacle segmentation method according to claim 1, characterized in that, Obtain the translational extrinsic parameters of multiple sensing devices in the device coordinate system, including: The coordinates of multiple sensing devices in the device coordinate system are determined as spatial position parameters, and the translation extrinsic parameters are determined based on the spatial position parameters. The device coordinate system is constructed with a preset feature point on the device under test as the origin, the x-axis direction parallel to the travel direction of the device under test, the y-axis direction parallel to the lateral direction of the device under test, and the z-axis direction perpendicular to the ground. The preset feature point is determined based on the geometric center of the lateral section of the device under test, and the lateral section is parallel to the travel direction.

3. The obstacle segmentation method according to claim 1, characterized in that, The attitude tilt angle parameters include a first sub-attitude tilt angle parameter and a second attitude tilt angle parameter; the rotational extrinsic parameters of multiple sensing devices in the device coordinate system are obtained, including: For any one of the plurality of sensing devices, the following steps are performed respectively: In the device coordinate system, the first sub-attitude tilt angle parameter is calculated based on the preset ground reference normal vector and the normal vector of the ground point cloud collected by any of the sensing devices. The first rotation quaternion of any of the sensing devices is determined based on the attitude tilt angle parameter. The first sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the x-axis of the device coordinate system. In the device coordinate system, a second sub-attitude tilt angle parameter is calculated based on a preset ground reference normal vector and the normal vector of the vertical plane collected by any of the sensing devices. The second rotation quaternion of any of the sensing devices is determined based on the attitude tilt angle parameter. The second sub-attitude tilt angle parameter is used to characterize the rotational deviation of the sensing device around the y-axis of the device coordinate system. The vertical plane is a vertical plane parallel to the y-axis in the device coordinate system. The product of the first rotational quaternion and the second rotational quaternion is determined as the rotational extrinsic parameter of any of the sensing devices.

4. The obstacle segmentation method according to claim 1, characterized in that, The calculation based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain the global point cloud includes: For the point cloud of the sensing device collected by any of the plurality of sensing devices, the point cloud of the sensing device is rotated from the sensing device coordinate system of any of the sensing devices to the device coordinate system based on the rotation extrinsic parameter of any of the sensing devices. Based on the translation extrinsic parameters of any sensing device, a translation transformation is performed on the rotated point cloud of the sensing device. The point clouds obtained by merging the point clouds from the multiple sensing devices after rotation and translation transformation are obtained to obtain the global point cloud.

5. The obstacle segmentation method according to claim 1, characterized in that, The step of performing ground segmentation on the global point cloud and removing the ground point cloud to obtain the obstacle point cloud includes: The global point cloud is converted into a polar coordinate system, and based on a preset angle and a preset radius, the polar coordinate system is divided into concentric rings to form multiple sector grids; Based on the reference ground normal vector within each of the plurality of sector grids, if the point cloud type in each sector grid is determined to be a ground point cloud, the point cloud is removed, and the remaining point cloud is determined to be the obstacle point cloud.

6. The obstacle segmentation method according to claim 5, characterized in that, The step of removing the point cloud based on the reference ground normal vector within each of the plurality of sector grids, and determining the remaining point cloud as the obstacle point cloud, when the point cloud type in each sector grid is determined to be a ground point cloud, includes: Obtain the reference ground normal vector within each of the plurality of sector grids; For any point cloud within each of the said sector grids, calculate the local normal vector of any point cloud, which is obtained based on a preset number of neighboring point clouds of any point cloud; If the angle between the local normal vector and the reference ground normal vector of the sector grid is less than or equal to a preset angle threshold, the point cloud is determined to be a ground point cloud and removed, and the sector grid corresponds to the local normal vector; Traverse the point clouds within all sector grids and identify the remaining point clouds as the obstacle point clouds.

7. The obstacle segmentation method according to claim 6, characterized in that, The step of calculating the local normal vector of any point cloud within each of the said sector grids includes: Obtain a preset number of neighboring point clouds for any given point cloud; Calculate the centroid of the nearest neighbor point cloud; Based on the deviation of the nearest point cloud relative to the centroid, a covariance matrix is ​​constructed; Solve for multiple sets of eigenvalues ​​and eigenvectors of the covariance matrix, and determine the eigenvector corresponding to the smallest eigenvalue as the local normal vector.

8. The obstacle segmentation method according to claim 1, characterized in that, The step of projecting the obstacle point cloud into a two-dimensional grid map includes: Extract the extreme values ​​of the obstacle point cloud in the plane dimension corresponding to the two-dimensional grid map, where the extreme values ​​include the maximum and minimum coordinates in the plane dimension; The size of the two-dimensional raster map is determined based on the difference between the coordinate extreme values ​​and the preset raster resolution. For each point in the obstacle point cloud, the grid coordinates of each point in the two-dimensional grid map are calculated based on the coordinate values ​​of each point in the plane dimension, the extreme values ​​of the coordinates, and the preset grid resolution. Based on the stated dimensions, the grid coordinates of all points are integrated to form the two-dimensional grid map.

9. The obstacle segmentation method according to claim 1, characterized in that, Based on the two-dimensional raster map, the obstacles corresponding to the obstacle point cloud are determined, including: At least one set of obstacle points in the two-dimensional grid map is obtained through region segmentation processing; Principal component analysis is performed on each set of obstacle points to determine the obstacles corresponding to each set of obstacle points.

10. The obstacle segmentation method according to claim 9, characterized in that, At least one set of obstacle points in the two-dimensional raster map is obtained through region segmentation processing, including: The two-dimensional grid map is subjected to connected component analysis to obtain at least one connected component, and each connected component corresponds to a set of obstacle points.

11. The obstacle segmentation method according to claim 9, characterized in that, Principal component analysis was performed on each set of obstacle points to determine the obstacles corresponding to each set, including: For any set of points in each set of obstacle points, principal component analysis is performed on the set of points based on the mean and covariance matrix of the set of points to obtain the eigenvectors. By inverse transformation of the transition matrix, the set of any point is mapped to a local coordinate system to obtain the target set of points. The local coordinates are constructed based on the feature vectors, and the transition matrix is ​​determined based on the mean and the covariance matrix. Based on the target point set, determine the obstacle bounding box of any point set, and the obstacle bounding box corresponds to an obstacle.

12. The obstacle segmentation method according to claim 11, characterized in that, Determining the obstacle bounding box for any point set based on the target point set includes: Extract the extreme values ​​of the target point set on each coordinate axis of the local coordinate system, wherein the extreme values ​​include the maximum and minimum coordinate values ​​in each coordinate axis direction; Based on the difference between the extreme values ​​of the coordinates and the preset resolution of the two-dimensional grid map, the length and width of the obstacle bounding box along each coordinate axis of the local coordinate system are calculated; Based on the mean of the set of points and the transition matrix, determine the center coordinates of the obstacle bounding box in the coordinate system of the two-dimensional grid map; Based on the rotation component of the transfer matrix, the rotation angle of the obstacle bounding box relative to the coordinate system of the two-dimensional grid map is determined; By integrating the length, width, center coordinates, and rotation angle, an obstacle bounding box is formed for any set of points.

13. An obstacle segmentation device, characterized in that, The device includes: The acquisition module is used to acquire translational and rotational extrinsic parameters of multiple sensing devices in the device coordinate system. The multiple sensing devices are set on the device under test corresponding to the device coordinate system. The translational extrinsic parameters are determined based on the spatial position parameters of each sensing device in the device coordinate system, and the rotational extrinsic parameters are determined based on the attitude tilt angle parameters of each sensing device in the device coordinate system. The calculation module is used to perform calculations based on the translation extrinsic parameters and the rotation extrinsic parameters to obtain a global point cloud, which is the stitching result of the point clouds of the multiple sensing devices in the device coordinate system; The segmentation module is used to perform ground segmentation on the global point cloud and remove the ground point cloud to obtain the obstacle point cloud; The projection module is used to project the obstacle point cloud into a two-dimensional grid map, and to determine the obstacles corresponding to the obstacle point cloud based on the two-dimensional grid map.

14. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the obstacle segmentation method as described in any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the obstacle segmentation method as described in any one of claims 1-12.

Images