Three-dimensional reconstruction method, electronic equipment, vehicle and computer program product

Abstract

The embodiment of the invention provides a three-dimensional reconstruction method, electronic equipment, a vehicle and a computer program product, and relates to the technical field of three-dimensional reconstruction. The method comprises the following steps: segmenting an initial point cloud map into a plurality of voxels, and screening a plurality of feature voxels from the plurality of voxels; wherein the initial point cloud map is obtained according to collected multi-frame point cloud data, and the plurality of feature voxels comprise line feature voxels and surface feature voxels; for each feature voxel in the plurality of feature voxels, determining a pose error of each feature point in the feature voxel according to a position relationship between each feature point in the feature voxel and a distribution region of the point clouds in the feature voxel; for each frame of point cloud data in the multiple frames of point cloud data, determining an optimized pose of the point cloud data according to pose errors of feature points in the point cloud data; and according to the optimized pose of each frame of point cloud data in the multiple frames of point cloud data, splicing each frame of point cloud data to obtain a target point cloud map. According to the embodiment of the invention, the target point cloud map can be accurately reconstructed.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional reconstruction, and in particular to a three-dimensional reconstruction method, electronic equipment, vehicle and computer program product. Background Art

[0002] The single-frame point cloud data collected by the radar contains the three-dimensional structural information of the target object within the radar field of view, but does not contain the three-dimensional structural information of the obstructed objects within the radar field of view and other objects outside the field of view. In order to completely reconstruct the target scene, it is necessary to move the radar to completely scan the target scene to collect multiple frames of point cloud data that completely cover the target scene, and to align the point clouds of the same target object in adjacent frames of point cloud data to obtain the relative pose between adjacent frames of point cloud data, determine the pose of each frame of point cloud data, and convert the three-dimensional structural information of the target object in each frame of point cloud data to the global coordinate system according to the pose of each frame of point cloud data to obtain a point cloud map and complete the reconstruction of the target scene.

[0003] In the above 3D reconstruction process, due to the error in the relative pose calculation between adjacent frames of point cloud data, the pose error of the current frame of point cloud data will be accumulated into the pose error of the next frame of point cloud data, so the obtained point cloud map deviates from the actual target scene. In addition to the measurement error of the vehicle-mounted radar itself, there are two main errors in the point cloud map: 1. Errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data; 2. Errors such as target blur or wall thickening introduced by the measurement error of the vehicle-mounted radar itself. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a three-dimensional reconstruction method, electronic equipment, vehicle and computer program product to achieve the technical effect of accurately reconstructing a target point cloud map.

[0005] In a first aspect, an embodiment of the present application provides a three-dimensional reconstruction method, comprising:

[0006] The initial point cloud map is divided into a plurality of voxels, and a plurality of characteristic voxels are selected from the plurality of voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the plurality of characteristic voxels include line characteristic voxels and surface characteristic voxels;

[0007] For each feature voxel of the plurality of feature voxels, determining a position and posture error of each feature point in the feature voxel according to a positional relationship between each feature point in the feature voxel and a distribution area of ​​a point cloud in the feature voxel;

[0008] For each frame of point cloud data in the multiple frames of point cloud data, determining an optimized pose of the point cloud data according to a pose error of a feature point in the point cloud data;

[0009] The target point cloud map is obtained by splicing each frame of point cloud data according to the optimized posture of each frame of point cloud data in the multiple frames of point cloud data.

[0010] In the above implementation process, the initial point cloud map is divided into multiple voxels, and multiple feature voxels including line feature voxels and surface feature voxels are selected from the multiple voxels. The pose error of each feature point in each feature voxel is determined according to the positional relationship between each feature point in each feature voxel and the distribution area of ​​the point cloud in each feature voxel. The optimized pose of each frame of point cloud data is determined according to the pose error of the feature point in each frame of point cloud data. The target point cloud map is obtained by splicing each frame of point cloud data according to the optimized pose of each frame of point cloud data. The correspondence between point and line and point to surface can be used instead to accurately determine the correspondence between feature points and target objects to perform point cloud registration of adjacent frames of point cloud data, which effectively reduces errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data in the three-dimensional reconstruction process, thereby accurately reconstructing the target point cloud map.

[0011] Furthermore, before dividing the initial point cloud map into a plurality of voxels and selecting a plurality of characteristic voxels from the plurality of voxels, the method further includes:

[0012] Acquire the multiple frames of point cloud data collected by the point cloud collection device;

[0013] Determine the initial posture of each frame of point cloud data according to the posture data collected by the inertial measurement unit at the collection time of each frame of point cloud data in the multiple frames of point cloud data;

[0014] The point cloud data of each frame is spliced ​​according to the initial posture of each frame of point cloud data to obtain the initial point cloud map.

[0015] In the above implementation process, by acquiring multiple frames of point cloud data collected by the point cloud acquisition device, the initial posture of each frame of point cloud data is determined according to the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, and each frame of point cloud data is spliced ​​according to the initial posture of each frame of point cloud data to obtain an initial point cloud map. The multiple frames of point cloud data can be spliced ​​with the help of the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, thereby realizing the rapid and accurate acquisition of the initial point cloud map.

[0016] Further, the screening of a plurality of characteristic voxels from the plurality of voxels comprises:

[0017] For each voxel of the plurality of voxels, determining a shape of a distribution area of ​​the point cloud within the voxel according to a distribution span of the point cloud within the voxel in a plurality of spatial dimensions;

[0018] When the shape of the distribution area of ​​the point cloud in the voxel approaches a line, determining the voxel as the line feature voxel;

[0019] When the shape of the distribution area of ​​the point cloud in the voxel approaches a plane, the voxel is determined to be the surface feature voxel.

[0020] In the above implementation process, the shape of the distribution area of ​​the point cloud in the voxel is determined directly according to the distribution span of the point cloud in the voxel in multiple spatial dimensions. When the shape of the distribution area of ​​the point cloud in the voxel is close to a line, the voxel is determined to be a line feature voxel; when the shape of the distribution area of ​​the point cloud in the voxel is close to a plane, the voxel is determined to be a surface feature voxel. In this way, multiple feature voxels including line feature voxels and surface feature voxels can be extracted quickly and accurately from multiple voxels.

[0021] Furthermore, determining the shape of the distribution area of ​​the point cloud in the voxel according to the distribution span of the point cloud in the voxel in multiple spatial dimensions includes:

[0022] Determine a plurality of eigenvalues ​​of a covariance matrix of the intra-voxel point cloud; wherein each eigenvalue of the plurality of eigenvalues ​​is used to characterize a distribution span of the intra-voxel point cloud in each of a plurality of spatial dimensions;

[0023] The shape of the distribution area of ​​the point cloud in the voxel is determined according to the multiple eigenvalues.

[0024] In the above implementation process, by determining multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, and determining the shape of the distribution area of ​​the point cloud within the voxel according to the distribution span of the point cloud within the voxel in multiple spatial dimensions represented by the multiple eigenvalues, the distribution span of the point cloud within the voxel in multiple spatial dimensions can be uniformly determined, thereby ensuring that multiple feature voxels including line feature voxels and surface feature voxels are extracted quickly and accurately from multiple voxels.

[0025] Further, determining the shape of the distribution area of ​​the point cloud in the voxel according to the multiple eigenvalues ​​includes:

[0026] Counting the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds in the voxel;

[0027] The shape of the distribution area of ​​the point cloud in the voxel is determined according to the size relationship between the ratio and the span threshold.

[0028] In the above implementation process, after determining multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds within the voxel is counted, and the shape of the distribution area of ​​the point cloud within the voxel is determined according to the size relationship between the ratio and the span threshold. This can more accurately determine the distribution span of the point cloud within the voxel in multiple spatial dimensions, thereby more accurately extracting multiple feature voxels including line feature voxels and surface feature voxels from multiple voxels.

[0029] Further, the span threshold includes a first span threshold and a second span threshold;

[0030] The determining the shape of the distribution area of ​​the point cloud in the voxel according to the size relationship between the ratio and the span threshold value includes:

[0031] When the ratio is less than the first span threshold, determining that the shape of the distribution area of ​​the point cloud in the voxel is close to a line;

[0032] When the ratio is greater than the second span threshold, it is determined that the shape of the distribution area of ​​the point cloud in the voxel approaches a plane.

[0033] In the above implementation process, after determining multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds within the voxel is counted, and when the ratio is less than the first span threshold, the shape of the distribution area of ​​the point cloud within the voxel is determined to be close to a line, and when the ratio is greater than the second span threshold, the shape of the distribution area of ​​the point cloud within the voxel is determined to be close to a plane. The ratio of the second largest value to the number of point clouds within the voxel can be used to quickly infer the distribution span of the point cloud within the voxel in each spatial dimension, thereby more quickly extracting multiple feature voxels including linear feature voxels and surface feature voxels from multiple voxels.

[0034] Further, the screening of a plurality of characteristic voxels from the plurality of voxels comprises:

[0035] In the case where the voxel is neither the line feature voxel nor the surface feature voxel, determining the voxel as a target voxel, and dividing the target voxel into a plurality of sub-voxels;

[0036] For each sub-voxel of the plurality of sub-voxels, if the shape of the distribution area of ​​the point cloud in the sub-voxel is close to a line or a plane, the sub-voxel is determined to be an extracted feature voxel, otherwise the sub-voxel is determined to be a non-feature voxel;

[0037] When it is determined that the sub-voxel is a non-feature voxel and the size of the sub-voxel is greater than or equal to the minimum size threshold, the sub-voxel is determined to be the next target voxel, and the step of dividing the target voxel into multiple sub-voxels is returned to execute until the size of all non-feature voxels is less than the minimum size threshold.

[0038] In the above implementation process, when any voxel is neither a line feature voxel nor a surface feature voxel, the voxel is determined to be a target voxel, and the voxel is further divided into multiple sub-voxels. Multiple feature voxels are extracted from the multiple sub-voxels, and non-feature voxels with a size greater than or equal to a minimum size threshold in the multiple sub-voxels are determined as the next target voxel. Segmentation and judgment of the shape of the point cloud distribution area are performed again until the recursive voxels are indivisible. The initial point cloud map can be voxel segmented more finely, thereby ensuring that multiple feature voxels including line feature voxels and surface feature voxels are completely extracted from multiple voxels.

[0039] Furthermore, determining the position and posture errors of each feature point in the feature voxel according to the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel includes:

[0040] For each line feature point in the line feature voxel, determining a position and posture error of the line feature point with the goal of minimizing the sum of squares of distances between the line feature point and the line on which the point cloud in the line feature voxel is located;

[0041] For each surface feature point in the surface feature voxel, the position and posture error of the surface feature point is determined with the goal of minimizing the sum of squares of distances between the surface feature point and a plane where the point cloud in the surface feature voxel is located.

[0042] In the above implementation process, the pose error of the line feature point is determined by minimizing the sum of the squares of the distances between the line feature point and the line where the line feature point is located, and the pose error of the surface feature point is determined by minimizing the sum of the squares of the distances between the surface feature point and the plane where the surface feature point is located. The pose error of the line feature point can be determined based on the point-to-line constraint, and the pose error of the surface feature point can be determined based on the point-to-surface constraint, which is conducive to accurately constructing the pose error of the feature point.

[0043] Furthermore, after stitching the frames of point cloud data according to the optimized poses of the frames of point cloud data in the multiple frames of point cloud data to obtain the target point cloud map, the method further includes:

[0044] For each first target point in the target point cloud map, determine the position of a first optimization point according to the positions of a plurality of first neighborhood points, and update the position of the first target point to the position of the first optimization point, so as to obtain an optimized point cloud map; wherein the plurality of first neighborhood points are within the neighborhood of the first target point;

[0045] For each second target point in the optimized point cloud map, the position of the second optimized point is determined according to the weights and positions of the plurality of second neighborhood points, and the position of the second target point is updated to the position of the second optimized point to obtain an optimal point cloud map; wherein the plurality of second neighborhood points are within the neighborhood of the second target point.

[0046] In the above implementation process, the optimal point cloud map is obtained by performing two-step filtering on the target point cloud map, which can reduce errors such as target blur or wall thickening introduced by the measurement error of the point cloud acquisition device itself, while avoiding distortion of local details, and further improving the accuracy of three-dimensional reconstruction.

[0047] In a second aspect, an embodiment of the present application provides a three-dimensional reconstruction device, including:

[0048] A feature voxel screening module is used to divide the initial point cloud map into multiple voxels and screen multiple feature voxels from the multiple voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the multiple feature voxels include line feature voxels and surface feature voxels;

[0049] A posture error determination module is used to determine, for each feature voxel of the plurality of feature voxels, a posture error of each feature point in the feature voxel according to a positional relationship between each feature point in the feature voxel and a distribution area of ​​a point cloud in the feature voxel;

[0050] An optimized posture determination module is used to determine the optimized posture of the point cloud data for each frame of point cloud data in the multiple frames of point cloud data according to the posture errors of feature points in the point cloud data;

[0051] The point cloud map reconstruction module is used to splice the frame point cloud data according to the optimized posture of each frame point cloud data in the multiple frames of point cloud data to obtain a target point cloud map.

[0052] In a third aspect, an embodiment of the present application provides an electronic device, comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; when the processor executes the computer program, the method described above is implemented.

[0053] In a fourth aspect, an embodiment of the present application provides a vehicle, comprising the electronic device as described above.

[0054] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes instructions, and when the instructions are executed by a computer, the computer implements the method as described above.

[0055] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A schematic diagram of a flow chart of a three-dimensional reconstruction method provided in the first embodiment of the present application;

[0058] Figure 2 A schematic diagram of the structure of a three-dimensional reconstruction device provided in the second embodiment of the present application;

[0059] Figure 3 A schematic diagram of the structure of an electronic device provided in the third embodiment of the present application. DETAILED DESCRIPTION

[0060] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0061] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0062] Radar is one of the commonly used sensors in the 3D reconstruction process. It mainly transmits signals to the target through the transmitter inside the radar, and receives the signals reflected from the target through the receiver inside the radar. Based on the TOF (Time of Flight) ranging method, the distance between the radar and the target is measured according to the round-trip flight time of the signal between the transmitter and the receiver, so as to collect single-frame point cloud data containing the 3D structure information of the target.

[0063] The single-frame point cloud data collected by the radar contains the 3D structure information of the target within the radar field of view, but does not contain the 3D structure information of the obstructed objects within the radar field of view and other objects outside the field of view. In order to completely reconstruct the target scene, the mobile radar needs to completely scan the target scene to collect multi-frame point cloud data that completely covers the target scene.

[0064] In related technologies, after the mobile radar collects multiple frames of point cloud data, the main process is to align the point clouds of the same target object in adjacent frames of point cloud data to obtain the relative pose between adjacent frames of point cloud data, determine the pose of each frame of point cloud data, and convert the three-dimensional structure information of the target object in each frame of point cloud data into a global coordinate system according to the pose of each frame of point cloud data to obtain a point cloud map and complete the reconstruction of the target scene.

[0065] In the above 3D reconstruction process, due to the error in the relative pose calculation between adjacent frames of point cloud data, the pose error of the current frame of point cloud data will be accumulated into the pose error of the next frame of point cloud data, so the obtained point cloud map deviates from the actual target scene. In addition to the measurement error of the vehicle-mounted radar itself, there are two main errors in the point cloud map: 1. Errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data; 2. Errors such as target blur or wall thickening introduced by the measurement error of the vehicle-mounted radar itself.

[0066] To this end, the present application proposes a three-dimensional reconstruction method, which divides the initial point cloud map into multiple voxels, selects multiple feature voxels including line feature voxels and surface feature voxels from the multiple voxels, determines the pose error of each feature point in each feature voxel according to the positional relationship between each feature point in each feature voxel and the distribution area of ​​the point cloud in each feature voxel, determines the optimized pose of each frame of point cloud data according to the pose error of the feature point in each frame of point cloud data, and splices each frame of point cloud data according to the optimized pose of each frame of point cloud data to obtain a target point cloud map. The point-to-line correspondence and the point-to-surface correspondence can be used instead to accurately determine the correspondence between the feature points and the target object to perform point cloud registration of adjacent frames of point cloud data, effectively reducing errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data during the three-dimensional reconstruction process, thereby achieving accurate reconstruction of the target point cloud map.

[0067] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0068] It should be understood that the vehicle described in the embodiments of the present application may be a pure electric vehicle, or a hybrid vehicle, a gas-electric hybrid vehicle, a fuel cell vehicle, or a vehicle of other energy types with a motor, and the embodiments of the present application are not limited thereto. The types of vehicles in the embodiments of the present application may include vehicles such as sedans, off-road vehicles, trucks, and buses, and the embodiments of the present application are not limited thereto. The vehicles in the embodiments of the present application may be vehicles with automatic intelligent driving, manned vehicles, or manned vehicles with assisted intelligent driving functions, and the embodiments of the present application are not limited thereto.

[0069] For the convenience of description and to make the solutions of the embodiments of the present application easier to understand, before describing the methods implemented in the present application, the terms involved in the embodiments of the present application are defined as follows:

[0070] SLAM (Simultaneous Localization and Mapping): is a technology that simultaneously realizes the positioning of the device itself and the construction of the environment map. It uses sensors such as cameras, lidar, and inertial measurement units to collect environmental information, and integrates this information through algorithms to determine the location of the device in an unknown environment and build an environmental map.

[0071] Inertial Measurement Unit (IMU): A device that measures the three-axis attitude angle (or angular rate) and acceleration of a device.

[0072] Bundle adjustment: also known as bundle method regional block adjustment, it takes a beam of light composed of an image as the basic unit of adjustment, and the collinear equation of the central projection as the basic equation of adjustment. Its basic idea is to take each beam (photo) as the basic adjustment unit, according to the equality of the external coordinates of the control point and the internal coordinates of the infill point, list the error equation according to the collinear condition equation, perform adjustment processing uniformly in the whole area, solve the exterior orientation elements of each photo, and then calculate the ground coordinates of the infill point according to the intersection of multiple photos.

[0073] Mean filtering: It is a linear filtering algorithm. It means determining a template for the target pixel on the image. The template includes the neighboring pixels of the target pixel, such as the template composed of 8 pixels around the target pixel, and then using the average pixel value of all pixels in the template to update the original pixel value of the target pixel.

[0074] WLOP (Weighted locally optimal projection) filtering: is a point cloud enhancement algorithm.

[0075] Combine the following Figure 1 A three-dimensional reconstruction method according to an embodiment of the present application is described. The method provided in the embodiment of the present application can be executed by a vehicle controller or a domain controller, and the following description is made by taking the domain controller as an example of the execution subject.

[0076] Please see Figure 1 , Figure 1 A schematic flow chart of a three-dimensional reconstruction method provided in the first embodiment of the present application. The first embodiment of the present application provides a three-dimensional reconstruction method, including steps S101 to S104:

[0077] S101, dividing the initial point cloud map into multiple voxels, and selecting multiple feature voxels from the multiple voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the multiple feature voxels include line feature voxels and surface feature voxels.

[0078] As an example, when three-dimensional reconstruction of the target scene is required, a mobile point cloud acquisition device, such as a radar, scans the target scene to collect multiple frames of point cloud data, and a domain controller obtains the multiple frames of point cloud data collected by the point cloud acquisition device, and obtains an initial point cloud map based on the multiple frames of point cloud data.

[0079] According to actual application requirements, a maximum size threshold is pre-set. After obtaining the initial point cloud map, the domain controller divides the initial point cloud map into multiple voxels according to the pre-set maximum size threshold.

[0080] It should be noted that voxel is the abbreviation of volume pixel, which represents a three-dimensional area. For example, the preset maximum size threshold is a·a·a, where a is a positive integer.

[0081] After obtaining a plurality of voxels, the domain controller selects a plurality of feature voxels from the plurality of voxels, wherein the plurality of feature voxels include line feature voxels and surface feature voxels.

[0082] S102 . For each feature voxel among the plurality of feature voxels, determine the position and posture error of each feature point in the feature voxel according to the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel.

[0083] As an example, after obtaining multiple feature voxels, the domain controller determines the distribution area of ​​the point cloud within each feature voxel, and determines the pose error of each feature point within the feature voxel based on the positional relationship between each feature point within the feature voxel and the distribution area of ​​the point cloud within the feature voxel, thereby obtaining the pose error of each feature point within multiple feature voxels.

[0084] In practical applications, for each line feature voxel, the feature point in the line feature voxel is the line feature point, the distribution area of ​​the point cloud in the line feature voxel is determined, and based on the algorithm idea of ​​the bundle adjustment algorithm, the position relationship between each line feature point in the line feature voxel and the distribution area of ​​the point cloud in the line feature voxel is used to determine the position and posture error of each line feature point in the line feature voxel; for each surface feature voxel, the feature point in the surface feature voxel is the surface feature point, the distribution area of ​​the point cloud in the surface feature voxel is determined, and based on the algorithm idea of ​​the bundle adjustment algorithm, the position and posture error of each surface feature point in the surface feature voxel is determined according to the position relationship between each surface feature point in the surface feature voxel and the distribution area of ​​the point cloud in the surface feature voxel.

[0085] S103 . For each frame of point cloud data in the multiple frames of point cloud data, determine an optimized pose of the point cloud data according to pose errors of feature points in the point cloud data.

[0086] As an example, after obtaining the pose errors of each feature point in multiple feature voxels, the domain controller determines the feature points in each frame of point cloud data, and determines the optimized pose of the frame of point cloud data based on the pose errors of the feature points in the frame of point cloud data, thereby obtaining the optimized pose of multiple frames of point cloud data.

[0087] Specifically, for each frame of point cloud data, the line feature points and / or surface feature points in the frame of point cloud data are determined, and the optimized pose of the frame of point cloud data is determined based on the pose errors of the line feature points and / or the pose errors of the surface feature points in the frame of point cloud data.

[0088] Among them, the line feature points and / or surface feature points include the following three situations: only line feature points; only surface feature points; both line feature points and surface feature points. The pose errors of line feature points and / or the pose errors of surface feature points include the following three situations: only the pose errors of line feature points; only the pose errors of surface feature points; both the pose errors of line feature points and the pose errors of surface feature points.

[0089] S104, stitching each frame of point cloud data according to the optimized posture of each frame of point cloud data in the multiple frames of point cloud data to obtain a target point cloud map.

[0090] As an exemplary embodiment, after obtaining the optimized poses of multiple frames of point cloud data, the domain controller splices each frame of point cloud data according to the optimized poses of each frame of point cloud data to obtain a target point cloud map.

[0091] Since the feature points in a single frame of point cloud data are sparse, it is difficult to accurately determine the correspondence between the feature points and the points on the target object, and the bundle adjustment algorithm is highly dependent on accurate point-to-point correspondence, so it is difficult to directly apply the bundle adjustment algorithm in laser SLAM. To address this problem, the present application proposes to replace the point-to-point correspondence with the point-to-line correspondence and the point-to-surface correspondence. By dividing the initial point cloud map into multiple voxels, and screening multiple feature voxels including line feature voxels and surface feature voxels from the multiple voxels, the line feature points and surface feature points in the multi-frame point cloud data can be determined. Through the algorithm idea based on the bundle adjustment algorithm, the pose error of each line feature point in the line feature voxel is determined according to the point-to-line constraint, and the pose error of each surface feature point in the surface feature voxel is determined according to the point-to-surface constraint. The optimized pose of each frame of point cloud data is determined according to the pose errors of the line feature points and the surface feature points in each frame of point cloud data, and each frame of point cloud data is spliced ​​according to the optimized pose of each frame of point cloud data to obtain a target point cloud map. The initial pose of each frame of point cloud data and the three-dimensional structure information of the target object contained in each frame of point cloud data can be optimized, effectively reducing errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data, and improving the accuracy of three-dimensional reconstruction.

[0092] The embodiment of the present application divides the initial point cloud map into multiple voxels, selects multiple feature voxels including line feature voxels and surface feature voxels from the multiple voxels, determines the pose error of each feature point in each feature voxel according to the positional relationship between each feature point in each feature voxel and the distribution area of ​​the point cloud in each feature voxel, determines the optimized pose of each frame of point cloud data according to the pose error of the feature point in each frame of point cloud data, and splices each frame of point cloud data according to the optimized pose of each frame of point cloud data to obtain a target point cloud map. The correspondence between point to line and point to surface can be used instead to accurately determine the correspondence between feature points and target objects to perform point cloud registration of adjacent frames of point cloud data, effectively reducing errors such as ghosting and offset introduced by relative pose errors between adjacent frames of point cloud data in the three-dimensional reconstruction process, thereby achieving accurate reconstruction of the target point cloud map.

[0093] In an optional embodiment, before the initial point cloud map is divided into multiple voxels and multiple feature voxels are screened from the multiple voxels, it also includes: acquiring multiple frames of point cloud data collected by a point cloud acquisition device; determining an initial pose of each frame of point cloud data based on posture data collected by an inertial measurement unit at the time of collection of each frame of point cloud data in the multiple frames of point cloud data; and splicing each frame of point cloud data according to the initial pose of each frame of point cloud data to obtain an initial point cloud map.

[0094] As an example, when three-dimensional reconstruction of the target scene is required, a mobile point cloud acquisition device, such as a radar, scans the target scene to collect multiple frames of point cloud data, and the inertial measurement unit collects the posture data of the point cloud acquisition device at the time of collecting each frame of point cloud data, so that the domain controller can obtain the multiple frames of point cloud data collected by the point cloud acquisition device and the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data.

[0095] After obtaining multiple frames of point cloud data collected by the point cloud acquisition device and the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, the domain controller determines the initial posture of each frame of point cloud data according to the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, and splices each frame of point cloud data according to the initial posture of each frame of point cloud data to obtain an initial point cloud map.

[0096] In practical applications, the odometer at the front end can also determine the initial pose of each frame of point cloud data based on the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, so that the domain controller at the back end can directly splice each frame of point cloud data according to the initial pose of each frame of point cloud data to obtain the initial point cloud map.

[0097] For example, the positions of the point cloud acquisition device and the inertial measurement unit are calibrated, the point cloud acquisition device and the inertial measurement unit are started, and the point cloud acquisition device and the inertial measurement unit are synchronized in time. The mobile point cloud acquisition device scans the target scene, collects multiple frames of point cloud data, and the inertial measurement unit synchronously collects the posture data of the point cloud acquisition device corresponding to each frame of point cloud data, and obtains the posture data of the point cloud acquisition device corresponding to the multiple frames of point cloud data. Since the posture data of the point cloud acquisition device corresponding to the first frame of point cloud data is the calibration posture of the point cloud acquisition device by default, after the odometer pre-processes the multiple frames of point cloud data, it first determines that the initial posture of the first frame of point cloud data is the calibration posture of the point cloud acquisition device, and then starts from the posture data of the point cloud acquisition device corresponding to the second frame of point cloud data, traverses the posture data of the point cloud acquisition device corresponding to each frame of point cloud data, integrates the posture data of the point cloud acquisition device corresponding to the current frame of point cloud data, and obtains the coarse-precision relative posture between the current frame of point cloud data and the previous frame of point cloud data. The coarse-precision relative posture is used as the initial value, and the current frame of point cloud data and the previous frame of point cloud data are calibrated. The point cloud of the same target object in the previous frame of point cloud data is registered, and the relative pose between the current frame of point cloud data and the previous frame of point cloud data is calculated to obtain the pose transformation matrix between the current frame of point cloud data and the previous frame of point cloud data, thereby obtaining the pose transformation matrix between all adjacent frames of point cloud data. Then, the coordinate system of the first frame of point cloud data is used as the global coordinate system. According to the pose transformation matrix between all adjacent frames of point cloud data, the coordinate system of each frame of point cloud data is transformed to the global coordinate system. Finally, the origin of the coordinate system of each frame of point cloud data on the global coordinate system is connected to obtain the initial pose trajectory. Each point on the initial pose trajectory represents the initial pose of each frame of point cloud data. The domain controller splices each frame of point cloud data according to the initial pose of each frame of point cloud data to obtain the initial point cloud map.

[0098] In the 3D reconstruction process of the front end, due to the error in the relative pose calculation between adjacent frames of point cloud data, the pose error of the current frame of point cloud data will be accumulated into the pose error of the next frame of point cloud data, so the initial point cloud map obtained deviates from the actual target scene. In addition to the point cloud acquisition device, such as the measurement error of the radar itself, the initial point cloud map mainly has two errors: 1. Errors such as ghosting and offset introduced by the relative pose error between adjacent frames of point cloud data; 2. Errors such as target blur or wall thickening introduced by the measurement error of the point cloud acquisition device itself. To address this problem, the back-end domain controller will continue to optimize the initial point cloud map and reconstruct the target point cloud map.

[0099] The embodiment of the present application obtains multiple frames of point cloud data collected by a point cloud acquisition device, determines the initial pose of each frame of point cloud data according to the pose data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, and splices each frame of point cloud data according to the initial pose of each frame of point cloud data to obtain an initial point cloud map. The multiple frames of point cloud data can be spliced ​​with the help of the pose data collected by the inertial measurement unit at the time of collecting each frame of point cloud data, thereby realizing fast and accurate acquisition of the initial point cloud map.

[0100] In an optional embodiment, the screening of multiple feature voxels from multiple voxels includes: for each voxel in the multiple voxels, determining the shape of the distribution area of ​​the point cloud in the voxel according to the distribution span of the point cloud in the voxel in multiple spatial dimensions; when the shape of the distribution area of ​​the point cloud in the voxel approaches a line, determining the voxel as a line feature voxel; when the shape of the distribution area of ​​the point cloud in the voxel approaches a plane, determining the voxel as a surface feature voxel.

[0101] As an exemplary embodiment, after obtaining multiple voxels, the domain controller determines, for each voxel, the shape of the distribution area of ​​the point cloud in the voxel based on the distribution span of the point cloud in the voxel in multiple spatial dimensions; when the shape of the distribution area of ​​the point cloud in the voxel approaches a line, the voxel is determined to be a line feature voxel, and the feature points in the line feature voxel are line feature points; when the shape of the distribution area of ​​the point cloud in the voxel approaches a plane, the voxel is determined to be a surface feature voxel, and the feature points in the surface feature voxel are surface feature points; when the shape of the distribution area of ​​the point cloud in the voxel approaches neither a line nor a plane, the voxel is determined to be neither a line feature voxel nor a surface feature voxel, and is a non-feature voxel.

[0102] For example, for each voxel, the distribution span of the point cloud in the voxel in the three spatial dimensions of X, Y and Z is determined according to the point cloud in the voxel. If the distribution span of the point cloud in the voxel in any two spatial dimensions is less than a preset first distribution span threshold, it means that the point cloud in the voxel tends to be distributed in a single spatial dimension, and the shape of the distribution area of ​​the point cloud in the voxel is determined to be close to a line; if the distribution span of the point cloud in the voxel in any two spatial dimensions is greater than a preset second distribution span threshold, wherein the second distribution span threshold is greater than the first distribution span threshold, then the shape of the distribution area of ​​the point cloud in the voxel tends to be close to a line. If the distribution span threshold is less than the distribution span threshold, it means that the point cloud in the voxel tends to be distributed on the plane formed by the two spatial dimensions, and the shape of the distribution area of ​​the point cloud in the voxel is determined to be close to the plane; if the distribution span of the point cloud in the voxel in the three spatial dimensions neither satisfies the condition of "the distribution span of the point cloud in the voxel in any two spatial dimensions is less than the first distribution span threshold", nor satisfies the condition of "the distribution span of the point cloud in the voxel in any two spatial dimensions is greater than the second distribution span threshold", then it is determined that the shape of the point cloud in the voxel is neither close to a line nor close to a plane.

[0103] In practical applications, multiple feature voxel templates can also be pre-stored, and the multiple feature voxel templates include line feature voxel templates and surface feature voxel templates. The point cloud in the voxel is fitted with the point cloud in each feature voxel template, and the fitting effect of the point cloud in the voxel and the point cloud in each feature voxel template is evaluated based on the residual sum of squares. If the residual sum of squares is less than a preset evaluation threshold, the fitting effect is evaluated to be up to standard, otherwise the fitting effect is evaluated to be not up to standard. When the fitting effect of the point cloud in the voxel and the point cloud in any line feature voxel template is up to standard, it is determined that the shape of the distribution area of ​​the point cloud in the voxel is close to a line. When the fitting effect of the point cloud in the voxel and the point cloud in any surface feature voxel template is up to standard, it is determined that the shape of the distribution area of ​​the point cloud in the voxel is close to a plane. When the fitting effect of the point cloud in the voxel and the point cloud in each feature voxel template is not up to standard, it is determined that the shape of the distribution area of ​​the point cloud in the voxel is neither close to a line nor close to a plane.

[0104] The embodiment of the present application determines the shape of the distribution area of ​​the point cloud within the voxel directly according to the distribution span of the point cloud within the voxel in multiple spatial dimensions. When the shape of the distribution area of ​​the point cloud within the voxel approaches a line, the voxel is determined to be a line feature voxel; when the shape of the distribution area of ​​the point cloud within the voxel approaches a plane, the voxel is determined to be a surface feature voxel. In this way, multiple feature voxels including line feature voxels and surface feature voxels can be extracted quickly and accurately from multiple voxels.

[0105] In an optional embodiment, determining the shape of the distribution area of ​​the point cloud within the voxel based on the distribution span of the point cloud within the voxel in multiple spatial dimensions includes: determining multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel; wherein each eigenvalue of the multiple eigenvalues ​​is used to characterize the distribution span of the point cloud within the voxel in each of the multiple spatial dimensions; and determining the shape of the distribution area of ​​the point cloud within the voxel based on the multiple eigenvalues.

[0106] As an example, after obtaining multiple voxels, the domain controller calculates the covariance matrix of the point cloud in each voxel based on the point cloud in the voxel, and then calculates multiple eigenvalues ​​of the covariance matrix, wherein each eigenvalue is used to characterize the distribution span of the point cloud in the voxel in each spatial dimension.

[0107] In practical applications, assuming that the points in the multi-frame point cloud data are distributed in three spatial dimensions of X, Y and Z, the covariance matrix of the point cloud within the voxel is a 3×3 matrix with three eigenvalues.

[0108] The specific process of calculating the covariance matrix of the point cloud in the voxel is as follows: 1. Calculate the mean vector: average the three-dimensional coordinates (x, y, x) of all points in the voxel to obtain a mean vector, which represents the center position of the point cloud in the voxel. 2. Calculate the covariance: For each point in the voxel, calculate the difference between the three-dimensional coordinates of the point and the mean vector. Specifically, calculate the difference between the x-coordinate of the point and the x-coordinate of the mean vector, the difference between the y-coordinate of the point and the y-coordinate of the mean vector, and the difference between the z-coordinate of the point and the z-coordinate of the mean vector. Calculate the difference between the x-coordinate of the point and the y-coordinate of the mean vector, the difference between the x-coordinate of the point and the z-coordinate of the mean vector, and the difference between the y-coordinate of the point and the x-coordinate of the mean vector. The difference between the z coordinate of the point and the mean vector, the difference between the z coordinate of the point and the x coordinate of the mean vector, and the difference between the z coordinate of the point and the y coordinate of the mean vector. The variance and covariance are calculated based on these differences. The covariance represents the degree of joint change between two variables. For the point cloud in the voxel, it is necessary to calculate the variance of xx, yy and zz on the same coordinate axis (that is, the autocovariance on each coordinate axis), as well as the covariance between different coordinate axes of xy, xz and yz (that is, the covariance between different coordinate axes). 3. Construct a covariance matrix: Construct a 3×3 matrix based on the covariance of the point cloud in the voxel to obtain the covariance matrix of the point cloud in the voxel. In the covariance matrix of the point cloud in the voxel, the diagonal elements are the autocovariance on each coordinate axis, and the non-diagonal elements are the covariance between different coordinate axes.

[0109] The specific process of calculating the three eigenvalues ​​of the covariance matrix is ​​as follows: for the given covariance matrix A, there exists a non-zero vector v and a scalar λ, such that Av=λv, where λ is the eigenvalue to be solved. By substituting the elements in the covariance matrix A into the determinant |A-λI|=0, where I is the identity matrix, a cubic polynomial about λ is obtained, and the cubic polynomial is solved to obtain three solutions of λ, namely the three eigenvalues ​​of the covariance matrix A. The three eigenvalues ​​of the covariance matrix A respectively represent the distribution span of the point cloud in the voxel in the three spatial dimensions of X, Y and Z.

[0110] The calculation method of the covariance matrix and the eigenvalue of the covariance matrix can also refer to the existing calculation method of the covariance matrix and the eigenvalue of the covariance matrix, which will not be described in detail here.

[0111] After obtaining the multiple eigenvalues ​​of the covariance matrix, the domain controller determines the shape of the distribution area of ​​the point cloud in the voxel according to the multiple eigenvalues ​​of the covariance matrix, because each eigenvalue is used to characterize the distribution span of the point cloud in the voxel in each spatial dimension.

[0112] The embodiment of the present application determines multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, determines the shape of the distribution area of ​​the point cloud within the voxel according to the distribution span of the point cloud within the voxel in multiple spatial dimensions represented by the multiple eigenvalues, and can uniformly determine the distribution span of the point cloud within the voxel in multiple spatial dimensions, thereby ensuring that multiple feature voxels including line feature voxels and surface feature voxels are extracted quickly and accurately from multiple voxels.

[0113] In an optional embodiment, determining the shape of the distribution area of ​​the point cloud in the voxel according to the multiple eigenvalues ​​includes: calculating the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds in the voxel; and determining the shape of the distribution area of ​​the point cloud in the voxel according to the size relationship between the ratio and the span threshold. The second largest value refers to the second largest value among the multiple eigenvalues ​​that is only smaller than the maximum value.

[0114] As an example, considering the number of point clouds within a voxel, in order to more accurately reflect the distribution span of the point clouds within the voxel in multiple spatial dimensions, the ratio of each eigenvalue to the number of point clouds within the voxel can be used to characterize the distribution span of the point clouds within the voxel in each spatial dimension.

[0115] After determining multiple eigenvalues ​​of the covariance matrix of the point cloud in any voxel, the domain controller selects the second largest eigenvalue from the multiple eigenvalues ​​as the second largest value, counts the number of point clouds in the voxel, calculates the ratio of the second largest value to the number of point clouds in the voxel, and determines the shape of the distribution area of ​​the point cloud in the voxel based on the size relationship between the ratio and a preset span threshold. When the shape of the point cloud in the voxel is linear, the voxel is determined to be a line feature voxel, and the points in the line feature voxel are line feature points; when the shape of the point cloud in the voxel is planar, the voxel is determined to be a surface feature voxel, and the points in the surface feature voxel are surface feature points; when the shape of the point cloud in the voxel is neither linear nor planar, the voxel is determined to be neither a line feature voxel nor a surface feature voxel, and is a non-feature voxel.

[0116] The embodiment of the present application determines multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, counts the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds within the voxel, and determines the shape of the distribution area of ​​the point cloud within the voxel according to the size relationship between the ratio and the span threshold. This can more accurately determine the distribution span of the point cloud within the voxel in multiple spatial dimensions, thereby more accurately extracting multiple feature voxels including line feature voxels and surface feature voxels from multiple voxels.

[0117] In an optional embodiment, the span threshold includes a first span threshold and a second span threshold; determining the shape of the distribution area of ​​the point cloud within the voxel based on the size relationship between the ratio and the span threshold includes: when the ratio is less than the first span threshold, determining that the shape of the distribution area of ​​the point cloud within the voxel is close to a line; when the ratio is greater than the second span threshold, determining that the shape of the distribution area of ​​the point cloud within the voxel is close to a plane.

[0118] As an example, according to actual application requirements, a span threshold is preset, and the span threshold includes a first span threshold and a second span threshold.

[0119] After determining multiple eigenvalues ​​of the covariance matrix of the point cloud in any voxel, the domain controller selects the second eigenvalue with the largest value from the multiple eigenvalues ​​as the second largest value, counts the number of point clouds in the voxel, calculates the ratio of the second largest value to the number of point clouds in the voxel, and compares the ratio with the first span threshold, and compares the ratio with the second span threshold. If the ratio is less than the first span threshold, it means that the point cloud in the voxel has a larger distribution span in only one spatial dimension, that is, the spatial dimension corresponding to the maximum value of the multiple eigenvalues, and the point cloud in the voxel tends to be distributed in a single space. In terms of dimension, it is determined that the shape of the distribution area of ​​the point cloud in the voxel is close to a line; if the ratio is greater than the second span threshold, it means that the point cloud in the voxel has a larger distribution span in at least two spatial dimensions, that is, the spatial dimensions corresponding to the maximum and second largest values ​​of the multiple eigenvalues, and the point cloud in the voxel tends to be distributed on the plane formed by these two spatial dimensions, and it is determined that the shape of the distribution area of ​​the point cloud in the voxel is close to a plane; if the ratio is neither less than the first span threshold nor greater than the second span threshold, it is determined that the shape of the point cloud in the voxel is neither close to a linear shape nor close to a plane.

[0120] For example, after determining the three eigenvalues ​​of the covariance matrix of the point cloud in any voxel, the domain controller selects the second eigenvalue with the largest value from the three eigenvalues ​​as the second largest value, counts the number of point clouds in the voxel, calculates the ratio of the second largest value to the number of point clouds in the voxel, and compares the ratio with the first span threshold, and compares the ratio with the second span threshold, wherein the second span threshold is greater than the first span threshold. If the ratio is less than the first span threshold, it means that the point cloud in the voxel has a larger distribution span in only one spatial dimension, that is, the spatial dimension corresponding to the maximum value of the three eigenvalues, and the number of point clouds in the voxel is large. The point cloud tends to be distributed in a single spatial dimension, and the shape of the distribution area of ​​the point cloud in the voxel is determined to be close to a line; if the ratio is greater than the second span threshold, it means that the distribution span of the point cloud in the voxel in two spatial dimensions, that is, the spatial dimensions corresponding to the maximum and second largest values ​​of the three eigenvalues, is large, and the point cloud in the voxel tends to be distributed on the plane formed by these two spatial dimensions, and the shape of the distribution area of ​​the point cloud in the voxel is determined to be close to a plane; if the ratio is neither less than the first span threshold nor greater than the second span threshold, it is determined that the shape of the point cloud in the voxel is neither close to a linear nor close to a plane.

[0121] The embodiment of the present application determines multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel, then counts the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds within the voxel, and when the ratio is less than a first span threshold, determines that the shape of the distribution area of ​​the point cloud within the voxel is close to a line, and when the ratio is greater than a second span threshold, determines that the shape of the distribution area of ​​the point cloud within the voxel is close to a plane. The ratio of the second largest value to the number of point clouds within the voxel can be used to quickly infer the distribution span of the point cloud within the voxel in each spatial dimension, thereby more quickly extracting multiple feature voxels including linear feature voxels and surface feature voxels from multiple voxels.

[0122] In an optional embodiment, the screening of multiple feature voxels from multiple voxels includes: when a voxel is neither a line feature voxel nor a surface feature voxel, determining the voxel as a target voxel, and dividing the target voxel into multiple sub-voxels; for each sub-voxel among the multiple sub-voxels, if the shape of the distribution area of ​​the point cloud within the sub-voxel approaches a line or a plane, determining the sub-voxel as an extracted feature voxel, otherwise determining the sub-voxel as a non-feature voxel; when the sub-voxel is determined to be a non-feature voxel and the size of the sub-voxel is greater than or equal to a minimum size threshold, determining the sub-voxel as the next target voxel, and returning to execute the step of dividing the target voxel into multiple sub-voxels until the sizes of all non-feature voxels are less than the minimum size threshold.

[0123] As an example, when the domain controller determines that any voxel is neither a line feature voxel nor a surface feature voxel, it determines that the voxel is a target voxel, and divides the target voxel into multiple sub-voxels according to a preset sub-level size threshold.

[0124] In practical applications, when a voxel is a cubic region, the voxel may be directly divided into 8 sub-voxels, where the sub-voxel is a cubic region smaller than the voxel.

[0125] After obtaining multiple sub-voxels, the domain controller determines the shape of the distribution area of ​​the point cloud in each sub-voxel. When the shape of the distribution area of ​​the point cloud in the sub-voxel approaches a line, the sub-voxel is determined to be a line feature voxel, and the feature points in the line feature voxel are line feature points; when the shape of the distribution area of ​​the point cloud in the sub-voxel approaches a plane, the sub-voxel is determined to be a surface feature voxel, and the feature points in the surface feature voxel are surface feature points; when the shape of the distribution area of ​​the point cloud in the sub-voxel approaches neither a line nor a plane, the sub-voxel is determined to be neither a line feature voxel nor a surface feature voxel, and is a non-feature voxel.

[0126] When the domain controller determines that the sub-voxel is a non-feature voxel, it determines whether the size of the sub-voxel is smaller than a preset minimum size threshold. If the size of the sub-voxel is greater than or equal to the minimum size threshold, the sub-voxel is considered to be divisible and the sub-voxel is allowed to be divided. If the size of the sub-voxel is smaller than the minimum size threshold, the sub-voxel is considered to be indivisible and is directly discarded.

[0127] In practical applications, when the voxel is in a cube area and the size of the voxel is a·a·a, the minimum size threshold may be preset to d·d·d, where a and d are positive integers and a>d.

[0128] When the domain controller determines that the size of the sub-voxel is greater than or equal to the minimum size threshold, it determines that the sub-voxel is the next target voxel and returns to execute the step of dividing the target voxel into multiple sub-voxels until the size of all non-feature voxels is less than the minimum size threshold, and all non-feature voxels are eliminated.

[0129] The embodiment of the present application determines that any voxel is a target voxel when it is neither a line feature voxel nor a surface feature voxel, continues to divide the voxel into multiple sub-voxels, extracts multiple feature voxels from the multiple sub-voxels, and determines that a non-feature voxel in the multiple sub-voxels whose size is greater than or equal to a minimum size threshold is the next target voxel, and again divides and judges the shape of the point cloud distribution area until the recursive voxels are indivisible. The initial point cloud map can be more finely voxel-segmented, thereby ensuring that multiple feature voxels including line feature voxels and surface feature voxels are completely extracted from the multiple voxels.

[0130] In an optional embodiment, the position and posture errors of each feature point in the feature voxel are determined based on the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel, including: for each line feature point in the line feature voxel, the position and posture error of the line feature point is determined with the goal of minimizing the sum of the squares of the distances between the line feature point and the line on which the point cloud in the line feature voxel is located; for each surface feature point in the surface feature voxel, the position and posture error of the surface feature point is determined with the goal of minimizing the sum of the squares of the distances between the surface feature point and the plane on which the point cloud in the surface feature voxel is located.

[0131] As an example, after obtaining multiple feature voxels, the domain controller determines the pose error of each line feature point in each line feature voxel based on the algorithm idea of ​​the bundle adjustment algorithm, with the goal of minimizing the sum of the squares of the distances between the line feature point and the line on which the point cloud in the line feature voxel is located, thereby determining the pose errors of all line feature points in each line feature voxel.

[0132] Similarly, for each surface feature voxel, based on the algorithmic idea of ​​the bundle adjustment algorithm, for each surface feature point in the surface feature voxel, the pose error of the surface feature point is determined with the goal of minimizing the sum of the squares of the distances between the surface feature point and the plane where the point cloud in the surface feature voxel is located, thereby determining the pose errors of all surface feature points in each surface feature voxel.

[0133] In practical applications, in order to determine the pose errors of all surface feature points in each surface feature voxel based on the same coordinate system, after obtaining multiple feature voxels, the domain controller needs to first convert all feature points in each feature voxel to the global coordinate system, and then determine the pose errors of all feature points in each feature voxel.

[0134] For example, suppose the initial coordinates of each feature point in a feature voxel are i={1,2,...,N}, N is the total number of feature points in the feature voxel. These feature points come from M frames of point cloud data, but they all correspond to the same feature (plane or line). The i-th feature point From the j-th frame point cloud data Sj , j = {1,2,...,M}, M is the total number of frames of point cloud data, and the poses of M frames of point cloud data are R j is the j-th frame point cloud data S j The rotation matrix, t j To represent the j-th frame point cloud data S j The translation matrix of SO(3) represents the three-dimensional rotation group. Represents a three-dimensional real space, then the coordinate transformation formula is shown in formula (1):

[0135]

[0136] In formula (1), p i is the transformed coordinate of the i-th feature point.

[0137] Based on the algorithm idea of ​​the bundle adjustment algorithm, the domain controller determines the pose error of each line feature point in the line feature voxel with the goal of minimizing the sum of the squares of the distances between the line feature point and the line where the line feature point is located. The pose error of the line feature point is shown in formula (2):

[0138]

[0139] In formula (2), T * is the adjoint matrix of T, n 线 is the normal vector of the line where the line feature point is located, n 线 * n 线 The conjugate vector of 线 T n 线 The transposed vector, q 线 is the point on the line where the line feature point is located, q 线 * for q 线 The conjugate vector of, I is the unit matrix, p i线 is the transformed coordinate of the i-th line feature point, i 线 ={1,2,...,N 线},N 线 is the total number of line feature points within the line feature voxel.

[0140] Based on the algorithm idea of ​​the bundle adjustment algorithm, for each surface feature point in the surface feature voxel, the pose error of the surface feature point is determined with the goal of minimizing the sum of the squares of the distances between the surface feature point and the plane where the surface feature point is located. The pose error of the surface feature point is shown in formula (3):

[0141]

[0142] In formula (3), T * is the adjoint matrix of T, n 面 is the normal vector of the plane where the surface feature point is located, n 面 * n 面 The conjugate vector of 面 T n 面 The transposed vector, q 面 is the point on the plane where the surface feature point is located, q 面 * for q 面 The conjugate vector of, I is the unit matrix, p i面 is the transformed coordinate of the i-th face feature point, i 面 ={1,2,...,N 面},N 线 It is the total number of surface feature points within the surface feature voxel.

[0143] The embodiments of the present application determine the pose error of line feature points by minimizing the sum of the squares of the distances between line feature points and the lines on which they are located, and determine the pose error of surface feature points by minimizing the sum of the squares of the distances between surface feature points and the planes on which they are located. This can determine the pose error of line feature points based on point-to-line constraints, and can determine the pose error of surface feature points based on point-to-surface constraints, which is conducive to accurately constructing the pose error of feature points.

[0144] In an optional embodiment, determining the optimized pose of the point cloud data based on the pose errors of the feature points in the point cloud data includes: using the least squares method to solve the minimum value of the pose errors of the feature points in the point cloud data to obtain the optimized pose of the point cloud data.

[0145] As an example, after obtaining the pose errors of all feature points in each feature voxel, the domain controller determines the feature points in each frame of point cloud data, and uses the least squares method to solve the minimum value of the pose errors of the feature points in the frame of point cloud data to obtain the optimized pose of the frame of point cloud data, thereby obtaining the optimized pose of multiple frames of point cloud data.

[0146] It is understood that the least square method is a mathematical optimization technique that finds the best function match for data by minimizing the sum of squares of errors. The least square method can be used to easily obtain unknown data and minimize the sum of squares of errors between the obtained data and the actual data.

[0147] The embodiment of the present application obtains the optimized pose of each frame of point cloud data by using the least squares method to solve the minimum pose error of the feature points in the frame of point cloud data, so as to quickly and accurately obtain the optimized pose of multiple frames of point cloud data.

[0148] In an optional embodiment, after the target point cloud map is obtained by stitching each frame of point cloud data according to the optimized posture of each frame of point cloud data in the multi-frame point cloud data, it also includes: for each first target point in the target point cloud map, according to the positions of multiple first neighborhood points, determining the position of the first optimized point, and updating the position of the first target point to the position of the first optimized point, to obtain an optimized point cloud map; wherein the multiple first neighborhood points are within the neighborhood of the first target point; for each second target point in the optimized point cloud map, according to the weights and positions of the multiple second neighborhood points, determining the position of the second optimized point, and updating the position of the second target point to the position of the second optimized point, to obtain an optimal point cloud map; wherein the multiple second neighborhood points are within the neighborhood of the second target point.

[0149] As an example, in order to reduce errors such as target blur or wall thickening introduced by measurement errors of point cloud acquisition devices, such as radar itself, the domain controller can continue to filter the target point cloud map after obtaining the target point cloud map to obtain the optimal point cloud map.

[0150] In the point cloud filtering process, along with the point cloud noise being filtered out, sharp surfaces, window frames, and pillar corners and other slender point clouds are usually filtered out or rounded together, resulting in distortion of local details. In order to avoid distortion of local details during the filtering process, the domain controller can use a two-step filtering process. First, the target point cloud map is mean filtered to remove outliers and obtain an optimized point cloud map. Finally, the optimized point cloud map is weighted locally optimal projection (WLOP) filtered to obtain the optimal point cloud data.

[0151] After obtaining the target point cloud map, the domain controller determines, based on the algorithm idea of ​​the mean filtering algorithm, for each first target point in the target point cloud map, multiple first neighborhood points in the neighborhood of the first target point, determines the position of the first optimization point according to the positions of the multiple first neighborhood points, and updates the position of the first target point to the position of the first optimization point to obtain the optimized point cloud map.

[0152] In practical applications, the specific process of mean filtering the target point cloud map is as follows: First, based on the point cloud in the target point cloud map, a data structure is constructed, such as kd-tree. Kd-tree is the abbreviation of k-dimensional tree, which is a tree data structure that stores instance points in k-dimensional space for fast retrieval. Then, all points in the point cloud are traversed, and all points in the neighborhood of the current point are quickly searched through the constructed kd-tree using the set neighborhood radius. The average value of all points in the neighborhood is calculated as the coordinates of the current point. Finally, the target point cloud map is updated based on the latest coordinates of all points in the point cloud to obtain an optimized point cloud map.

[0153] The above mean filtering algorithm can also refer to the existing mean filtering algorithm, which will not be described in detail here.

[0154] After obtaining the optimized point cloud map, the domain controller determines multiple second neighborhood points in the neighborhood of the second target point for each second target point in the optimized point cloud map based on the algorithm idea of ​​the WLOP filtering algorithm, determines the position of the second optimized point according to the weights and positions of the multiple second neighborhood points, and updates the position of the second target point to the position of the second optimized point to obtain the optimal point cloud map.

[0155] In practical applications, the specific process of selecting appropriate parameters (the number of points K involved in filtering and the neighborhood radius) and performing WLOP filtering on the optimized point cloud map is as follows: 1. Initialization: Randomly select K points from the optimized point cloud map to build a data structure, such as kd-tree, and calculate the weight of each point according to the point cloud density. 2. Local optimization: Traverse each point, use the set neighborhood radius to search all points in the neighborhood of the current point quickly through the constructed kd-tree, and calculate a weighted local optimal point according to the weights of all points in the neighborhood. The weighted local optimal point is the point with the smallest weighted distance from all points in the neighborhood to the point, and the coordinates of the current point are updated to the coordinates of the weighted local optimal point. 3. Projection: After local optimization, for each point, the normal vectors of all points in the neighborhood of the point are used to determine whether the point is on a plane or a curved surface. According to the judgment result, the point is projected onto a suitable plane or curved surface to obtain the optimal point cloud map, so that the real details of the target scene can be restored without distortion of local details.

[0156] The above WLOP filtering algorithm may also refer to the existing WLOP filtering algorithm, which will not be described in detail here.

[0157] The embodiment of the present application obtains the optimal point cloud map by performing two-step filtering on the target point cloud map, which can reduce errors such as target blurring or wall thickening introduced by the measurement errors of the point cloud acquisition device itself, while avoiding distortion of local details, and further improving the accuracy of three-dimensional reconstruction.

[0158] Please see Figure 2 , Figure 2 A schematic diagram of the structure of a three-dimensional reconstruction device provided for the second embodiment of the present application. The second embodiment of the present application provides a three-dimensional reconstruction device, including: a feature voxel screening module 201, used to divide the initial point cloud map into multiple voxels, and screen multiple feature voxels from the multiple voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the multiple feature voxels include line feature voxels and surface feature voxels; a pose error determination module 202, used to determine the pose error of each feature point in the feature voxel for each feature voxel in the multiple feature voxels according to the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel; an optimized pose determination module 203, used to determine the optimized pose of the point cloud data for each frame of point cloud data in the multiple frames of point cloud data according to the pose error of the feature point in the point cloud data; a point cloud map reconstruction module 204, used to splice each frame of point cloud data according to the optimized pose of each frame of point cloud data in the multiple frames of point cloud data to obtain a target point cloud map.

[0159] In an optional embodiment, the feature voxel screening module 201 is also used to obtain multiple frames of point cloud data collected by the point cloud acquisition device before dividing the initial point cloud map into multiple voxels and screening multiple feature voxels from the multiple voxels; determine the initial pose of each frame of point cloud data according to the posture data collected by the inertial measurement unit at the time of collecting each frame of point cloud data in the multiple frames of point cloud data; and splice each frame of point cloud data according to the initial pose of each frame of point cloud data to obtain an initial point cloud map.

[0160] In an optional embodiment, the screening of multiple feature voxels from multiple voxels includes: for each voxel in the multiple voxels, determining the shape of the distribution area of ​​the point cloud in the voxel according to the distribution span of the point cloud in the voxel in multiple spatial dimensions; when the shape of the distribution area of ​​the point cloud in the voxel approaches a line, determining the voxel as a line feature voxel; when the shape of the distribution area of ​​the point cloud in the voxel approaches a plane, determining the voxel as a surface feature voxel.

[0161] In an optional embodiment, determining the shape of the distribution area of ​​the point cloud within the voxel based on the distribution span of the point cloud within the voxel in multiple spatial dimensions includes: determining multiple eigenvalues ​​of the covariance matrix of the point cloud within the voxel; wherein each eigenvalue of the multiple eigenvalues ​​is used to characterize the distribution span of the point cloud within the voxel in each of the multiple spatial dimensions; and determining the shape of the distribution area of ​​the point cloud within the voxel based on the multiple eigenvalues.

[0162] In an optional embodiment, determining the shape of the distribution area of ​​the point cloud within the voxel based on multiple eigenvalues ​​includes: counting the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds within the voxel; and determining the shape of the distribution area of ​​the point cloud within the voxel based on the size relationship between the ratio and a span threshold.

[0163] In an optional embodiment, the span threshold includes a first span threshold and a second span threshold; determining the shape of the distribution area of ​​the point cloud within the voxel based on the size relationship between the ratio and the span threshold includes: when the ratio is less than the first span threshold, determining that the shape of the distribution area of ​​the point cloud within the voxel is close to a line; when the ratio is greater than the second span threshold, determining that the shape of the distribution area of ​​the point cloud within the voxel is close to a plane.

[0164] In an optional embodiment, the screening of multiple feature voxels from multiple voxels includes: when a voxel is neither a line feature voxel nor a surface feature voxel, determining the voxel as a target voxel, and dividing the target voxel into multiple sub-voxels; for each sub-voxel among the multiple sub-voxels, if the shape of the distribution area of ​​the point cloud within the sub-voxel approaches a line or a plane, determining the sub-voxel as an extracted feature voxel, otherwise determining the sub-voxel as a non-feature voxel; when the sub-voxel is determined to be a non-feature voxel and the size of the sub-voxel is greater than or equal to a minimum size threshold, determining the sub-voxel as the next target voxel, and returning to execute the step of dividing the target voxel into multiple sub-voxels until the sizes of all non-feature voxels are less than the minimum size threshold.

[0165] In an optional embodiment, the position and posture errors of each feature point in the feature voxel are determined based on the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel, including: for each line feature point in the line feature voxel, the position and posture error of the line feature point is determined with the goal of minimizing the sum of the squares of the distances between the line feature point and the line on which the point cloud in the line feature voxel is located; for each surface feature point in the surface feature voxel, the position and posture error of the surface feature point is determined with the goal of minimizing the sum of the squares of the distances between the surface feature point and the plane on which the point cloud in the surface feature voxel is located.

[0166] In an optional embodiment, the point cloud map reconstruction module 204 is also used for, after stitching each frame of point cloud data according to the optimized posture of each frame of point cloud data in the multi-frame point cloud data to obtain the target point cloud map, for each first target point in the target point cloud map, determining the position of the first optimized point according to the positions of multiple first neighborhood points, and updating the position of the first target point to the position of the first optimized point to obtain an optimized point cloud map; wherein the multiple first neighborhood points are within the neighborhood of the first target point; for each second target point in the optimized point cloud map, determining the position of the second optimized point according to the weights and positions of the multiple second neighborhood points, and updating the position of the second target point to the position of the second optimized point to obtain an optimal point cloud map; wherein the multiple second neighborhood points are within the neighborhood of the second target point.

[0167] The implementation process of the functions and effects of each module in the above-mentioned device is specifically described in the implementation process of the corresponding steps in the above-mentioned method, which will not be repeated here.

[0168] Please see Figure 3 , Figure 3 The third embodiment of the present application provides an electronic device 30, comprising a processor 301, a memory 302, and a computer program stored in the memory 302 and configured to be executed by the processor 301; when the processor 301 executes the computer program, the method described in the first embodiment of the present application is implemented and the same beneficial effects can be achieved.

[0169] The processor 301 may implement the method described in the first embodiment of the present application when reading the computer program from the memory 302 via the bus 303 and executing the computer program.

[0170] Processor 301 can process digital signals and can include various computing structures, such as complex instruction set computer structure, reduced instruction set computer structure, or a structure that implements a combination of multiple instruction sets. In some examples, processor 301 can be a microprocessor.

[0171] The memory 302 may be used to store instructions executed by the processor 301 or data related to the execution of instructions. These instructions and / or data may include codes for implementing some functions or all functions of one or more modules described in the embodiments of the present application. The processor 301 of this embodiment may be used to execute instructions in the memory 302 to implement the method described in the first embodiment of the present application. The memory 302 includes a dynamic random access memory, a static random access memory, a flash memory, an optical memory, or other memory known to those skilled in the art.

[0172] The fourth embodiment of the present application provides a vehicle, including the electronic device described in the third embodiment of the present application, and can achieve the same beneficial effects as the third embodiment.

[0173] The fifth embodiment of the present application provides a computer program product, which includes instructions. When the instructions are executed by a computer, the computer implements the method described in the first embodiment of the present application and can achieve the same beneficial effects.

[0174] The method described in the first embodiment of the present application can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computer, the process or function described in each embodiment of the present application is executed in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device, a core network device, an OAM (Open Application Model) or other programmable device.

[0175] The computer program or instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instructions may be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it may also be an optical medium, such as a digital video disk; it may also be a semiconductor medium, such as a solid-state drive. The computer-readable storage medium may be a volatile or non-volatile storage medium, or may include both volatile and non-volatile types of storage media.

[0176] The sixth embodiment of the present application provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the method described in the first embodiment of the present application, and can achieve the same beneficial effects as the method described in the first embodiment of the present application.

[0177] In summary, the embodiments of the present application provide a three-dimensional reconstruction method, an electronic device, a vehicle and a computer program product, wherein the three-dimensional reconstruction method comprises: dividing an initial point cloud map into multiple voxels, and selecting multiple feature voxels from the multiple voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the multiple feature voxels include line feature voxels and surface feature voxels; for each feature voxel in the multiple feature voxels, determining the pose error of each feature point in the feature voxel according to the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel; for each frame of point cloud data in the multiple frames of point cloud data, determining the optimized pose of the point cloud data according to the pose error of the feature point in the point cloud data; and splicing each frame of point cloud data according to the optimized pose of each frame of point cloud data in the multiple frames of point cloud data to obtain a target point cloud map. The embodiment of the present application divides the initial point cloud map into multiple voxels, selects multiple feature voxels including line feature voxels and surface feature voxels from the multiple voxels, determines the pose error of each feature point in each feature voxel according to the positional relationship between each feature point in each feature voxel and the distribution area of ​​the point cloud in each feature voxel, determines the optimized pose of each frame of point cloud data according to the pose error of the feature point in each frame of point cloud data, and splices each frame of point cloud data according to the optimized pose of each frame of point cloud data to obtain a target point cloud map. The correspondence between point to line and point to surface can be used instead to accurately determine the correspondence between feature points and target objects to perform point cloud registration of adjacent frames of point cloud data, effectively reducing errors such as ghosting and offset introduced by relative pose errors between adjacent frames of point cloud data in the three-dimensional reconstruction process, thereby achieving accurate reconstruction of the target point cloud map.

[0178] The above embodiments are only described as examples. The various embodiments of the present application can be combined or nested with each other, and the present application is not limited to this.

[0179] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0180] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0181] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0182] The above description is only an embodiment of the present application and is not intended to limit the scope of protection of the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the scope of protection of the present application. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0184] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

Claims

1. A three-dimensional reconstruction method, characterized in that: include: The initial point cloud map is divided into a plurality of voxels, and a plurality of characteristic voxels are selected from the plurality of voxels; wherein the initial point cloud map is obtained based on the collected multi-frame point cloud data, and the plurality of characteristic voxels include line characteristic voxels and surface characteristic voxels; For each feature voxel of the plurality of feature voxels, determining a position and posture error of each feature point in the feature voxel according to a positional relationship between each feature point in the feature voxel and a distribution area of ​​a point cloud in the feature voxel; For each frame of point cloud data in the multiple frames of point cloud data, determining an optimized pose of the point cloud data according to a pose error of a feature point in the point cloud data; The target point cloud map is obtained by splicing each frame of point cloud data according to the optimized posture of each frame of point cloud data in the multiple frames of point cloud data.

2. The method according to claim 1, characterized in that Before dividing the initial point cloud map into a plurality of voxels and selecting a plurality of characteristic voxels from the plurality of voxels, the method further includes: Acquire the multiple frames of point cloud data collected by the point cloud collection device; Determine the initial posture of each frame of point cloud data according to the posture data collected by the inertial measurement unit at the collection time of each frame of point cloud data in the multiple frames of point cloud data; The point cloud data of each frame is spliced ​​according to the initial posture of each frame of point cloud data to obtain the initial point cloud map.

3. The method according to claim 1, characterized in that The step of screening a plurality of characteristic voxels from the plurality of voxels comprises: For each voxel of the plurality of voxels, determining a shape of a distribution area of ​​the point cloud within the voxel according to a distribution span of the point cloud within the voxel in a plurality of spatial dimensions; When the shape of the distribution area of ​​the point cloud in the voxel approaches a line, determining the voxel as the line feature voxel; When the shape of the distribution area of ​​the point cloud in the voxel approaches a plane, the voxel is determined to be the surface feature voxel.

4. The method according to claim 3, characterized in that The determining the shape of the distribution area of ​​the point cloud in the voxel according to the distribution span of the point cloud in the voxel in multiple spatial dimensions includes: Determine a plurality of eigenvalues ​​of a covariance matrix of the intra-voxel point cloud; wherein each eigenvalue of the plurality of eigenvalues ​​is used to characterize a distribution span of the intra-voxel point cloud in each of a plurality of spatial dimensions; The shape of the distribution area of ​​the point cloud in the voxel is determined according to the multiple eigenvalues.

5. The method according to claim 4, characterized in that The step of determining the shape of the distribution area of ​​the point cloud within the voxel according to the multiple eigenvalues ​​includes: Counting the ratio of the second largest value among the multiple eigenvalues ​​to the number of point clouds in the voxel; The shape of the distribution area of ​​the point cloud in the voxel is determined according to the size relationship between the ratio and the span threshold.

6. The method according to claim 5, characterized in that The span threshold comprises a first span threshold and a second span threshold; The determining the shape of the distribution area of ​​the point cloud in the voxel according to the size relationship between the ratio and the span threshold value includes: When the ratio is less than the first span threshold, determining that the shape of the distribution area of ​​the point cloud in the voxel is close to a line; When the ratio is greater than the second span threshold, it is determined that the shape of the distribution area of ​​the point cloud in the voxel approaches a plane.

7. The method according to claim 3, characterized in that The step of screening a plurality of characteristic voxels from the plurality of voxels comprises: In the case where the voxel is neither the line feature voxel nor the surface feature voxel, determining the voxel as a target voxel, and dividing the target voxel into a plurality of sub-voxels; For each sub-voxel of the plurality of sub-voxels, if the shape of the distribution area of ​​the point cloud in the sub-voxel is close to a line or a plane, the sub-voxel is determined to be an extracted feature voxel, otherwise the sub-voxel is determined to be a non-feature voxel; When it is determined that the sub-voxel is a non-feature voxel and the size of the sub-voxel is greater than or equal to the minimum size threshold, the sub-voxel is determined to be the next target voxel, and the step of dividing the target voxel into multiple sub-voxels is returned to execute until the size of all non-feature voxels is less than the minimum size threshold.

8. The method according to any one of claims 1 to 7, characterized in that: The determining, according to the positional relationship between each feature point in the feature voxel and the distribution area of ​​the point cloud in the feature voxel, the position and posture error of each feature point in the feature voxel comprises: For each line feature point in the line feature voxel, determining a position and posture error of the line feature point with the goal of minimizing the sum of squares of distances between the line feature point and the line on which the point cloud in the line feature voxel is located; For each surface feature point in the surface feature voxel, the position and posture error of the surface feature point is determined with the goal of minimizing the sum of squares of distances between the surface feature point and a plane where the point cloud in the surface feature voxel is located.

9. The method according to any one of claims 1 to 7, characterized in that: After stitching the frames of point cloud data according to the optimized poses of the frames of point cloud data in the multiple frames of point cloud data to obtain the target point cloud map, the method further includes: For each first target point in the target point cloud map, determine the position of a first optimization point according to the positions of a plurality of first neighborhood points, and update the position of the first target point to the position of the first optimization point, so as to obtain an optimized point cloud map; wherein the plurality of first neighborhood points are within the neighborhood of the first target point; For each second target point in the optimized point cloud map, the position of the second optimized point is determined according to the weights and positions of the plurality of second neighborhood points, and the position of the second target point is updated to the position of the second optimized point to obtain an optimal point cloud map; wherein the plurality of second neighborhood points are within the neighborhood of the second target point.

10. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor; when the processor executes the computer program, the method according to any one of claims 1 to 9 is implemented.

11. A vehicle, characterized in that: Comprising the electronic device according to claim 10.

12. A computer program product, characterized in that The computer program product comprises instructions which, when executed by a computer, cause the computer to implement the method according to any one of claims 1 to 9.

Images