Three-dimensional perception and reconstruction method, electronic device, and storage medium

Abstract

The embodiment of the application discloses a three-dimensional perception and reconstruction method, equipment and a storage medium, relates to the technical field of three-dimensional reconstruction, and is invented to improve the accuracy of three-dimensional reconstruction. Current frame data collected by all sensors is acquired, and the current frame data collected by the sensors is preprocessed to obtain preprocessed current data; the preprocessed current data is read, input into a feature perception model, and key feature points of the preprocessed current data are extracted to obtain a feature vector; feature vectors of reference frames in all sensors are read, and the feature vectors of each frame data of all sensors that are read are spliced into global feature data through a rigid transformation matrix; the global feature data is read, and the global feature data is post-processed and visualized. The embodiment of the application is used for three-dimensional reconstruction in environmental perception.

Description

Technical Field

[0001] This invention relates to the field of sensor fusion technology, and in particular to a three-dimensional perception and reconstruction method, electronic device and storage medium. Background Technology

[0002] As a vital strategic energy resource, coal mine safety and intelligent operation are of paramount importance. With the advancement of policies such as the "Intelligent Mine Construction Standards," unmanned underground operations (such as driverless mining trucks and intelligent tunneling equipment) are becoming a development trend. Three-dimensional perception and reconstruction of underground roadways are the core foundation for achieving autonomous operation and intelligent obstacle avoidance of unmanned equipment, playing a crucial role in improving the safety, visualization, and intelligence levels of coal mining. According to data from the China Coal Industry Association, by 2024, my country had over 1,200 intelligent mining faces. However, problems such as insufficient perception accuracy and low modeling efficiency caused by the complex underground environment still constrain further improvements in the level of intelligence.

[0003] However, the harshness of the underground coal mine environment far exceeds that of surface environments: the tunnel cross-sections are mostly trapezoidal or rectangular, with narrow spaces that easily obstruct the view; there is no natural light underground, relying on localized explosion-proof lighting fixtures, with light intensity generally below 50 Lux and uneven light distribution; at the same time, the presence of flammable and explosive gases such as methane underground places strict requirements on the explosion-proof rating of sensors (must reach Ex d I Mb level or above), further limiting the selection and deployment of high-performance sensors. These characteristics collectively lead to a triple challenge for 3D perception and reconstruction: "unclear visibility, inaccurate measurement, and incomplete stitching."

[0004] Although existing technologies have achieved certain results in 3D perception and reconstruction of coal mine roadways, the following significant shortcomings still exist: First, the sensor deployment of existing solutions lacks specificity and does not take into account the constraints of underground space and explosion-proof requirements. Single or unreasonable sensor layouts are prone to blind spots, making it impossible to achieve full coverage scanning of the roadway roof, floor, and sidewalls, affecting the integrity of the 3D model, which in turn leads to defects in the data source for data fusion and reduces the fusion effect.

[0005] Secondly, traditional point cloud registration methods often rely on a single algorithm. For example, the ICP algorithm is sensitive to the initial position and has low iteration efficiency, while the NDT algorithm has insufficient matching accuracy in low-texture areas and lacks robust design for downhole repeating structures and dynamic interference, resulting in large stitching errors and low efficiency. They have failed to be deeply integrated with data fusion technology to improve registration stability.

[0006] Furthermore, existing technologies are not perfect for preprocessing raw point cloud data in downhole. They do not design multi-level filtering strategies that take into account the characteristics of dust and noise. Point cloud noise and redundant data affect the accuracy of subsequent feature extraction and registration. At the same time, the lack of efficient data compression methods leads to excessively long processing times for large-scale point cloud data, making it difficult to meet the needs of real-time modeling and also restricting the efficiency of data fusion. Summary of the Invention

[0007] In view of the above problems, embodiments of the present invention provide a three-dimensional perception and reconstruction method, electronic device and computer storage medium based on multi-sensor fusion, which can realize pixel-level fusion of multiple sensors and improve the accuracy of target recognition.

[0008] In a first aspect, embodiments of the present invention provide a three-dimensional perception and reconstruction method, comprising: Acquire the current frame data collected by all sensors, and preprocess the current frame data collected by the sensors to obtain the preprocessed current data; Read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain a feature vector; Read the feature vectors of the reference frames from all sensors, and then use a rigid transformation matrix to concatenate the feature vectors of each frame of data from all sensors into global feature data. Read the global feature data, post-process the global feature data, and visualize it.

[0009] In one implementation, the step of acquiring current frame data from all sensors and preprocessing the current frame data to obtain preprocessed current data includes: Read the current data collected by the laser sensor and denoise the current data using adaptive median filtering; The current data after denoising is divided into spatial voxels based on a preset voxel unit value, and the average coordinate of all data points in each spatial voxel is calculated to generate a representative point corresponding to each spatial voxel.

[0010] In one embodiment, the step of acquiring current frame data collected by all sensors and preprocessing the current frame data collected by the sensors to obtain preprocessed current data further includes: The current data collected by the vision sensor is acquired, and the current data is denoised by Gaussian filtering to obtain the denoised current data. The denoised current data is converted to grayscale to obtain the preprocessed current data.

[0011] In one implementation, the step of reading the preprocessed current data, inputting it into a feature-aware model, and extracting key feature points from the preprocessed current data to obtain a feature vector includes: Read the preprocessed current data and detect key points in the current data that have scale invariance; Each key point in the current data is converted into a vector descriptor of a specified dimension; Vector descriptors that meet the filtering rules are selected, and the filtered vector descriptors are merged to obtain the feature vector.

[0012] In one implementation, the step of reading the preprocessed current data, inputting it into a feature-aware model, and extracting key feature points from the preprocessed current data to obtain a feature vector further includes: Read the preprocessed current data, calculate the adjacent region of each data point in the preprocessed current data with a preset radius value, and obtain the neighborhood point set corresponding to each data point in the preprocessed current data; For each data point in the neighborhood point set, calculate the covariance matrix, and perform eigenvalue decomposition on the covariance matrix of each data point to select an initial feature point set; and perform non-maximum suppression on the initial feature point set to obtain key feature points. Read the key feature points in each of the neighborhood point sets and generate a feature histogram for each of the key feature points in the neighborhood point sets; perform weighted aggregation of the feature histograms of the key feature points in each of the neighborhood point sets and the feature histograms of the key feature points in the adjacent neighborhood point sets to obtain the feature vector of each data point in the current data.

[0013] In one implementation, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data using a rigid transformation matrix includes: The first frame of data collected is selected as the reference frame, a global coordinate system is established, and the feature vector of each data point in the reference frame is used as the target feature vector. The nearest neighbor matching algorithm is used to calculate the similarity between the feature vector of each data point in each frame of data and the feature vector of each data point in the reference frame. The data points with the highest similarity form a feature matching pair, thus obtaining a set of feature matching pairs for each frame of data. Feature matching pairs with set values ​​are extracted to form an sampling set; and an initial rigid transformation model is calculated based on the sampling set to obtain the initial rotation matrix and the initial translation vector. By traversing all feature matching pairs and minimizing the objective function, a coarse registration rigid transformation model is generated, yielding the coarse registration rotation matrix and the coarse registration translation vector.

[0014] Secondly, embodiments of the present invention provide an apparatus comprising: The data preprocessing module is used to acquire the current frame data collected by the sensor and preprocess the current frame data collected by the sensor to obtain the preprocessed current data. The feature extraction module is used to read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain a feature vector. The feature concatenation module is used to read the feature vectors of the previous frame data and concatenate the feature vectors of each frame data into global feature data through a rigid transformation matrix. The feature post-processing module is used to read the global feature data, post-process the global feature data, and visualize it.

[0015] Thirdly, a computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the three-dimensional perception and reconstruction method described in any of the foregoing embodiments.

[0016] The embodiments of the present invention provide a three-dimensional perception and reconstruction method, equipment, and computer-readable storage medium. By proposing a three-dimensional perception and reconstruction method for underground coal mine roadways based on multi-sensor fusion, a multi-sensor collaborative hardware system is constructed to achieve full coverage of the roadway. A multi-level preprocessing process is designed to lay a high-quality data foundation. A two-stage point cloud stitching strategy is proposed to improve registration accuracy. At the same time, multi-sensor pixel-level fusion is achieved to improve the accuracy of three-dimensional reconstruction. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating a three-dimensional perception and reconstruction method provided for an embodiment of the present invention; Figure 2 A schematic diagram showing the comparison of data preprocessing before and after, provided for embodiments of the present invention; Figure 3 This is a schematic diagram of a module in an apparatus provided for an embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0021] Example 1 In a first aspect, embodiments of the present invention provide a three-dimensional perception and reconstruction method. By proposing a three-dimensional perception and reconstruction method for underground coal mine roadways based on multi-sensor fusion, a multi-sensor collaborative hardware system is constructed to achieve full coverage of the roadway. A multi-level preprocessing process is designed to lay a high-quality data foundation. A two-stage point cloud stitching strategy is proposed to improve registration accuracy. At the same time, multi-sensor pixel-level fusion is achieved to improve the accuracy of three-dimensional reconstruction.

[0022] like Figure 1 As shown, the three-dimensional perception and reconstruction method provided by the embodiments of the present invention includes: Step S10: Obtain the current frame data collected by all sensors, and preprocess the current frame data collected by the sensors to obtain the preprocessed current data; Step S20: Read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain the feature vector; Step S30: Read the feature vectors of the reference frames from all sensors, and use a rigid transformation matrix to concatenate the feature vectors of each frame of data from all sensors into global feature data. Step S40: Read the global feature data, post-process the global feature data, and visualize it.

[0023] In step S10, acquiring the current frame data collected by all sensors and preprocessing the current frame data collected by the sensors to obtain preprocessed current data includes: Read the current data collected by the laser sensor and denoise the current data using adaptive median filtering; The current data after denoising is divided into spatial voxels based on a preset voxel unit value, and the average coordinate of all data points in each spatial voxel is calculated to generate a representative point corresponding to each spatial voxel.

[0024] Specifically, preprocessing the current frame data acquired by the sensor, when preprocessing the point cloud data acquired by the lidar sensor, may include: Straight-through filtering to remove ground points: By utilizing the elevation difference between the tunnel floor and the main structure, an elevation threshold is set (e.g., 0.5m above the tunnel floor is the threshold) to quickly eliminate redundant ground points, reduce computational burden and interference, and avoid the influence of ground points on geometric feature extraction in data fusion.

[0025] Adaptive median filtering for denoising: The initial neighborhood size d is determined by the median nearest neighbor distance of the point cloud. Simulated Gaussian noise is added to optimize the RMSE threshold. Points within the neighborhood of each point are sorted and the median is output to suppress outlier noise. The RMSE value between the denoised point cloud and the original point cloud is calculated after filtering, using the following formula: Where (x0, y0, z0) are points in the original point cloud, (x, y, z) are the coordinates of the denoised point cloud, and m is the number of median filter points to ensure that the quality of the denoised point cloud meets the data fusion requirements.

[0026] Octree voxel downsampling: Construct an octree to divide the space into several voxels (e.g., voxel size is set to 0.1m). Calculate the average coordinates of all points within each voxel V to generate representative points, as shown in the following formula: Where |V| represents the number of points within voxel V. This method effectively preserves the overall geometric features of the point cloud while reducing the amount of data, and its measured results are as follows. Figure 2 As shown.

[0027] In another implementation, the step of acquiring current frame data from all sensors and preprocessing the current frame data to obtain preprocessed current data further includes: The current data collected by the vision sensor is acquired, and the current data is denoised by Gaussian filtering to obtain the denoised current data. The denoised current data is converted to grayscale to obtain the preprocessed current data.

[0028] Specifically, a two-dimensional vision sensor is used to acquire images, and preprocessing such as filtering and grayscale conversion is performed to obtain the preprocessed current data.

[0029] In step S20, the preprocessed current data is read, and the adjacency region of each data point in the preprocessed current data is calculated with a preset radius value to obtain the neighborhood point set corresponding to each data point in the preprocessed current data. For each data point in the neighborhood point set, calculate the covariance matrix, and perform eigenvalue decomposition on the covariance matrix of each data point to select an initial feature point set; and perform non-maximum suppression on the initial feature point set to obtain key feature points. Read the key feature points in each of the neighborhood point sets and generate a feature histogram for each of the key feature points in the neighborhood point sets; perform weighted aggregation of the feature histograms of the key feature points in each of the neighborhood point sets and the feature histograms of the key feature points in the adjacent neighborhood point sets to obtain the feature vector of each data point in the current data.

[0030] Specifically, after reading the preprocessed current data, the ISS algorithm is used to extract feature points, which may include: constructing a local neighborhood for each point, and processing each point in the preprocessed point cloud data. (i=1,2,...,N, where N is the total number of point clouds), construct the local neighborhood of each point; within the local neighborhood of each point, use the k-nearest neighbor search method to find all nearest neighbors of each point within the set neighborhood radius, forming a neighborhood point set. .

[0031] For each neighborhood point set Calculate the covariance matrix First, obtain the neighborhood point set. The center of mass, Based on the decentralized neighborhood points, the covariance matrix is ​​calculated using the following formula: By analyzing the covariance matrix Perform eigenvalue decomposition to select an initial set of feature points that meet the criteria.

[0032] Non-maximum suppression is applied to the initial feature point set to remove salient points within local regions, resulting in the key feature point set. .

[0033] For the obtained key feature point set The system generates local feature histograms, which can quantitatively describe the shape features of a point's local neighborhood and have a certain degree of rigidity invariance. Finally, it outputs a feature vector with a fixed dimension.

[0034] In another implementation, the step of reading the preprocessed current data, inputting it into a feature-aware model, and extracting key feature points from the preprocessed current data to obtain a feature vector includes: Read the preprocessed current data and detect key points in the current data that have scale invariance; Each key point in the current data is converted into a vector descriptor of a specified dimension; Vector descriptors that meet the filtering rules are selected, and the filtered vector descriptors are merged to obtain the feature vector.

[0035] Specifically, the SIFT algorithm is used to detect keypoints in the preprocessed data. Keypoints include coordinate, scale, and orientation information. Then, a SIFT descriptor of a set dimension is generated for each keypoint. If the filtering rule is set as: keypoint response value > threshold, features located in the middle region of the image can be retained as feature descriptors within this range.

[0036] In step S30, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data using a rigid transformation matrix includes: The first frame of data collected is selected as the reference frame, a global coordinate system is established, and the feature vector of each data point in the reference frame is used as the target feature vector. The nearest neighbor matching algorithm is used to calculate the similarity between the feature vector of each data point in each frame of data and the feature vector of each data point in the reference frame. The data points with the highest similarity form a feature matching pair, thus obtaining a set of feature matching pairs for each frame of data. Feature matching pairs with set values ​​are extracted to form an sampling set; and an initial rigid transformation model is calculated based on the sampling set to obtain the initial rotation matrix and the initial translation vector. By traversing all feature matching pairs and minimizing the objective function, a coarse registration rigid transformation model is generated, yielding the coarse registration rotation matrix and the coarse registration translation vector.

[0037] In one implementation, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data through a rigid transformation matrix includes: Read the feature vector acquired by the vision sensor, perform minimum nearest neighbor matching between the global feature vector and the feature vector of the vision sensor, and select the data points that are paired between the feature vector of the vision sensor and the global feature data.

[0038] In another implementation, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data through a rigid transformation matrix further includes: Using the coarse registration rigid transformation model as the initial value, transform each feature matching pair, calculate the distance error of each feature matching pair, and take the average distance error as the current global error; If the change between the current global error and the previous global error is within the set error threshold, the iteration converges and the rigid transformation model is stopped from being updated.

[0039] Furthermore, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data through a rigid transformation matrix also includes: Using the coarse registration rigid transformation model as the initial value, transform each feature matching pair, calculate the distance error of each feature matching pair, and take the average distance error as the current global error; If the change between the current global error and the previous global error is greater than the set error threshold, then update the current rigid transformation model. The updated rigid transformation model is used to transform each feature matching pair, and the distance error of each feature matching pair is calculated. The updated average distance error is then taken as the updated global error.

[0040] Furthermore, the step of reading the feature vectors of the reference frames from all sensors and concatenating the feature vectors of each frame of data from all sensors into global feature data through a rigid transformation matrix also includes: After the rigid transformation model stops updating, obtain the target rigid transformation model; The target rigid transformation model is used to transform the feature matching pairs in each frame of data read to obtain feature matching pairs aligned with the reference frame. Each transformed frame of data is concatenated with the base frame to obtain global feature data.

[0041] Specifically, matching point pairs are established based on the similarity metric of FPFH feature descriptors (e.g., Euclidean distance less than a threshold). Secondary filtering is performed by combining the similarity of visual texture features (e.g., SIFT feature matching distance less than a threshold). The total least squares (TLS) algorithm is then used to solve for the rigid body transformation parameters (rotation matrix R and translation vector t), achieving preliminary alignment of multi-view point clouds. The rigid body transformation model is as follows: In the formula, ϵi represents the error term in the point cloud registration process, reflecting interference such as data noise and corresponding point matching residuals. By minimizing the error (e.g., using the ICP algorithm to iteratively optimize R and t), accurate estimation of the point cloud pose parameters R and t is achieved, completing the spatial alignment of point clouds from different viewpoints. A commonly used error minimization objective function is the sum of squared Euclidean distances between corresponding points: By optimizing the objective function described above, the optimal rotation matrix and translation vector can be obtained. Specifically, this can be achieved through the following steps: Calculate the centroids of the two point clouds: Calculate the centroid coordinates: Construct the covariance matrix: Perform singular value decomposition (SVD) on H: Calculate the rotation matrix: If det(R) = -1, the solution needs to be corrected (for example, by inverting the last column of V).

[0042] Calculate the translation vector: To systematically evaluate the performance of the multi-radar point cloud stitching algorithm, this study uses RMSE, Mean Squared Error (MSE), and Mean Absolute Error (MAE) as core quantitative indicators.

[0043] The step of reading the global feature data, post-processing the global feature data, and outputting it includes: Read the global feature data, calculate and delete the points in the global feature data whose Euclidean distance is less than a preset threshold, and obtain the deduplicated global feature data. Obtain the normal vector and curvature of each data point in the deduplicated global feature data; The cavity region is calculated based on the normal vector and curvature of each data point; Based on the normal vector, curvature, and coordinates of neighboring points in the cavity region, interpolation is performed in the cavity region to generate data points with uniform density, thereby obtaining the repaired global feature data. The repaired global feature data is output and visualized.

[0044] Example 2 This invention also provides an apparatus, such as... Figure 3 As shown, it includes: The data preprocessing module is used to acquire the current frame data collected by the sensor and preprocess the current frame data collected by the sensor to obtain the preprocessed current data. The feature extraction module is used to read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain a feature vector. The feature concatenation module is used to read the feature vectors of the previous frame data and concatenate the feature vectors of each frame data into global feature data through a rigid transformation matrix. The feature post-processing module is used to read the global feature data, post-process the global feature data, and visualize it.

[0045] This paper proposes a method for 3D perception and reconstruction of underground coal mine roadways based on multi-sensor fusion. It constructs a multi-sensor collaborative hardware system to achieve full coverage of the roadway, designs a multi-level preprocessing process to lay a high-quality data foundation, proposes a two-stage point cloud stitching strategy to improve registration accuracy, and realizes pixel-level fusion of multiple sensors to improve the accuracy of 3D reconstruction.

[0046] Example 3 This invention also provides a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement the three-dimensional perception and reconstruction method described in any of the foregoing embodiments.

[0047] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0048] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A three-dimensional perception and reconstruction method, characterized in that, include: Acquire the current frame data collected by all sensors, and preprocess the current frame data collected by the sensors to obtain the preprocessed current data; Read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain a feature vector; Read the feature vectors of the reference frames from all sensors, and then use a rigid transformation matrix to concatenate the feature vectors of each frame of data from all sensors into global feature data. Read the global feature data, post-process the global feature data, and visualize it.

2. The three-dimensional perception and reconstruction method according to claim 1, characterized in that, The process of acquiring current frame data from all sensors and preprocessing the current frame data to obtain preprocessed current data includes: Read the current data collected by the laser sensor and denoise the current data using adaptive median filtering; The current data after denoising is divided into spatial voxels based on a preset voxel unit value, and the average coordinate of all data points in each spatial voxel is calculated to generate a representative point corresponding to each spatial voxel.

3. The three-dimensional perception and reconstruction method according to claim 1, characterized in that, The step of acquiring current frame data from all sensors and preprocessing the current frame data to obtain preprocessed current data further includes: The current data collected by the vision sensor is acquired, and the current data is denoised by Gaussian filtering to obtain the denoised current data. The denoised current data is converted to grayscale to obtain the preprocessed current data.

4. The three-dimensional perception and reconstruction method according to claim 1, characterized in that, The process of reading the preprocessed current data, inputting it into the feature perception model, and extracting the key feature points of the preprocessed current data to obtain a feature vector includes: Read the preprocessed current data and detect key points in the current data that have scale invariance; Each key point in the current data is converted into a vector descriptor of a specified dimension; Vector descriptors that meet the filtering rules are selected, and the filtered vector descriptors are merged to obtain the feature vector.

5. The three-dimensional perception and reconstruction method according to claim 1, characterized in that, The step of reading the preprocessed current data, inputting it into the feature perception model, and extracting the key feature points of the preprocessed current data to obtain a feature vector further includes: Read the preprocessed current data, calculate the adjacent region of each data point in the preprocessed current data with a preset radius value, and obtain the neighborhood point set corresponding to each data point in the preprocessed current data; For each data point in the neighborhood point set, calculate the covariance matrix, and perform eigenvalue decomposition on the covariance matrix of each data point to select an initial feature point set; and perform non-maximum suppression on the initial feature point set to obtain key feature points. Read the key feature points in each of the neighborhood point sets and generate a feature histogram for each of the key feature points in the neighborhood point sets; perform weighted aggregation of the feature histograms of the key feature points in each of the neighborhood point sets and the feature histograms of the key feature points in the adjacent neighborhood point sets to obtain the feature vector of each data point in the current data.

6. The three-dimensional perception and reconstruction method according to claim 1, characterized in that, The process involves reading the feature vectors of the reference frames from all sensors, and then concatenating the feature vectors of each frame of data from all sensors into global feature data using a rigid transformation matrix. This includes: The first frame of data collected is selected as the reference frame, a global coordinate system is established, and the feature vector of each data point in the reference frame is used as the target feature vector. The nearest neighbor matching algorithm is used to calculate the similarity between the feature vector of each data point in each frame of data and the feature vector of each data point in the reference frame. The data points with the highest similarity form a feature matching pair, thus obtaining a set of feature matching pairs for each frame of data. Feature matching pairs with set values ​​are extracted to form an sampling set; and an initial rigid transformation model is calculated based on the sampling set to obtain the initial rotation matrix and the initial translation vector. By traversing all feature matching pairs and minimizing the objective function, a coarse registration rigid transformation model is generated, yielding the coarse registration rotation matrix and the coarse registration translation vector.

7. A piece of equipment, characterized in that, The data preprocessing module is used to acquire the current frame data collected by the sensor and preprocess the current frame data collected by the sensor to obtain the preprocessed current data. The feature extraction module is used to read the preprocessed current data, input it into the feature perception model, and extract the key feature points of the preprocessed current data to obtain a feature vector. The feature concatenation module is used to read the feature vectors of the previous frame data and concatenate the feature vectors of each frame data into global feature data through a rigid transformation matrix. The feature post-processing module is used to read the global feature data, post-process the global feature data, and visualize it.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs that can be executed by one or more processors to implement the three-dimensional perception and reconstruction method according to any one of claims 1 to 6.

Images