Radar and visual data fusion three-dimensional reconstruction method and system based on external rotating shaft

Abstract

The invention relates to the technical field of three-dimensional reconstruction, and provides a radar and visual data fusion three-dimensional reconstruction method and system based on an external rotating shaft, and the method comprises the steps: driving a radar to rotate by 360 degrees through a control motor, and synchronously collecting radar point cloud data and camera image data; according to the rotation angle of the motor and the timestamp of the collected data, space coordinate conversion is carried out on each frame of point cloud data; splicing the point cloud data based on a three-dimensional rigid body motion model in combination with a rotation matrix and spatial translation to obtain a three-dimensional point cloud picture; and projecting the three-dimensional point cloud image and the image feature points to a unit spherical surface, optimizing external parameters by using a least square method, and coloring the point cloud to generate colored point cloud data. Through high-precision synchronous control, a precise point cloud splicing technology, an innovative coloring method and an optimized system architecture, the technical problem that an existing point cloud splicing technology is poor in precision is successfully solved, the precision of three-dimensional reconstruction is improved, and meanwhile real-time performance and stability are achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional reconstruction, in particular to a radar and visual data fusion three-dimensional reconstruction method and system based on an external rotating shaft. BACKGROUND

[0002] In recent years, the collection and reconstruction technology of three-dimensional point cloud data has been widely applied in virtual reality, autonomous driving, cultural heritage protection, construction engineering and other fields. At present, the three-dimensional reconstruction technology based on radar and visual data fusion as a new research direction has made certain progress. Existing technologies are usually divided into two categories: one is the fusion technology based on laser radar (LiDAR) and vision, and the other is the fusion technology based on other sensors such as ultrasonic sensors and cameras.

[0003] Point cloud stitching is one of the core technologies in three-dimensional reconstruction. In order to realize seamless stitching of multiple frames of point cloud, existing technologies usually rely on traditional feature matching or optimization-based registration algorithms. However, due to the sparsity and noise of point cloud data, the accuracy and stability of point cloud stitching are poor, especially in large-scale data stitching, the problems of inaccurate point cloud registration and insufficient reconstruction accuracy often occur.

[0004] In order to solve this problem, some technologies introduce IMU (Inertial Measurement Unit) to assist point cloud stitching. IMU can provide real-time attitude and acceleration information to help improve the spatial positioning and registration accuracy of point cloud. However, the introduction of IMU can improve the positioning accuracy, but also brings additional computational burden. The data of IMU needs to be fused with radar or visual data in real time, which greatly increases the amount of calculation, especially in the process of high-frequency data collection and processing.

[0005] Laser radar (LiDAR) provides high-precision point cloud data, but lacks color and texture information, affecting the visual effect of three-dimensional reconstruction. Therefore, point cloud is usually fused with camera image data for coloring. However, due to the differences in perspective, resolution and distortion between radar and camera, the matching of point cloud and image is often not accurate, resulting in coloring errors. In addition, the time and space synchronization problems of radar and camera are particularly prominent in high dynamic scenes, further reducing the accuracy of point cloud coloring and the quality of three-dimensional reconstruction.

[0006] The above problems directly affect the performance of the three-dimensional reconstruction system based on radar and visual fusion in practical applications, so there is an urgent need for new technical solutions to improve the accuracy, real-time performance and stability of the system. SUMMARY

[0007] In view of this, the application provides a radar and visual data fusion three-dimensional reconstruction method and system based on an external rotating shaft, which improves the synchronization control accuracy, optimizes the point cloud splicing algorithm, improves the point cloud coloring accuracy, and efficiently processes data, thereby solving the problems of poor synchronization accuracy, inaccurate splicing, large calculation amount, and sparse point cloud in the prior art, and improving the accuracy, real-time performance, and stability of the three-dimensional reconstruction system.

[0008] In one aspect to achieve the above object, the application provides a radar and visual data fusion three-dimensional reconstruction method based on an external rotating shaft, comprising the following steps:

[0009] Controlling the motor to drive the radar to rotate 360 degrees and synchronously collecting radar point cloud data and camera image data;

[0010] According to the motor rotation angle and the timestamp of the collected data, performing spatial coordinate conversion on each frame of point cloud data;

[0011] Based on a three-dimensional rigid body motion model, combining a rotation matrix and spatial translation, splicing the point cloud data to obtain a three-dimensional point cloud map;

[0012] Projecting the three-dimensional point cloud map and image feature points onto a unit sphere and performing point cloud coloring using the least squares method to generate color point cloud data.

[0013] Further, the process of synchronously collecting radar point cloud data and camera image data comprises:

[0014] Generating a pulse signal through an STM32 single-chip microcomputer to control the motor to rotate according to a preset angle, and simultaneously performing angle control based on a 51 harmonic reducer and a 16 subdivision drive.

[0015] Further, the process of performing spatial coordinate conversion on each frame of point cloud data according to the motor rotation angle and the timestamp of the collected data comprises:

[0016] At each pulse signal trigger, recording the current timestamp and the corresponding motor angle in real time, simultaneously sending a GPS analog pulse signal every second, and receiving a GPS message, and synchronizing the time of the point cloud data through the analog GPS timestamp.

[0017] Further, the process of splicing the point cloud data based on a three-dimensional rigid body motion model, combining a rotation matrix and spatial translation comprises:

[0018] Determining the accurate angle of each sampling point in the motor rotation process through the corresponding relationship between the timestamp of each point in the point cloud data and the rotation angle of the motor;

[0019] Performing rotation transformation on the point cloud data according to the motor rotation axis and the rotation angle;

[0020] Calculate the translation from the point to the rotation axis based on the rotation angle of each sampling point;

[0021] The point is translated according to the translation amount, and rotated according to the rotation matrix to obtain the final position and construct a three-dimensional point cloud model.

[0022] Furthermore, the process of projecting the three-dimensional point cloud map and image feature points onto a unit sphere includes:

[0023] The three-dimensional coordinates of each point in the three-dimensional point cloud are normalized to obtain a unit vector, and the Cartesian coordinates of the point cloud on the unit sphere are obtained.

[0024] For each feature point in the image, normalize it and then project it onto a unit sphere to obtain the Cartesian coordinates of the image feature points on the unit sphere.

[0025] Furthermore, the process of point cloud coloring using the least squares method includes:

[0026] With minimizing the Euclidean distance between the radar projection point and the image projection point as the optimization objective, the rotation matrix and translation vector are obtained through iterative optimization using the least squares method, thus determining the external parameter relationship between the radar coordinate system and the camera coordinate system.

[0027] Based on the rotation matrix and the translation vector, all radar point clouds and image pixels will be transformed into and aligned in a unit spherical coordinate system.

[0028] Interpolation is used to extract the corresponding color values ​​from the image and assign them to the corresponding point cloud.

[0029] On the other hand, to achieve the above objectives, this invention proposes a radar and vision data fusion 3D reconstruction system based on an external rotating shaft. An STM32 microcontroller is used to control the synchronous movement of the motor and the radar. By synchronizing the GPS timestamp of the radar and the rotation angle of the motor, it is ensured that all sampling and data recording are matched with the rotation of the motor during the 360° rotation of the radar.

[0030] Radar is used to perform spatial coordinate transformation on each frame of point cloud data collected during a 360° rotation by using real-time feedback of angle information, calculate the relative position of each point cloud data point, and adjust the coordinate position according to the rotation angle.

[0031] The point cloud stitching module is used to accurately register and stitch point cloud data collected from different angles, process the spatial coordinate differences of each frame of point cloud, and achieve seamless stitching through optimized algorithms to generate a three-dimensional point cloud map covering a 360° rotation range.

[0032] The camera is used to stop 3-6 times during a 360-degree rotation of the radar, take 3-6 photos in sequence, and stitch them together into a panoramic image with a horizontal and vertical ratio of 2:1 using PTGUI;

[0033] The data fusion module is used to select corresponding key points in radar point clouds and panoramic images, project the point cloud using the least squares method, calculate and fit the external parameters of the radar and camera, and project the point cloud data onto the camera image plane based on the external parameters to obtain enhanced color point cloud data.

[0034] Furthermore, the STM32 microcontroller achieves time synchronization between radar data and motor angle by simulating GPS timestamps.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] This invention employs an STM32 microcontroller as the core control unit, combined with a precise motor and radar synchronization control method. This allows for precise temporal and spatial synchronization between each radar data point and the motor's rotation angle. Through a high-precision motor control system combined with a 51-harmonic reducer and 16-microstep drive, the motor rotation angle accuracy reaches 0.0022 degrees, significantly improving the timeliness and accuracy of point cloud data acquisition. Compared to traditional systems, this invention can acquire higher-precision 360° panoramic point cloud data in a shorter time, providing a high-quality data foundation for subsequent point cloud stitching and 3D reconstruction.

[0037] This invention employs a point cloud stitching technique based on a three-dimensional rigid body motion model. During point cloud acquisition, a timestamp is appended to each point cloud data point, and precise alignment is achieved through rotation matrices and displacement vectors. This effectively solves the accuracy degradation problem caused by sparsity and noise during point cloud stitching. Compared to traditional feature matching methods, this invention significantly improves the accuracy and speed of point cloud stitching.

[0038] This invention innovatively employs a point cloud coloring method based on unit spherical projection. Through precise optimization of radar and camera extrinsic parameters, it achieves high-precision alignment between point clouds and images. The extrinsic parameters, optimized using the least squares method, perfectly align each point cloud with feature points in the image on a unit sphere, avoiding coloring inconsistencies caused by camera intrinsic parameter errors or distortion. Each point cloud data point accurately captures the RGB color information from the image, thus generating a high-quality color point cloud. Compared to traditional methods, the point cloud coloring effect of this invention exhibits higher color consistency and less color error, typically controlled within 2%.

[0039] The system of this invention not only possesses high-precision data acquisition and processing capabilities but also exhibits excellent real-time performance. Through the collaborative work of the STM32 and RK3588 motherboards, real-time acquisition of radar data and synchronization of motor rotation angles are ensured, and the entire data acquisition and reconstruction process can be completed in a short time. Experiments show that the system can complete approximately 100 frames per second of point cloud data stitching in a 300-frame data stitching process with 20,000 points per frame.

[0040] Experimental results show that, through optimized extrinsic parameters and iterative optimization using the least squares method, this invention can achieve high-precision projection alignment between images and point clouds. Compared with traditional methods, this invention exhibits higher robustness in feature point selection and matching, and demonstrates more stable performance under different lighting conditions, viewing angles, and dynamic scenes. The final generated color point cloud data possesses high visualization quality and can provide high-precision 3D reconstruction results for fields such as cultural relic restoration, virtual reality, and autonomous driving. Attached Figure Description

[0041] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings:

[0042] Figure 1 This is a schematic diagram of the radar and vision data fusion 3D reconstruction system based on an external rotating shaft according to the present invention.

[0043] Figure 2 This is a framework diagram of the radar and vision data fusion 3D reconstruction system based on an external rotating shaft according to the present invention;

[0044] Figure 3 This is a schematic diagram of a high-precision 3D point cloud model generated using the method of the present invention;

[0045] Figure 4 This is a schematic diagram of key point selection in point cloud data in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram illustrating the selection of key points in image data in an embodiment of the present invention.

[0047] Figure 6 This is a schematic diagram illustrating the correspondence between key points in point cloud data and image data generated using the method of the present invention.

[0048] Figure 7 This is a point cloud coloring effect diagram using the method of the present invention. Detailed Implementation

[0049] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] This embodiment proposes a 3D reconstruction method and system based on the fusion of radar and visual data from an external rotating axis, such as... Figure 1 As shown, the system uses an STM32 microcontroller for precise synchronous control of the motor and radar, and uses an RK3588 motherboard as the core computing unit to achieve precise synchronous control of the motor and radar. It can efficiently and accurately collect and stitch together 360° panoramic point cloud data to generate a seamless 3D spatial point cloud map, and fuse it with camera data to complete the generation of color point cloud and achieve 3D reconstruction.

[0051] This invention primarily uses an external rotating shaft to achieve 360° scanning of the radar and colorization of point clouds. It mainly addresses the problems of poor point cloud stitching accuracy and low spatial synchronization accuracy between the radar and the camera in existing technologies.

[0052] 1. Synchronization control of motor and radar:

[0053] To achieve high-precision synchronization between motor and radar data, this invention uses an STM32 microcontroller as the control core, combined with its powerful timer function, to precisely control the correspondence between the motor angle and the radar data acquisition. The specific operation steps are as follows:

[0054] First, a precise pulse signal is generated using the STM32's timer module to control the motor's rotation at a set angle. The motor's rotation angle is typically limited to 1.8 degrees per pulse by its inherent accuracy. To improve angle control precision, this system employs a 51-harmonic reducer and a 16-microstep drive. The high reduction ratio of the harmonic reducer and the combination of the microstep drive improve the angle accuracy of each motor pulse to 0.0022 degrees, thereby significantly reducing the motor's rotation angle error.

[0055] like Figure 2 As shown, during motor control, the STM32's timer triggers a pulse signal, and the motor rotates by the corresponding angle according to the pulse command. Each time a timer pulse is triggered, the STM32's interrupt service routine records the current timestamp and the corresponding motor angle in real time. This process ensures high-precision spatiotemporal consistency between each motor rotation and data acquisition.

[0056] To ensure precise time synchronization between the data acquired by the motor and the radar, another timer module in the STM32 microcontroller is used to work synchronously, sending one pulse per second and receiving GPS messages. This simulates a GPS timestamp to synchronize the radar data time. This means that the motor's rotation pulses and the GPS simulated pulse signal begin synchronously, ensuring that each radar data acquisition and the motor's rotation angle are aligned at the same time point, avoiding data misalignment caused by time delays or synchronization errors.

[0057] This synchronous control scheme ensures that the data acquisition of the motor and radar is synchronized throughout the entire 360° rotation process, so that each frame of radar data can perfectly correspond to the rotation angle of the motor, which significantly improves the timeliness and accuracy of point cloud data acquisition and provides a high-quality data foundation for subsequent point cloud stitching and 3D reconstruction.

[0058] 2. Stitching together multiple point clouds:

[0059] During point cloud data acquisition, the system records the timestamp of each sampling point and its x, y, z coordinates in 3D space. This data provides crucial temporal and spatial information for subsequent point cloud stitching. The stitching process is based on a 3D rigid body motion model, combining rotation matrices and spatial translation to accurately register and merge data from different frames. The specific steps are as follows:

[0060] First, during the rotation, the STM32 records the rotation angle and its corresponding timestamp. Then, by using the correspondence between the timestamp of each point in the point cloud data and the rotation angle of the motor, the precise angle of each sampling point during the motor's rotation is determined. Assume the rotation angle of the timestamp at each sampling point is θ. i The timestamp for each sampling point is t. i The rotation angle of each point can then be calculated using the following formula:

[0061]

[0062] in, This means finding the motor rotation angle corresponding to the timestamp recorded by the STM32 during the rotation by using the timestamp of the point cloud data.

[0063] Once the rotation angle at each point is determined, we need to consider the motor's rotation axis v and the rotation angle θ. i Perform a rotation transformation on the point cloud data. The rotation axis is v = (v... x ,v y ,v y ) is a unit vector, with a rotation angle θ. i This determines the calculation of the rotation matrix. Based on this, the rotation matrix R( (v) can be expressed by Rodrigues's rotation formula:

[0064]

[0065] in, It is an antisymmetric matrix of the rotation axis v, used to represent rotation operations. Using the rotation matrix, each point in the point cloud can be rotated relative to the rotation axis.

[0066] Next, based on the rotation angle of each sampling point, we calculate the translation of the point from the rotation axis. Assume the relative translation between the rotation axis and the original coordinate system of the point cloud is t, where t = (t... x ,t y ,t z ), representing the position vector from the rotation axis to the point (i.e., the translation of the point from the rotation axis). For each point P i =(x i ,y i ,z i ), its translated position The update can be performed using the following formula:

[0067]

[0068] Where t is the translation vector from the point to the axis of rotation.

[0069] After rotating the translated point, we obtain the final position:

[0070]

[0071] Each point in the above process is first translated onto the rotation axis, then rotated using a rotation matrix, and finally restored to the global coordinate system.

[0072] Through this series of operations, all the collected point cloud data can be precisely registered according to the motor rotation angle and rotation axis, and the data of multiple point cloud frames can be seamlessly stitched together.

[0073] Figure 3 This is a schematic diagram of a high-precision 3D point cloud model generated using the method of this invention. Through rotation and translation transformation of multiple frames of point clouds, the system generates a complete and seamless 3D point cloud model, which fully preserves the spatial structure and detailed information of the data.

[0074] This point cloud stitching method relies on a three-dimensional rigid body motion model. Through precise calculation of rotation matrices and displacement vectors, it ensures that point cloud data collected at different times and spatial locations can be accurately aligned and merged, significantly improving the accuracy and quality of the stitching results.

[0075] 3. Point cloud coloring:

[0076] Point cloud coloring first requires obtaining the extrinsic parameters of the radar and camera. Then, using these extrinsic parameters, the radar point cloud and the captured panoramic image are projected onto a unit spherical coordinate system and aligned for coloring. This method eliminates the influence of the camera's intrinsic parameters, uniformly aligning all data on the unit sphere.

[0077] This example selects key points corresponding to radar point clouds and images through object geometry, transforms the key points in the point cloud and images into unit spherical coordinates, and uses the least squares optimization method to align each key point to obtain extrinsic parameters.

[0078] Radar key points projected onto a unit sphere:

[0079] The selection of key points in the point cloud in this example is as follows: Figure 4 As shown, assume the key point of the radar is P. i =(x i ,y i ,z i Project these points onto a unit sphere. Normalize the 3D coordinates of each point to obtain a unit vector:

[0080]

[0081] here, These are coordinates on the unit sphere, representing the Cartesian coordinates of the point cloud on the unit sphere.

[0082] Image key points projected onto a unit sphere:

[0083] The selection of key image points in this embodiment is as follows: Figure 5 As shown (blue dots represent key points in the image), for each feature point in the image ( , The image coordinates are located in the range [0, W] (assuming the image width is W pixels and the image height is H pixels). To facilitate spherical mapping, it is normalized to the range [-1, 1] using the following formula:

[0084]

[0085] in, and These are the normalized coordinates, corresponding to the x and y coordinates of the image, respectively. The normalization operation ensures that all feature points are mapped to the standard coordinate range of [-1, 1].

[0086] Normalized coordinates ( , It needs to be converted to latitude in a spherical coordinate system. ( ) and longitude (θ). Specifically, latitude Longitude θ is calculated using the following formulas:

[0087]

[0088] Through this conversion, latitude Limited to [- , Within the range of π, longitude θ is restricted to the range of [-π,π], ensuring the correct position of the feature point on the sphere.

[0089] The obtained spherical coordinates ( Convert θ to Cartesian coordinates on a unit sphere. Calculate the Cartesian coordinates using the following formula. (x,y,z):

[0090]

[0091]

[0092]

[0093] here, These are coordinates on the unit sphere, representing the Cartesian coordinates of a pixel on the unit sphere.

[0094] Least squares optimization:

[0095] Here, we obtain the Cartesian coordinates of radar point cloud key points and image key points on a unit sphere. and The spatial transformation of keypoints in the point cloud is optimized using the least squares method, including the rotation matrix R and the translation vector T, to maximize the overlap between radar points projected onto a unit sphere and image keypoints. The optimization objective is to minimize the Euclidean distance between radar projection points and image projection points.

[0096]

[0097] By using the least squares iterative optimization method, the rotation matrix R and translation vector T are obtained, thereby determining the external parameter relationship between the radar coordinate system and the camera coordinate system.

[0098] Figure 6 (Green dots represent key points in the point cloud, and blue dots represent key points in the image.) This is the optimized point cloud feature points projected onto the image. The feature points are highly aligned, demonstrating the accurate matching between the point cloud and the image.

[0099] Coloring:

[0100] After obtaining the extrinsic parameters, we can transform the radar point cloud and image pixels to a unit spherical coordinate system and align them using a rotation matrix R and a translation vector T. Then, we can extract the corresponding color values ​​from the image using interpolation methods. And assign these color values ​​to the corresponding point clouds:

[0101]

[0102] This method establishes a precise correspondence between radar point clouds and images by projecting their positions onto a unit sphere, thus achieving point cloud coloring without relying on the camera's intrinsic parameter matrix. This approach avoids the effects of camera distortion and viewpoint differences, improving the accuracy and quality of point cloud coloring.

[0103] In summary, this invention, through high-precision synchronous control, accurate point cloud stitching technology, innovative coloring methods, and optimized system architecture, successfully solves several problems in existing technologies, significantly improving the accuracy, real-time performance, and stability of 3D reconstruction. It has broad application prospects in multiple fields (such as virtual reality, autonomous driving, cultural relic preservation, and architectural engineering), and can provide more realistic and accurate 3D environment reconstruction effects.

[0104] Example 2

[0105] This embodiment provides the specific operation flow of the radar and vision data fusion 3D reconstruction system based on an external rotating axis, as shown below:

[0106] Hardware setup:

[0107] 1. Mount the radar onto the stepper motor via a connector, and determine the translation amount of the rotation axis and the radar to the rotation axis based on the data of the connector.

[0108] Synchronization control of motor and radar:

[0109] 2. Use an STM32 microcontroller to control the synchronous movement of the motor and the radar.

[0110] 3. By synchronizing the GPS timestamp of the radar with the rotation angle of the motor, it is ensured that all sampling and data recording are precisely matched with the rotation of the motor during the 360° rotation of the radar.

[0111] 4. Whenever the motor rotates, the radar acquires data synchronously, forming a complete radar field of view coverage, ensuring the timeliness and accuracy of data acquisition.

[0112] Point cloud data acquisition and spatial transformation

[0113] 5. During the 360° rotation of the radar, spatial coordinate transformation is performed on the point cloud data collected in each frame by using the angle information fed back in real time.

[0114] 6. Calculate the relative position of each point cloud data point, and adjust its coordinate position according to the rotation angle to ensure the continuity of data within the panoramic range.

[0115] Point cloud stitching and stitching:

[0116] 7. Perform precise registration and stitching of point cloud data collected from different angles.

[0117] 8. The algorithm is used to process the spatial coordinate differences of each frame of point cloud, and the algorithm is optimized to achieve seamless stitching, finally generating a complete 3D point cloud map covering the entire 360° rotation range.

[0118] 9. The stitched point cloud map can display the complete three-dimensional environment and ensure the accuracy and continuity of the data.

[0119] Radar and camera data fusion:

[0120] 10. The radar stops six times during a 360-degree rotation, taking six photos in sequence, which are then stitched together by PTGUI into a panoramic image with a horizontal and vertical aspect ratio of 2:1.

[0121] 11. Select the corresponding key points in the radar point cloud and panoramic image, project the radar point cloud using the least squares method, and calculate and fit the external parameters of the radar and camera.

[0122] 12. Based on external parameters, radar data is projected onto the camera image plane, thereby combining color image information with point cloud data to obtain enhanced color point cloud data, thus completing the visualization of the three-dimensional environment.

[0123] 3D Reconstruction and Display

[0124] 13. Using the fused color point cloud data, perform 3D reconstruction to display a complete 3D environment model.

[0125] 14. By combining high-quality point clouds and images, it provides accurate and detailed 3D scene reconstruction results, which are widely used in scene reproduction, cultural relic restoration, game model construction and other fields.

[0126] Figure 7 This is a schematic diagram of a color point cloud generated using the method of the present invention. Figure 7 As can be seen, the color point cloud generated by the method of the present invention has a high degree of consistency with the RGB information of the original image, with uniform coloring and rich details, showing a high-quality three-dimensional reconstruction effect.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A three-dimensional reconstruction method based on the fusion of radar and visual data using an external rotating axis, characterized in that, Includes the following steps: The motor is controlled to drive the radar to rotate 360°, simultaneously collecting radar point cloud data and camera image data. Based on the motor rotation angle and the timestamp of the collected data, spatial coordinate transformation is performed on each frame of point cloud data; The point cloud data is stitched together based on a three-dimensional rigid body motion model combined with rotation matrix and spatial translation to obtain a three-dimensional point cloud map; The three-dimensional point cloud map and image feature points are projected onto a unit sphere, and the extrinsic parameters are optimized using the least squares method to perform point cloud coloring, generating colored point cloud data.

2. The method according to claim 1, characterized in that, The process of simultaneously acquiring radar point cloud data and camera image data includes: The STM32 microcontroller generates pulse signals to control the motor to rotate at a preset angle, while the angle is controlled by a 51 harmonic reducer and a 16 microstepping drive.

3. The method according to claim 1, characterized in that, The process of spatial coordinate transformation for each frame of point cloud data, based on the motor rotation angle and the timestamp of the acquired data, includes: Each time a pulse signal is triggered, the current timestamp and the corresponding motor angle are recorded in real time. At the same time, a GPS analog pulse signal is sent every second, and GPS messages are received. The time of the point cloud data is synchronized by using the analog GPS timestamp.

4. The method according to claim 1, characterized in that, The process of stitching together the point cloud data based on a three-dimensional rigid body motion model combined with rotation matrices and spatial translation includes: By establishing the correspondence between the timestamp of each point in the point cloud data and the rotation angle of the motor, the precise angle of each sampling point during the motor rotation process is determined. The point cloud data is rotated and transformed according to the motor's rotation axis and rotation angle; Calculate the translation from the point to the rotation axis based on the rotation angle of each sampling point; The point is translated according to the translation amount, and rotated according to the rotation matrix to obtain the final position and construct a three-dimensional point cloud model.

5. The method according to claim 1, characterized in that, The process of projecting the three-dimensional point cloud map and image feature points onto a unit sphere includes: The three-dimensional coordinates of each point in the three-dimensional point cloud are normalized to obtain a unit vector, and the Cartesian coordinates of the point cloud on the unit sphere are obtained. For each feature point in the image, normalize it and then project it onto a unit sphere to obtain the Cartesian coordinates of the image feature points on the unit sphere.

6. The method according to claim 1, characterized in that, The process of point cloud coloring using the least squares method includes: With minimizing the Euclidean distance between the radar projection point and the image projection point as the optimization objective, the rotation matrix and translation vector are obtained through iterative optimization using the least squares method, thus determining the external parameter relationship between the radar coordinate system and the camera coordinate system. Based on the rotation matrix and the translation vector, all radar point clouds and image pixels will be transformed into and aligned in a unit spherical coordinate system. Interpolation is used to extract the corresponding color values ​​from the image and assign them to the corresponding point cloud.

7. A 3D reconstruction system based on radar and vision data fusion using an external rotating shaft, characterized in that, include: An STM32 microcontroller is used to control the synchronous movement of the motor and the radar. By synchronizing the GPS timestamp of the radar and the rotation angle of the motor, it ensures that all sampling and data recording are matched with the rotation of the motor during the 360° rotation of the radar. Radar is used to perform spatial coordinate transformation on each frame of point cloud data collected during a 360° rotation by using real-time feedback of angle information, calculate the relative position of each point cloud data point, and adjust the coordinate position according to the rotation angle. The point cloud stitching module is used to accurately register and stitch point cloud data collected from different angles, process the spatial coordinate differences of each frame of point cloud, and achieve seamless stitching through optimized algorithms to generate a three-dimensional point cloud map covering a 360° rotation range. The camera is used to stop 3-6 times during a 360-degree rotation of the radar, take 3-6 photos in sequence, and stitch them together into a panoramic image with a horizontal and vertical ratio of 2:1 using PTGUI; The data fusion module is used to select corresponding key points in radar point clouds and panoramic images, project the point cloud using the least squares method, calculate and fit the external parameters of the radar and camera, and project the point cloud data onto the camera image plane based on the external parameters to obtain enhanced color point cloud data.

8. The system according to claim 1, characterized in that, The STM32 microcontroller achieves time synchronization between radar data and motor angle by simulating GPS timestamps.

Images