Omnidirectional point cloud fusion method based on multi-view structured light

Abstract

The invention provides an omnidirectional point cloud fusion method based on multi-view structured light, and the method comprises the steps: enabling a plurality of groups of cameras and projectors to surround a target object, so as to guarantee that the total visual angle of the plurality of cameras can cover the target object; performing projection calibration by using a calibration plate with a solid black dot array pattern so as to determine internal and external parameters and relative positions of the camera and the projector; the method comprises the following steps: projecting a specific image by using a projector, and carrying out image acquisition by using a camera to obtain a structured light image and corresponding depth information, and obtaining an initial three-dimensional point cloud; performing noise filtering and feature extraction on the initial three-dimensional point cloud to obtain a smooth three-dimensional point cloud and a descriptor used for representing local information; registering the smooth three-dimensional point cloud according to the internal and external parameters of the camera to obtain a preliminarily registered omnidirectional point cloud; and performing feature fusion by using a feature fusion strategy of multi-view data to obtain an omnidirectional point cloud. According to the method, the accuracy and the efficiency of three-dimensional reconstruction are improved while the limitation of a visual angle is avoided.

Description

Technical Field

[0001] The present application belongs to the technical field of structured light three-dimensional reconstruction, and in particular, relates to an omnidirectional point cloud fusion method based on multi-eye structured light. Background Art

[0002] Structured light technology is an active 3D imaging method that projects a known pattern (such as stripes, grids, or dots) onto the surface of an object, and then uses a camera to capture the pattern deformation caused by the shape of the object's surface. By decoding and calculating the captured image, the 3D shape information of the object's surface can be restored. Currently, structured light technology is widely used in industrial inspection, 3D printing, gesture recognition, virtual reality and other fields.

[0003] Multi-eye structured light technology is based on the monocular structured light system. It uses multiple cameras to capture the pattern deformation of structured light projection from different angles to obtain richer and more comprehensive three-dimensional information. The image acquired by each camera represents the three-dimensional information of the object surface at that perspective. The image data captured by multiple cameras can be registered and fused to generate a complete three-dimensional model. However, how to efficiently and accurately register point cloud data from multiple perspectives is the main challenge facing the current multi-eye structured light technology. The registration accuracy directly affects the quality of the final three-dimensional model. In addition, each camera and projector in the multi-eye structured light system needs to be accurately calibrated. Any slight calibration error will lead to cumulative errors in three-dimensional reconstruction. Summary of the invention

[0004] In view of this, the purpose of this application is to provide an omnidirectional point cloud fusion method based on multi-view structured light, which uses multiple cameras to scan the target object in all directions, combined with the grating pattern projected by the projector, to obtain rich structured light images and depth information, and finally generate high-precision omnidirectional point cloud data. This method effectively avoids the limitation of viewing angle, is suitable for high-quality 3D reconstruction in complex scenes, and significantly improves the accuracy and efficiency of 3D reconstruction.

[0005] The present application provides an omnidirectional point cloud fusion method based on multi-view structured light, the method comprising:

[0006] Arrange multiple groups of cameras and projectors around a target object to ensure that the total viewing angle of the multiple cameras can cover the target object; wherein each group includes two cameras and two projectors alternately fixed on fixed poles;

[0007] Projection calibration was performed using a calibration plate with a solid black dot array pattern to determine the intrinsic and extrinsic parameters and relative positions of each camera and projector;

[0008] Projecting a specific image using a projector, and capturing an image of the target object using a camera to obtain structured light images and corresponding depth information under multiple viewing angles, and obtaining initial three-dimensional point cloud data under multiple viewing angles;

[0009] Performing noise filtering and feature extraction on the initial three-dimensional point cloud data under multiple viewing angles to obtain smooth three-dimensional point cloud data under multiple viewing angles and a descriptor for characterizing local information of the smooth three-dimensional point cloud data;

[0010] According to the intrinsic and extrinsic parameters of each camera, the smoothed 3D point cloud data under multiple viewing angles are registered to obtain the preliminarily registered omnidirectional point cloud data;

[0011] By utilizing the feature fusion strategy of multi-view data, the feature fusion of the initially registered omnidirectional point cloud data is performed based on the descriptor to obtain the omnidirectional point cloud data.

[0012] Furthermore, the projection calibration is performed using a calibration plate having a solid black dot array pattern to determine the intrinsic and extrinsic parameters and relative positions of each camera and projector, including:

[0013] For each fixed column, the cameras on the fixed column are calibrated in pairs using Zhang Zhengyou's calibration method to obtain the intrinsic and extrinsic parameters and relative position relationship of the two cameras on the fixed column;

[0014] For each fixed column, the two projectors on the fixed column are calibrated in turn by Zhang Zhengyou calibration method to obtain the internal and external parameters of the two projectors on the fixed column and their relative position relationship with the camera;

[0015] Using Zhang Zhengyou calibration method, the fixed columns are calibrated in pairs in turn to obtain the relative position relationship between each fixed column.

[0016] Furthermore, the projector is used to project a specific image, and the camera is used to capture the image of the target object to obtain structured light images and corresponding depth information under multiple viewing angles, and obtain initial three-dimensional point cloud data under multiple viewing angles, including:

[0017] Using multiple projectors to sequentially project horizontal stripe images and Gray code images onto the surface of the target object from different viewing angles, and using multiple cameras to capture the modulated grating images and Gray code images of the surface of the target object from different viewing angles;

[0018] For the modulated grating image and Gray code image at each viewing angle, the continuous phase value of each point on the surface of the target object at that viewing angle is obtained by the Gray code-phase shift method, and according to the mapping relationship between phase and height, as well as the intrinsic and extrinsic parameters of the camera and projector at that viewing angle, the height information of the target object is mapped to the camera coordinate system to obtain the initial three-dimensional point cloud data at that viewing angle.

[0019] Furthermore, the noise filtering and feature extraction of the initial three-dimensional point cloud data under multiple viewing angles to obtain smooth three-dimensional point cloud data under multiple viewing angles and a descriptor for characterizing local information of the smooth three-dimensional point cloud data includes:

[0020] For the initial 3D point cloud data under each viewing angle, outliers in the initial 3D point cloud data are removed by a filtering method based on statistical analysis to obtain reliable 3D point cloud data;

[0021] For the credible 3D point cloud data under each viewing angle, Gaussian filtering technology is applied to perform smoothing processing to obtain smooth 3D point cloud data, and SHOT feature extraction is performed on the smooth 3D point cloud data to obtain a descriptor for characterizing the local information of the smooth 3D point cloud data.

[0022] Furthermore, the smooth three-dimensional point cloud data under multiple perspectives are registered according to the intrinsic and extrinsic parameters of each camera to obtain preliminarily registered omnidirectional point cloud data, including:

[0023] According to the intrinsic and extrinsic parameters of each camera, the position and direction of the smoothed 3D point cloud data at each viewing angle are adjusted to transform the smoothed 3D point cloud data at multiple viewing angles into the same global coordinate system to obtain preliminarily registered omnidirectional point cloud data.

[0024] Furthermore, the feature fusion strategy using multi-view data performs feature fusion on the preliminarily registered omnidirectional point cloud data based on the descriptor to obtain omnidirectional point cloud data, including:

[0025] Based on descriptors under multiple perspectives, feature points with high discrimination are selected from smooth 3D point cloud data under multiple perspectives;

[0026] Match the feature points under multiple perspectives to correct the mismatched point pairs in the initially registered omnidirectional point cloud data, and obtain the secondary registered omnidirectional point cloud data;

[0027] A global optimization function based on minimizing the point pair distance is constructed, and the IPC algorithm is used to optimize and merge the secondary registered omnidirectional point cloud data to obtain the omnidirectional point cloud data.

[0028] The omnidirectional point cloud fusion method based on multi-view structured light provided in this application uses multiple cameras to scan the target object in all directions, combined with the grating pattern projected by the projector, to obtain rich structured light images and depth information, and finally generate high-precision omnidirectional point cloud data. This method effectively avoids the limitation of viewing angle, is suitable for high-quality 3D reconstruction in complex scenes, and significantly improves the accuracy and efficiency of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A flow chart of an omnidirectional point cloud fusion method based on multi-eye structured light provided in an embodiment of the present application is shown;

[0030] Figure 2 A schematic diagram showing the arrangement of multiple groups of cameras and projectors surrounding a target object provided by an embodiment of the present application is shown;

[0031] Figure 3 A schematic diagram of the structure of each group of cameras and projectors provided in an embodiment of the present application is shown;

[0032] The reference numerals include: 10 - stage, 15 - fixed column, 20 - camera, 25 - projector. DETAILED DESCRIPTION

[0033] In order to make the purpose, technical solution and advantages of the technical solution more clear, the technical solution is further described in detail below in conjunction with specific implementation methods. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the technical solution.

[0034] See also Figure 1 , Figure 1 FIG. 1 is a flow chart of an omnidirectional point cloud fusion method based on multi-view structured light provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0035] S101 , arranging multiple groups of cameras and projectors around a target object to ensure that a total viewing angle of the multiple cameras can cover the target object.

[0036] Each group includes two cameras and two projectors alternately fixed on fixed poles.

[0037] In this step, see Figure 2 and Figure 3 , Figure 2 The figure shows a schematic diagram of the arrangement of multiple groups of cameras and projectors around a target object provided by an embodiment of the present application. When arranging the cameras and projectors, each camera can cover a different part of the target object, reducing the view angle occlusion, so as to ensure that the total view angle of multiple cameras can cover the target object. Figure 3The diagram shows the structure of each set of cameras and projectors provided in the embodiment of the present application. Each set includes two cameras and two projectors alternately fixed on fixed columns, and the fixed columns provide a stable support structure to ensure that the camera and the projector maintain a fixed relative position and angle during the scanning process.

[0038] It should be noted that the number of fixed columns (i.e., the number of groups of cameras and projectors arranged around the target object) is determined according to the dimension of the target object. The target object can be divided into two-dimensional and three-dimensional. When the target object is in the same plane, it is a two-dimensional target, and at least one fixed column is required for scanning; when the target object is not in the same plane, it is a three-dimensional target, and at least three fixed columns are required for scanning.

[0039] In addition, the target object can be fixed on a moving stage, and the target object can be scanned from multiple perspectives by controlling the rotation and movement of the stage.

[0040] S102, performing projection calibration using a calibration plate with a solid black dot array pattern to determine the intrinsic and extrinsic parameters and relative positions of each camera and projector.

[0041] In the specific implementation, the intrinsic and extrinsic parameters and relative positions of each camera and projector are determined by the following steps:

[0042] Step 1021: For each fixed column, the cameras on the fixed column are calibrated in pairs by Zhang Zhengyou calibration method to obtain the intrinsic and extrinsic parameters and relative position relationship of the two cameras on the fixed column.

[0043] In this step, for each fixed column, adjust the calibration plate posture so that the two cameras on the fixed column can take more than 10 sets of calibration plate images, and use Zhang Zhengyou calibration method to obtain the rotation matrix Rc and translation matrix Tc of the two cameras on the fixed column respectively.

[0044] Step 1022: for each fixed column, calibrate the two projectors on the fixed column in turn by using the Zhang Zhengyou calibration method to obtain the internal and external parameters of the two projectors on the fixed column and their relative position relationships with the camera.

[0045] In this step, for each fixed column, two projectors on the fixed column are turned on respectively, and a group of multi-frequency fringe patterns in the u direction and the v direction are projected onto the calibration plate, and one of the cameras on the fixed column is used to simultaneously capture the calibration plate image; wherein the multi-frequency fringe pattern used for projection is three groups of four-step phase-shifted images with different frequencies; based on the calibration plate image captured by the camera, the absolute phases in the u direction and the v direction are obtained by using the four-step phase-shift algorithm and the multi-frequency heterodyne phase decomposition principle, thereby calculating the center of the i-th calibration point in the camera image plane The pixel coordinates of the corresponding points in the projector image plane In order to obtain the relative position relationship between the two projectors and the camera.

[0046] Furthermore, the posture of the calibration plate is adjusted, and the same method is used to shoot more than 10 sets of calibration plate images in different postures, and the pixel coordinates of more than 10 sets of calibration point center projector images are calculated. The Zhang Zhengyou calibration method is used to complete the calibration of the internal and external parameters of the two projectors: the rotation matrix Rp and the translation matrix Tp.

[0047] In addition, according to the obtained camera parameters Rc, Tc and projector parameters Rp, Tp, use the following formula to obtain R and T, and complete the calibration of each fixed column. After the calibration is completed, the position of the camera and projector is fixed, and the relative position cannot be changed, otherwise recalibration is required.

[0048]

[0049] Where i represents the i-th camera and the i-th projector.

[0050] Step 1023: Use Zhang Zhengyou calibration method to calibrate the fixed columns in pairs in turn to obtain the relative position relationship between each fixed column.

[0051] In this step, the cameras in the adjacent fixed columns shoot the same calibration plate, and the relative position relationship between the two adjacent fixed columns is obtained using the Zhang Zhengyou calibration method. The above steps are repeated until each fixed column is calibrated.

[0052] S103, projecting a specific image using a projector, and capturing images of the target object using a camera to obtain structured light images and corresponding depth information under multiple viewing angles, and obtaining initial three-dimensional point cloud data under multiple viewing angles.

[0053] In specific implementation, the initial three-dimensional point cloud data under multiple perspectives can be obtained through the following steps:

[0054] Step 1031: Use multiple projectors to sequentially project horizontal stripe images and Gray code images onto the surface of the target object from different viewing angles, and use multiple cameras to capture modulated grating images and Gray code images of the surface of the target object from different viewing angles.

[0055] In this step, for each fixed column, a projector located between two cameras is used to project a horizontal stripe image and a Gray code image onto the surface of the target object in sequence, and two cameras are used to capture the modulated grating image and Gray code image of the surface of the target object at the same time; the other fixed columns adopt the same projection method to obtain the modulated grating image and Gray code image under multiple viewing angles.

[0056] Step 1032: for the modulated grating image and Gray code image at each viewing angle, the continuous phase value of each point on the surface of the target object at that viewing angle is obtained by the Gray code-phase shift method, and according to the mapping relationship between the phase and the height, as well as the internal and external parameters of the camera and the projector at that viewing angle, the height information of the target object is mapped to the camera coordinate system to obtain the initial three-dimensional point cloud data at that viewing angle.

[0057] In this step, for each modulated grating image and Gray code image at each viewing angle, the phase information of each stripe in the modulated grating image is first obtained by the Gray code-phase shift method. Assuming that there are M different phase shifts, for each phase φ m , the intensity I at a point on the surface of the object m It can be expressed as:

[0058]

[0059] Where I0(x,y) is the background intensity, I1(x,y) is the modulation degree, and φ(x,y) is the phase of a point on the surface of the object.

[0060] Secondly, by solving the brightness of multiple modulated grating images, the wrapped phase φ(x,y) is obtained using the following formula:

[0061]

[0062] Again, the period of the fringes is obtained by decoding the Gray code image and combined with the phase image to solve the continuous phase value of each point on the surface of the object.

[0063] Finally, according to the mapping relationship between phase and height, combined with the intrinsic and extrinsic parameters of the camera and projector at this viewing angle, the height information of the target object surface is mapped to the camera coordinate system, and the X, Y, and Z coordinates of each pixel in three-dimensional space are calculated to obtain the initial three-dimensional point cloud data at this viewing angle.

[0064] S104, performing noise filtering and feature extraction on the initial three-dimensional point cloud data under multiple viewing angles to obtain smooth three-dimensional point cloud data under multiple viewing angles and a descriptor for characterizing local information of the smooth three-dimensional point cloud data.

[0065] In a specific implementation, the smooth three-dimensional point cloud data under multiple viewing angles and the descriptor used to characterize the local information of the smooth three-dimensional point cloud data can be obtained by the following steps:

[0066] Step 1041 : For the initial three-dimensional point cloud data at each viewing angle, outliers in the initial three-dimensional point cloud data are removed by a filtering method based on statistical analysis to obtain credible three-dimensional point cloud data.

[0067] In this step, for each point in the initial three-dimensional point cloud data, the Euclidean distance and the average distance between the point and the specified number of nearest neighboring points are calculated; the distance threshold is set according to the global statistical data, and the points whose distance deviates from the average value by more than a certain standard deviation are marked as outliers. All point cloud data marked as outliers are eliminated to obtain reliable three-dimensional point cloud data.

[0068] Step 1042: For the credible three-dimensional point cloud data under each viewing angle, apply Gaussian filtering technology to perform smoothing processing to obtain smoothed three-dimensional point cloud data, and perform SHOT feature extraction on the smoothed three-dimensional point cloud data to obtain a descriptor for characterizing the local information of the smoothed three-dimensional point cloud data.

[0069] In this step, first, for each point p in the credible 3D point cloud data, select its neighborhood point set N(p), use the Gaussian kernel function, and use the following formula to perform weighted average calculation on the points in the neighborhood to obtain the position of the new smoothed point p' to form the smoothed 3D point cloud data:

[0070]

[0071] Where q is a point in the neighborhood N(p), d(p, q) is the distance between point p and the neighborhood point q, and G(d(p, q)) is the Gaussian weight.

[0072] Then, for each point p' in the smoothed 3D point cloud data, neighborhood points are selected and divided into multiple sub-regions. In each sub-region, the normal vector directions of all neighborhood points are calculated, and a histogram of the normal vector directions is constructed. The normal vector histograms of all sub-regions are combined into a descriptor for characterizing the local information of the point p'.

[0073] S105 . According to the intrinsic and extrinsic parameters of each camera, the smoothed three-dimensional point cloud data under multiple viewing angles are registered to obtain preliminarily registered omnidirectional point cloud data.

[0074] In specific implementation, the following steps can be used to obtain the preliminarily registered omnidirectional point cloud data:

[0075] Step 1051: adjust the position and direction of the smoothed 3D point cloud data at each viewing angle according to the intrinsic and extrinsic parameters of each camera, so as to transform the smoothed 3D point cloud data at multiple viewing angles into the same global coordinate system to obtain preliminarily registered omnidirectional point cloud data.

[0076] In this step, the smoothed 3D point cloud data of each viewpoint is preliminarily aligned based on the extrinsic parameters of each camera obtained by calibration.

[0077] Specifically, for each perspective of the smoothed 3D point cloud data P cam(in the camera coordinate system), which can be converted to the global coordinate system by the following transformation:

[0078] P global =RP cam +T; (5)

[0079] Where R and T are the rotation matrix and translation matrix of the calibrated camera coordinate system relative to the global coordinate system.

[0080] S106. Using a feature fusion strategy for multi-view data, perform feature fusion on the omnidirectional point cloud data to be fused based on the descriptor to obtain omnidirectional point cloud data.

[0081] In specific implementation, omnidirectional point cloud data can be obtained through the following steps:

[0082] Step 1061: Based on the descriptors under multiple viewing angles, feature points with high distinguishability are selected from the smoothed three-dimensional point cloud data under multiple viewing angles.

[0083] In this step, by analyzing the histogram features of the descriptor, points with higher discrimination of the descriptor are selected as feature points.

[0084] Step 1062: Match the feature points under multiple viewing angles to correct the mismatched point pairs in the initially registered omnidirectional point cloud data, and obtain the secondarily registered omnidirectional point cloud data.

[0085] In this step, descriptors are used to match feature points in the smoothed three-dimensional point cloud data at different viewpoints to find corresponding point pairs, and the similarity of descriptors is evaluated using the Euclidean distance metric to ensure the accuracy of the matching, eliminate mismatched point pairs in the preliminary registered omnidirectional point cloud data, and obtain more accurate secondary registered omnidirectional point cloud data.

[0086] Step 1063: construct a global optimization function based on minimizing the point pair distance, and use the IPC algorithm to optimize and merge the secondary registered omnidirectional point cloud data to obtain omnidirectional point cloud data.

[0087] In this step, a global optimization function that minimizes the point pair distance is constructed, and the ICP algorithm is used to further optimize the alignment of the point cloud and minimize the registration error between multi-view point clouds. The optimized point clouds from multiple viewpoints are merged, and the points in the overlapping area are weighted averaged to generate a unified omnidirectional point cloud data.

[0088] The above contents are only preferred embodiments of the present invention. For ordinary technicians in this field, many changes can be made in the specific implementation methods and application scopes based on the ideas of the present technical content. As long as these changes do not deviate from the concept of the present invention, they all fall within the scope of protection of this patent.

Claims

1. An omnidirectional point cloud fusion method based on multi-view structured light, characterized in that: The method comprises: Arrange multiple groups of cameras and projectors around a target object to ensure that the total viewing angle of the multiple cameras can cover the target object; wherein each group includes two cameras and two projectors alternately fixed on fixed poles; Projection calibration was performed using a calibration plate with a solid black dot array pattern to determine the intrinsic and extrinsic parameters and relative positions of each camera and projector; Projecting a specific image using a projector, and capturing an image of the target object using a camera to obtain structured light images and corresponding depth information under multiple viewing angles, and obtaining initial three-dimensional point cloud data under multiple viewing angles; Performing noise filtering and feature extraction on the initial three-dimensional point cloud data under multiple viewing angles to obtain smooth three-dimensional point cloud data under multiple viewing angles and a descriptor for characterizing local information of the smooth three-dimensional point cloud data; According to the intrinsic and extrinsic parameters of each camera, the smoothed 3D point cloud data under multiple viewing angles are registered to obtain the preliminarily registered omnidirectional point cloud data; By utilizing the feature fusion strategy of multi-view data, the feature fusion of the initially registered omnidirectional point cloud data is performed based on the descriptor to obtain the omnidirectional point cloud data.

2. The method according to claim 1, characterized in that The projection calibration is performed using a calibration plate with a solid black dot array pattern to determine the intrinsic and extrinsic parameters and relative positions of each camera and projector, including: For each fixed column, the cameras on the fixed column are calibrated in pairs using Zhang Zhengyou's calibration method to obtain the intrinsic and extrinsic parameters and relative position relationship of the two cameras on the fixed column; For each fixed column, the two projectors on the fixed column are calibrated in turn by Zhang Zhengyou calibration method to obtain the internal and external parameters of the two projectors on the fixed column and their relative position relationship with the camera; Using Zhang Zhengyou calibration method, the fixed columns are calibrated in pairs in turn to obtain the relative position relationship between each fixed column.

3. The method according to claim 1, characterized in that The specific image is projected by a projector, and the target object is captured by a camera to obtain structured light images and corresponding depth information under multiple viewing angles, and initial three-dimensional point cloud data under multiple viewing angles are obtained, including: Using multiple projectors to sequentially project horizontal stripe images and Gray code images onto the surface of the target object from different viewing angles, and using multiple cameras to capture the modulated grating images and Gray code images of the surface of the target object from different viewing angles; For the modulated grating image and Gray code image at each viewing angle, the continuous phase value of each point on the surface of the target object at that viewing angle is obtained by the Gray code-phase shift method, and according to the mapping relationship between phase and height, as well as the intrinsic and extrinsic parameters of the camera and projector at that viewing angle, the height information of the target object is mapped to the camera coordinate system to obtain the initial three-dimensional point cloud data at that viewing angle.

4. The method according to claim 1, characterized in that The noise filtering and feature extraction of the initial three-dimensional point cloud data under multiple viewing angles to obtain smooth three-dimensional point cloud data under multiple viewing angles and a descriptor for characterizing local information of the smooth three-dimensional point cloud data includes: For the initial 3D point cloud data under each viewing angle, outliers in the initial 3D point cloud data are removed by a filtering method based on statistical analysis to obtain reliable 3D point cloud data; For the credible 3D point cloud data under each viewing angle, Gaussian filtering technology is applied to perform smoothing processing to obtain smooth 3D point cloud data, and SHOT feature extraction is performed on the smooth 3D point cloud data to obtain a descriptor for characterizing the local information of the smooth 3D point cloud data.

5. The method according to claim 1, characterized in that The smooth three-dimensional point cloud data under multiple perspectives are registered according to the intrinsic and extrinsic parameters of each camera to obtain preliminarily registered omnidirectional point cloud data, including: According to the intrinsic and extrinsic parameters of each camera, the position and direction of the smoothed 3D point cloud data at each viewing angle are adjusted to transform the smoothed 3D point cloud data at multiple viewing angles into the same global coordinate system to obtain preliminarily registered omnidirectional point cloud data.

6. The method according to claim 1, characterized in that The feature fusion strategy using multi-view data is used to perform feature fusion on the preliminarily registered omnidirectional point cloud data based on the descriptor to obtain omnidirectional point cloud data, including: Based on descriptors under multiple perspectives, feature points with high discrimination are selected from smooth 3D point cloud data under multiple perspectives; Match the feature points under multiple perspectives to correct the mismatched point pairs in the initially registered omnidirectional point cloud data, and obtain the secondary registered omnidirectional point cloud data; A global optimization function based on minimizing the point pair distance is constructed, and the IPC algorithm is used to optimize and merge the secondary registered omnidirectional point cloud data to obtain the omnidirectional point cloud data.

Images