Three-dimensional reconstruction and pose estimation system and method based on binocular structured light

Abstract

The invention discloses a three-dimensional reconstruction and pose estimation system and method based on binocular structured light, and the method comprises the steps: obtaining binocular structured light image pairs and attitude angles of a detected part at different preset attitude angles, and synchronously binding the attitude angles with the binocular structured light images; reconstructing dense three-dimensional point clouds of the surface of the measured part in each attitude by utilizing parallax information between binocular images and combining structured light coding and triangulation principles; performing initial pose estimation by using an efficient multi-view algorithm, and performing registration on the dense three-dimensional point cloud under each pose by using a multi-scale geometric constraint ICP algorithm based on an initial pose estimation result and the pose angle to obtain a registered complete point cloud; and geometric completion is carried out on sparse point clouds in the obtained registered complete point clouds through a PCN model, and a global complete three-dimensional point cloud model of the measured part is generated. The method has good precision and robustness in reconstruction of the complex special-shaped part, and the system is light, rapid in deployment and stable in measurement.

Description

Technical Field

[0001] This invention relates to the technical field of machine vision and 3D reconstruction, and in particular to a 3D reconstruction and pose estimation system and method, which is especially suitable for the 3D reconstruction of non-standard irregular parts. Background Technology

[0002] With the development of intelligent manufacturing and industrial automation, 3D vision inspection technology has been widely used in tasks such as part identification, defect detection, and reverse modeling. Traditional 3D reconstruction systems are mostly based on principles such as laser scanning and binocular stereo vision, and their research focuses mainly on reconstruction accuracy, speed, and algorithm optimization.

[0003] Currently, while laser scanners offer high precision, they are expensive, bulky, and have limited scanning speed, making them difficult to deploy quickly in the field. Structured light systems, on the other hand, offer high acquisition speed and good stability; however, for irregularly shaped targets with different measurement ranges and orientations, frequent adjustments to equipment parameters and recalibration are often required, resulting in cumbersome operations and high technical barriers.

[0004] In existing research, binocular structured light systems have improved the effects of occlusion and enhanced measurement robustness to some extent by combining the advantages of stereo vision and active light sources. However, most systems are fixed assembly structures, lacking modularity and portability, and cannot adapt to complex environments and rapid deployment requirements.

[0005] Furthermore, existing 3D reconstruction platforms mostly use fixed cameras and static object modes for data acquisition. To obtain a complete model, additional extrinsic parameter calibration or manual rotation of the target object is required. This can easily lead to problems such as occlusion, missed scans, or inaccurate pose estimation when dealing with non-standard or irregularly shaped parts. At the same time, pose registration algorithms usually rely on complex feature extraction and initial pose estimation steps, resulting in a heavy processing flow that is susceptible to point cloud occlusion and noise interference.

[0006] Therefore, how to build a lightweight, mobile, easy-to-deploy, and highly adaptable 3D reconstruction platform, especially suitable for rapid acquisition and attitude recovery of irregular targets, as well as point cloud completion after 3D reconstruction of target objects, remains an important technical problem that urgently needs to be solved in the field of visual inspection.

[0007] With the increasing demand for personalized manufacturing and high-precision inspection, the 3D reconstruction and attitude estimation of non-standard and irregular-shaped parts have become an indispensable key link in the manufacturing industry. Due to their complex shapes, varying dimensions, and irregular edges, traditional registration or recognition algorithms based on rule models are difficult to adapt to these parts.

[0008] Currently, most 3D inspection equipment commonly used in industrial settings is fixed in structure, which limits its application environment and makes parameter adjustments cumbersome. Especially under different distance conditions, repeated system calibration and physical adjustments are often required to ensure accuracy, resulting in low efficiency and hindering rapid deployment and mobile inspection scenarios.

[0009] On the other hand, the acquisition method when the object under test is stationary limits the acquisition of the complete geometric shape. Especially when there is a self-occluding structure, the scanning view cannot cover all feature areas, resulting in missing 3D models or inaccurate pose estimation, which seriously affects subsequent applications.

[0010] Therefore, building a system that integrates high-precision acquisition capabilities, rapid deployment capabilities, and highly adaptive attitude adjustment capabilities has become an urgent need for technological development. Summary of the Invention

[0011] To address the technical problems of existing 3D reconstruction systems, such as cumbersome calibration, low versatility, and lack of portability, this invention proposes a portable 3D reconstruction system and method based on binocular structured light. In the binocular multi-angle attitude image acquisition module, different quick-release positions are designed for the camera position to allow for acquisition of targets at different distances. Combined with a three-degree-of-freedom rotation platform with gyroscope attitude feedback, target images from multiple angles and different perspectives are acquired. This enables high-precision acquisition of non-standard irregular parts in any attitude, calibration-free installation, and highly robust registration, solving the problems of "cumbersome calibration, low versatility, and lack of portability" in existing systems and providing a new approach for mobile and automated inspection in industrial settings.

[0012] To achieve the above objectives, the technical solution of this invention is as follows: a three-dimensional reconstruction and pose estimation method based on binocular structured light, comprising the following steps: S1. Acquire binocular structured light image pairs and attitude angles of the part under test at different preset attitude angles, and synchronously bind the attitude angles with the binocular structured light images. S2. Based on the acquired binocular structured light image pairs, the disparity information between the binocular images is used to reconstruct the dense three-dimensional point cloud of the surface of the measured part in each pose by combining structured light coding and triangulation principles. S3. Based on the dense 3D point cloud under each pose and the pose angle obtained in step S1, the initial pose estimation is performed using an efficient multi-viewpoint algorithm. Based on the initial pose estimation result and the pose angle, the dense 3D point cloud under each pose is registered using the multi-scale geometrically constrained ICP algorithm to obtain the registered complete point cloud. S4. Using the PCN model, geometric completion is performed on the sparse point cloud in the obtained registered complete point cloud to generate a global complete 3D point cloud model of the tested part.

[0013] Preferably, the three-dimensional attitude estimation result is corrected by calculating the pose deviation between the measured part and the reference part, and the pose deviation between the measured part and the true value, and taking the difference between the two pose deviations as the measurement error and the attitude deviation difference.

[0014] Preferably, the system for acquiring binocular structured light image pairs and attitude angles of the tested part at different preset attitude angles includes: The binocular multi-angle attitude image acquisition module is used to perform coded projection on the part under test and acquire binocular structured light image pairs of the part under test at each attitude angle under the coded projection state. A three-degree-of-freedom rotation platform with gyroscope attitude feedback includes a three-degree-of-freedom rotation platform and an IMU sensor module. The three-degree-of-freedom rotation platform is connected to the IMU sensor module. The three-degree-of-freedom rotation platform is used to adjust the attitude of the tested part by rotating around the X, Y, and Z axes, so that the tested part presents a preset attitude relative to the binocular multi-angle attitude image acquisition module. The IMU sensor module is used to record the attitude angle of the three-degree-of-freedom rotation platform in real time, and bind the attitude angle with the binocular structured light image acquired by the binocular multi-angle attitude image acquisition module through a timestamp mechanism. The 3D reconstruction and pose estimation module is used to receive and perform 3D reconstruction and pose estimation of the tested part based on attitude angles and binocular structured light image pairs. The IMU sensor module is connected to the 3D reconstruction and pose estimation module, the binocular multi-angle pose image acquisition module is connected to the 3D reconstruction and pose estimation module, and the 3D reconstruction and pose estimation module is located in the main control unit of the host computer.

[0015] Preferably, the binocular multi-angle pose image acquisition module includes a housing and a left camera, a structured light projector, and a right camera disposed inside the housing; the left camera, structured light projector, and right camera are arranged sequentially in the horizontal direction inside the housing; The outer shell includes a lower shell and an upper cover, with the upper cover hinged to one side of the lower shell; the lower shell has three through holes on the side facing the three-degree-of-freedom rotating platform, and the size and shape of the three through holes are respectively matched with the front end shapes of the left camera, the structured light projector and the right camera; The bottom of the lower housing is provided with multiple slots, which can be removably used to accommodate the left camera, the structured light projector, and the right camera.

[0016] Preferably, the method for synchronously binding the attitude angle with the binocular structured light image is as follows: During the acquisition process, the main control unit sends a unified hardware trigger command to the left camera, right camera, and structured light projector of the binocular multi-angle attitude image acquisition module; after receiving the trigger signal, the left and right cameras start exposure in strict synchronization to ensure that the acquisition time of the binocular image pair is completely consistent with the exposure window; the structured light projector outputs the corresponding pattern during the camera exposure according to the preset coding sequence; the main control unit assigns a unified timestamp to each frame of acquired data and synchronously binds it to the real-time attitude angle recorded by the IMU sensor module to form a three-dimensional data association of image-attitude-projection sequence; The card slot includes a first card slot and a second card slot for placing the structured light projector. The first card slot is located at the center of the bottom of the lower housing. Three second card slots at different distances are provided on each side of the first card slot for placing the left camera and the right camera, respectively. The three different distances include three sets of settings: close distance, medium distance, and long distance. The baseline parameter configuration files of the cameras placed in the three second card slots at different distances are stored in the main control unit. The distance measurement ranges for the three gear settings are 300-600 mm, 600-1000 mm, and 1000-1500 mm, respectively. The three-degree-of-freedom rotating platform is equipped with a detachable buckle mechanism that matches the part being tested.

[0017] Preferably, based on the geometry of the part under test and the required ranging range, a suitable gear is selected, and the left camera, right camera, and structured light projector are assembled. The part under test is fixed at the center of the three-degree-of-freedom rotating platform, and the IMU sensor module is activated. Driven by the three-degree-of-freedom rotating platform, the attitude of the part under test is sequentially adjusted to different preset attitude angles according to a preset sequence. After each preset attitude angle is adjusted, the corresponding attitude angle is recorded by the IMU sensor module, and the main control unit sends a unified hardware trigger pulse: the left and right cameras are synchronously exposed based on the pulse signal, and a pair of binocular structured light images under structured light coded projection is acquired. The structured light projector outputs a preset coded pattern sequence within the camera exposure window. The method for reconstructing the dense three-dimensional point cloud of the surface of the tested part in each pose is as follows: a stereo matching algorithm is used, and depth recovery is performed independently for each pair of binocular structured light images to obtain the dense three-dimensional point cloud.

[0018] Preferably, the efficient multi-view algorithm introduces gyroscope attitude priors to construct attitude constraint terms, thereby improving the stability of the initial pose estimation under occlusion or viewpoint shift conditions. The ICP algorithm employs a multi-scale iterative constraint strategy for multi-stage optimization, integrating feature point matching, point cloud normal direction consistency constraints, and attitude guidance brought by attitude angles to optimize point alignment accuracy.

[0019] Preferably, in the initial pose estimation, the efficient multi-viewpoint algorithm represents each 3D point as a linear combination of four virtual control points: ;in, For the first i A three-dimensional point, For the first j One virtual control point, For the first j The linear coefficients of the virtual control points satisfy: =1; After mapping the virtual control points to the camera coordinate system, and combining them with the camera intrinsic parameter matrix K, the following projection relationship is formed: ; In the formula, For the first i The scale factor of a three-dimensional point , The first i The two-dimensional pixel coordinates of a three-dimensional point on the image plane For the first j The three-dimensional coordinate vectors of each virtual control point in the camera coordinate system; An attitude-guided optimization model is constructed by minimizing the attitude deviation as a regularization term: ; In the formula, Let be the vector of pose parameters to be optimized; This represents the output of the EPNP pose estimation algorithm, which takes the pose parameter vector θ as input. θ gyro For attitude priors, it refers to the attitude angle vector that is acquired and synchronously bound in real time in step S1; The objective function of the ICP algorithm with multi-scale geometric constraints is: ; in, and These are the corresponding points in the target point cloud and the source point cloud, respectively. It is a rigid transformation matrix. For matching point sets.

[0020] Preferably, the PCN model is built on an encoder-decoder architecture, and the workflow includes: generating a coarse point cloud through a global feature extraction step; then restoring the missing regions based on local folding operations, and finally outputting a complete point cloud with continuous structure and complete details; In the encoder, global feature extraction is achieved through a combination of multilayer perceptron and max pooling operations; in the decoder, the FoldingNet architecture is used to map the two-dimensional mesh to three-dimensional space to restore the point cloud morphology and fill in the missing regions.

[0021] Preferably, the training phase of the PCN model uses Chamfer distance as the main loss function to measure the generated point cloud. Alignment degree with the target point cloud P: ; In the formula, Represents the target point cloud P and the generated point cloud Q̂ Chamfer distance between them This represents a single point in the target point cloud P. Indicates the generation of point clouds Q̂ A single point in; The encoder takes an incomplete 3D point cloud X as input, which is a sparse point cloud within the registered complete point cloud. Global features are extracted by stacking two PointNet layers. A shared multilayer perceptron is used to map each point in the 3D point cloud X to a high-dimensional point feature F1. Max pooling is then performed on all high-dimensional point features F1 to obtain a global feature vector g. The global feature vector g is broadcast and concatenated to each high-dimensional point feature F1 to obtain an enhanced feature F2. After another set of shared multilayer perceptrons and max pooling operations, the final global feature vector v is extracted and used as the encoder output for input to the downstream decoder. The decoder maps the global feature vector v to an s×3 three-dimensional point coordinate matrix through a multilayer perceptron and a reconstruction layer, generating a coarse point cloud Ycoarse. A u×u two-dimensional grid is constructed around each point in the coarse point cloud Ycoarse, where u is the resolution parameter of the two-dimensional grid. The coordinates of this two-dimensional grid, the global feature vector v, and the three-dimensional coordinates of the point are concatenated along the feature dimension to obtain the input vector. The input vector undergoes a nonlinear transformation through a shared multilayer perceptron to generate patch coordinates of the three-dimensional surface corresponding to the coarse point. These patch coordinates are added to the corresponding coarse point coordinates and converted into global coordinates. All global coordinates are merged to obtain the finely completed point cloud Ydetail, which serves as the final output of the decoder. The difference between the generated point cloud and the real point cloud is measured by the chamfer distance loss and the earth-movement distance loss between the coarse point cloud Ycoarse and the ground truth point cloud. The fine point cloud is generated by the decoder through local folding operations based on the coarse point cloud Ycoarse. The difference between the generated point cloud and the real complete point cloud is measured by the chamfer distance loss.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention is particularly suitable for high-precision modeling and attitude recognition of non-standard irregular parts. The binocular multi-angle attitude image acquisition module adopts a modular black box structure with multiple preset positions, enabling rapid assembly and calibration-free deployment of the camera and projector under different ranging conditions. Through a three-degree-of-freedom rotation platform with gyroscope attitude feedback, it achieves multi-angle and multi-attitude 3D data acquisition of the target object, suitable for high-precision point cloud acquisition of target parts with complex attitudes. The invention utilizes the proposed Efficient Perspective Multi-Point (EPMP) and Iterative Closest Point (ICP) algorithms, employing attitude-guided EPMP with gyroscope prior information and ICP with multi-scale geometric constraints, significantly enhancing the stability and accuracy of point cloud fusion. Simultaneously, a PCN model is introduced for point cloud completion, achieving structural restoration and complete modeling of occluded areas. Experiments show that this invention exhibits good accuracy and robustness in the reconstruction of complex irregular parts, with an average error of less than 0.5 mm and an attitude error of less than 0.02. The system of this invention is lightweight, quick to deploy, and provides stable measurement. It is suitable for continuous inspection and reverse modeling of irregularly shaped parts in industrial settings and has good potential for widespread application. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is an overall structural diagram of the binocular structured light-based 3D reconstruction and pose estimation system of the present invention.

[0025] Figure 2 for Figure 1 The diagram shows the structure of the binocular multi-angle pose image acquisition module, where (a) is a top view after opening the cover and (b) is a bottom view of (a).

[0026] Figure 3 This is a flowchart of the three-dimensional reconstruction and pose estimation method based on binocular structured light of the present invention.

[0027] Figure 4 This is a diagram of the PCN network structure of the present invention. Figure 5 The images shown are of the physical object and model in one embodiment of the present invention.

[0028] Figure 6 This is a color point cloud diagram according to one embodiment of the present invention.

[0029] In the diagram, 1 is the left camera, 2 is the structured light projector, 3 is the outer casing, 4 is the right camera, 5 is the rotating platform, 6 is the IMU sensor module, and 7 is the host computer. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Example 1 A 3D reconstruction and pose estimation system based on binocular structured light, such as Figure 1 As shown, it includes: The binocular multi-angle attitude image acquisition module is used to perform coded projection on the part under test and acquire binocular structured light image pairs of the part under test at each attitude angle in the coded projection state.

[0032] A three-degree-of-freedom rotating platform with gyroscope attitude feedback consists of a three-degree-of-freedom rotating platform 5 and an IMU sensor module, wherein: The three-degree-of-freedom rotation platform is used to adjust the attitude of the part under test by rotating around the X-axis (pitch), Y-axis (yaw), and Z-axis (roll), so that the part under test presents a preset geometric attitude relative to the binocular multi-angle attitude image acquisition module, ensuring that the full-view image data required for 3D reconstruction is covered.

[0033] The IMU sensor module is used to record the attitude angles of the three-degree-of-freedom rotating platform 5 in real time, and binds the attitude angles with the binocular structured light images acquired by the binocular multi-angle attitude image acquisition module through a time synchronization mechanism, so as to provide prior information for subsequent attitude estimation.

[0034] The 3D reconstruction and pose estimation module is used to receive and perform 3D reconstruction and pose estimation of the tested part based on the attitude angle and binocular structured light image pairs.

[0035] The three-degree-of-freedom rotating platform is connected to the IMU sensor module, which in turn is connected to the 3D reconstruction and pose estimation module. The binocular multi-angle pose image acquisition module is also connected to the 3D reconstruction and pose estimation module. The 3D reconstruction and pose estimation module is located in the host computer 7.

[0036] The binocular multi-angle attitude image acquisition module establishes communication with the main control unit (host computer 7) in the 3D reconstruction and pose estimation module through a data interface. To ensure precise control of binocular synchronous imaging and projection timing, the main control unit issues unified hardware trigger commands to the left and right cameras and the structured light projector of the binocular multi-angle attitude image acquisition module during the acquisition process: after receiving the trigger signal, the left and right cameras strictly synchronize their exposure to ensure that the acquisition time of the binocular image pair is completely consistent with the exposure window; the structured light projector outputs the corresponding pattern according to the preset encoding sequence (such as Gray code, phase-shifted fringes) during the camera exposure period, realizing the timing closed loop of "same-frame projection-same-frame acquisition". The main control unit assigns a unified timestamp to each frame of acquired data and synchronously binds it to the real-time attitude angle recorded by the IMU sensor, forming a three-dimensional data association of image-attitude-projection sequence. If multiple encoded patterns are required, the system cyclically executes the projection-acquisition process according to the periodic trigger signal to ensure that all sequence frames are strictly aligned in the time dimension.

[0037] The binocular multi-angle pose image acquisition module includes a housing 3 and a left camera 1, a structured light projector 2, and a right camera 4 disposed inside the housing 3; inside the housing 3, the left camera 1, the structured light projector 2, and the right camera 4 are arranged in sequence along the horizontal direction.

[0038] like Figure 2 As shown, the outer casing 3 includes a lower casing and an upper cover. The upper cover is hinged to one side of the lower casing and is foldably connected to the lower casing to facilitate opening and closing of the upper cover, thus enabling equipment maintenance and replacement installation. Figure 2 As shown in (b), the bottom of the lower housing is provided with multiple slots, which can be removably used to accommodate the left camera 1, the structured light projector 2, and the right camera 4. In use, the left camera 1, the structured light projector 2, or the right camera 4 currently placed in the slot can be manually pulled out by opening the top cover, and the required left camera, structured light projector, or right camera can be inserted into its corresponding slot for replacement.

[0039] Specifically, the slot for placing the structured light projector 2 is located in the center. The size and shape of the slot are matched to the structured light projector, ensuring stable support for it. Three second slots at different distances are located on each side of the central slot, for placing the left camera 1 and the right camera 4 respectively. These three distances include near, medium, and far ranges. The baseline parameter configuration files for the cameras (left camera 1 and right camera 4) located in the three second slots at different distances are stored in the main control unit.

[0040] The positions of the left camera 1 and right camera 4 within the lower housing can be adjusted by selecting different gear positions to meet diverse usage scenarios and functional requirements. After switching gear positions, the operator needs to manually select the current gear number (e.g., close-range / medium-range / long-range) in the main control unit's system software interface. Based on the manually selected gear number, the main control unit calls the pre-stored camera baseline parameter configuration file, enabling rapid deployment without calibration.

[0041] In practical use, operators can manually switch gears according to the size of the target being measured and the distance measurement requirements. The mechanical slot mechanism will fix the camera and structured light projector in the preset position of the corresponding gear, ensuring the stability of the baseline length and viewing angle parameters. At the same time, the software interface displays a list of selectable gears and the appropriate distance measurement range. Operators need to manually confirm the current gear selection to ensure the device correspondence.

[0042] The three gear settings are designated as first, second, and third gears, corresponding to 3D reconstruction needs at close range (approximately 300–600 mm), medium range (approximately 600–1000 mm), and long range (approximately 1000–1500 mm), respectively. The intrinsic parameters and baseline parameters for each gear setting are calibrated at the factory and fixed in the configuration file. After switching gears, the operator manually loads the corresponding configuration file to avoid repeating the calibration process.

[0043] In use, based on the size of the part being measured and the distance measurement requirements, select the appropriate gear from the three gear positions for insertion, and manually select the corresponding gear number in the software interface to load parameters. This allows for rapid replacement of the camera module without manual calibration, achieving plug-and-play functionality and convenient deployment. The left and right cameras maintain a unified physical structure with the structured light projector 2, adapting to different distance measurement conditions only by changing their installation positions.

[0044] Specifically, the first gear is suitable for close-range high-precision measurement scenarios, with a range of 300 mm to 600 mm. It is mainly suitable for small, complex, irregularly shaped parts with a volume of less than 200 mm (such as electronic connectors, precision gears, small industrial components, etc.), and can provide the highest density point cloud data and the smallest measurement error.

[0045] The second setting is suitable for medium-distance conventional measurement scenarios, with a distance range of 600 mm to 1000 mm. It is mainly suitable for medium-sized parts with a volume between 200 mm and 500 mm (such as industrial joints, pipe elbows, mechanical connection parts, etc.), ensuring measurement accuracy while taking into account the field of view coverage.

[0046] The third setting is suitable for long-distance, wide-field-of-view measurement scenarios, with a ranging range of 1000 mm to 1500 mm. It is suitable for large, irregularly shaped components with a volume size greater than 500 mm (such as complex pipe components, large mold parts, etc.), facilitating overall contour acquisition and posture recognition.

[0047] By flexibly selecting different gear settings, a balance is achieved between high-precision detail recovery and large-scale overall modeling, based on the size of the measured part and the measurement accuracy requirements, thus improving adaptability and efficiency. All gear angles and baseline parameters are pre-calibrated experimentally, eliminating the need for manual calibration. The corresponding intrinsic parameter configurations are loaded by default during use, avoiding repeated calibration processes and effectively simplifying on-site operation procedures.

[0048] The lower housing has three through holes on one side facing the three-degree-of-freedom rotating platform 5. The size and shape of the through holes are matched with the front end of the left camera 1, the structured light projector 2 and the right camera 4, respectively, so that the front end of the left camera 1, the structured light projector 2 and the right camera 4 are exposed to ensure that a complete field of view is obtained.

[0049] The lower housing is made of high-strength material with vibration resistance, and the vibration attenuation coefficient of the material is not less than 0.05. Its impact absorption capacity meets the industrial level 4 vibration protection standard (such as GB / T 2423.5-2019). Applicable materials include polymer composite materials (such as ABS+PC alloy), high-density engineering plastics (such as PA66), and lightweight metal alloys (such as magnesium-aluminum alloy), balancing rigidity and lightweight requirements. The lower housing also supports the replacement of different types of support platforms or lifting bases, ensuring the stability and portability of the binocular multi-angle posture image acquisition module in varied industrial environments, suitable for rapid deployment and mobile operation in complex scenarios.

[0050] The three-degree-of-freedom rotating platform 5 is equipped with a detachable locking mechanism that matches the part under test to ensure its stability during the acquisition of binocular structured light images. The three-degree-of-freedom rotating platform supports pitch, roll, and yaw adjustments, providing flexible attitude adjustment capabilities for multi-view acquisition. The attitude angles of the three-degree-of-freedom rotating platform are recorded in real time by the IMU sensor module 6 and synchronized with the binocular structured light image pair to guide attitude estimation and point cloud fusion, improving the stability and robustness of registration.

[0051] Example 2 like Figure 3 As shown, a 3D reconstruction and pose estimation method based on binocular structured light is applied to the 3D reconstruction and pose estimation module, including the following steps: S1. Acquire binocular structured light image pairs and attitude angles of the part under test at different preset attitude angles, and synchronously bind the attitude angles with the binocular structured light images.

[0052] Start the 3D reconstruction and pose estimation system and complete initialization and parameter loading. Based on the geometry of the part being measured and the required distance range, select the appropriate range (near, medium, or far) from the binocular multi-angle pose image acquisition module, and complete the assembly of the left camera 1, right camera 4, and structured light projector 2.

[0053] The part under test is fixed to the center of the three-degree-of-freedom rotary platform using a detachable snap-fit ​​mechanism, and the IMU sensor module 6 is activated. The three-degree-of-freedom rotary platform supports attitude adjustment in three directions: pitch, roll, and yaw. It has a high-precision electronic control drive mechanism to ensure controllable and stable attitude during multi-view shooting.

[0054] Driven by a three-degree-of-freedom rotation platform, the orientation of the part under test is sequentially adjusted to different preset orientation angles according to a preset sequence. After each part is adjusted to a preset orientation angle, the orientation angle is recorded by the IMU sensor module 6. At this time, the main control unit sends a unified hardware trigger pulse: the left and right cameras are strictly synchronized in exposure based on this pulse signal, acquiring image pairs under structured light coded projection; the structured light projector outputs a preset coded pattern sequence within the camera exposure window. The main control unit assigns a unique timestamp to the current binocular image pair and synchronously records the real-time orientation angle fed back by the IMU sensor module 6, completing the three-dimensional binding of the image-orientation-projection sequence. By covering as much viewpoint information as possible through the above process, a spatiotemporally consistent data foundation is provided for subsequent initial pose estimation and point cloud registration.

[0055] S2. Based on the acquired binocular structured light image pairs, the disparity information between the binocular images is used to reconstruct the dense three-dimensional point cloud of the surface of the measured part in each pose by combining structured light coding and triangulation principles, providing a basis for subsequent pose estimation and point cloud registration.

[0056] The processing of binocular structured light image pairs employs high-precision stereo matching algorithms, such as the semi-global matching (SGM) algorithm or a matching method based on cost volume and dynamic programming, and independently performs depth restoration for each pair of binocular structured light image pairs.

[0057] The acquired dense 3D point cloud is used to reflect the local geometric details of the surface of the measured part and serves as a key input for subsequent attitude estimation and point cloud registration processes.

[0058] S3. Using the Efficient Perspective Multi-Point (EPMP) algorithm, initial pose estimation is performed based on the dense 3D point cloud in each pose and the pose angle obtained in step S1. Based on the initial pose estimation result and the pose angle, the ICP (Iterative Closest Point) algorithm with multi-scale geometric constraints is used to register the dense 3D point cloud in each pose to obtain the registered complete point cloud.

[0059] To improve the accuracy of pose estimation and the robustness of the registration process, this invention proposes a combined EPMP and ICP algorithm based on existing algorithms. EPMP introduces gyroscope attitude priors to construct attitude constraints, effectively improving the stability of initial pose estimation under occlusion or viewpoint shift conditions. ICP employs a multi-scale iterative constraint strategy (Multi-scale Constrained ICP), integrating feature point matching, normal direction consistency constraints, and attitude sensor guidance mechanisms to improve the accuracy and convergence efficiency of multi-view point cloud registration.

[0060] Specifically, the EPMP algorithm is obtained by improving the EPNP algorithm. The EPMP algorithm improves the stability of the initial pose estimation under conditions of occlusion and changing angles by introducing a pose deviation regularization term.

[0061] In the initial pose estimation, the EPNP algorithm no longer relies solely on the projection correspondence between 3D and 2D points, but instead represents each 3D point as a linear combination of four virtual control points: (1) in, For the first i A three-dimensional point, For the first j One virtual control point, For the first j The linear coefficients of the virtual control points satisfy: =1 (2) After mapping the virtual control points to the camera coordinate system, and combining them with the camera intrinsic parameter matrix K, the following projection relationship is formed: (3) In the formula, For the first i The scale factor of a three-dimensional point , The first i The two-dimensional pixel coordinates of a three-dimensional point on the image plane For the first j The three-dimensional coordinate vector of a virtual control point in the camera coordinate system.

[0062] To enhance the robustness of the initial pose estimation process to attitude shifts and occlusions, attitude priors provided by the gyroscope are further introduced. θ gyro This is the attitude angle vector acquired and synchronously bound in real time by the IMU sensor module 6 in step S1. This vector contains three components: pitch angle, yaw angle, and roll angle. An attitude-guided optimization model is constructed by minimizing the attitude deviation as a regularization term: (4) In the formula, The vector of pose parameters to be optimized. This represents the output of the EPNP pose estimation algorithm, which takes the pose parameter vector θ as input.

[0063] The EPMP algorithm significantly improves the stability of initial attitude estimation when initial registration conditions are not ideal.

[0064] The multi-scale geometrically constrained ICP algorithm adopts a pyramid-style hierarchical strategy for multi-stage optimization. By combining the consistency of point cloud normal vector direction with the attitude angle obtained in step S1, it improves the accuracy and convergence speed of point cloud alignment and realizes global fusion of multi-view point clouds.

[0065] After initial pose estimation, the registration stage for dense 3D point clouds under each pose is entered. A multi-scale geometrically constrained ICP algorithm is employed to optimize point alignment accuracy. The basic objective function of the multi-scale geometrically constrained ICP algorithm is: (5) in, and These are the corresponding points in the target point cloud and the source point cloud, respectively. Let T be the set of matching points obtained through feature matching and nearest neighbor search, and let T be the rigid transformation matrix estimated by the Iterative Nearest Point (ICP) algorithm based on the set of matching points. Coarse-to-fine iterative registration under a multi-scale pyramid structure, combined with the consistency of normal directions and the attitude guidance provided by the attitude angles obtained in step S1, further improves the convergence efficiency and adaptability to complex geometric disturbances.

[0066] S4. Geometrically complete the sparse point cloud in the registered complete point cloud obtained in step S3 using the PCN model to generate a global complete three-dimensional point cloud model of the tested part. The global complete three-dimensional point cloud model has a continuous structure, complete boundaries, and high quality.

[0067] To address the issue of missing point clouds due to occlusion, blind spots, or limited local view, a Point Completion Network (PCN) model is employed to geometrically complete sparse point clouds. The PCN model utilizes a two-stage structure of global feature encoding and local detail restoration to predict the 3D morphology of invisible regions. The PCN model is built on an encoder-decoder architecture, and its workflow includes: first, generating a coarse point cloud through global feature extraction; then, restoring the missing regions based on local folding operations; and finally, outputting a complete point cloud with a continuous structure and complete details.

[0068] Specifically, such as Figure 4 As shown, in the encoder, global feature extraction is achieved through a combination of multilayer perceptron (MLP) and max pooling operation; in the decoder, the FoldingNet architecture is used to map the two-dimensional mesh to three-dimensional space, thereby completing the restoration of point cloud morphology and the completion of missing regions.

[0069] To optimize the quality of the complete 3D point cloud model generated by the PCN model, Chamfer distance is used as the main loss function during the PCN model training phase to measure the quality of the generated point cloud. Alignment degree with the target point cloud P: (6) In the formula, Represents the target point cloud P and the generated point cloud Q̂ Chamfer distance between them This represents a single point in the target point cloud P. Indicates the generation of point clouds Q̂ A single point in the array.

[0070] The encoder takes an incomplete 3D point cloud X as input, i.e., a sparse point cloud within the registered complete point cloud. Global features are extracted by stacking two PointNet layers, supporting unordered point cloud input while maintaining permutation invariance. A shared multilayer perceptron (MLP) is used to map each point in the 3D point cloud X to a high-dimensional point feature F1, and max pooling is performed on all high-dimensional point features F1 to obtain a global feature vector g, representing the global geometric information of the 3D point cloud X. The global feature vector g is broadcast and concatenated to each high-dimensional point feature F1 to obtain an enhanced feature F2. This enhanced feature F2 is then processed by another set of shared multilayer perceptrons and max pooling operations to extract the final global feature vector v, which serves as the encoder output and input to the downstream decoder.

[0071] Decoder: First, a coarse point cloud Ycoarse is generated based on the global feature vector v. This process is implemented by a multilayer perceptron (MLP) and a reshape layer, mapping the global features into an s×3 3D point coordinate matrix. Then, a u×u 2D mesh is constructed around each point in the coarse point cloud Ycoarse, where u is the resolution parameter of the local mesh, controlling the number of refined points around each coarse point. The 2D mesh coordinates, the global feature vector v, and the 3D coordinates of the point are concatenated along the feature dimension to obtain the input vector. This input vector undergoes a nonlinear transformation using a shared MLP to generate the 3D surface patch coordinates corresponding to the coarse point. The patch is then added to the corresponding coarse point coordinates and converted to global coordinates. All patches are merged to obtain the finely completed point cloud Ydetail, which serves as the final output of the decoder.

[0072] Loss functions: Chamfer distance (CD) and Earth Mover's distance (EMD) loss are used between the coarse point cloud Ycoarse and the coarse ground truth. CD loss is also used between the fine point cloud Y_fine generated by the decoder through local folding operations on the coarse point cloud Ycoarse and the ground truth point cloud Y_gt to measure the difference between the generated point cloud and the real complete point cloud. The network is trained to generate more accurate completion results by optimizing these differences.

[0073] The PCN model's network structure possesses excellent global contour inference and local detail reconstruction capabilities. It can perform high-quality geometric completion of regions with breaks, sparseness, and noise points after multi-view stitching, resulting in significant improvements in the boundary continuity, morphological integrity, and industrial usability of the obtained global complete 3D point cloud model of the tested part.

[0074] S5. Based on the point cloud registration and preliminary attitude estimation, the pose deviation between the measured part and the reference part is further calculated, as well as the pose deviation between the measured part and the real CAD model. The difference between the two is used as the measurement error and attitude deviation difference to correct the 3D attitude estimation results. First, using the registration results obtained in step S4, combined with the Iterative Closest Point (ICP) algorithm or feature-based registration method, the point cloud of the measured part and the point cloud of the reference part are aligned to obtain the pose deviation between them; then, the point cloud of the measured part is aligned with the real value point cloud generated from the standard CAD model to obtain the pose deviation from the real value; subsequently, the difference between the two is compared, and this difference is fed back to the attitude estimation as a correction amount. Through this correction process, the cumulative deviation caused by complex geometric interference and acquisition errors is effectively reduced, ensuring that the final output 3D digital model has high accuracy and reliability.

[0075] Through structured light imaging, improved attitude estimation and registration algorithms, and deep learning-driven point cloud completion mechanisms, this 3D digital model demonstrates excellent performance in terms of boundary contours, surface continuity, and attitude consistency, making it suitable for key application scenarios such as subsequent industrial parts library construction, reverse design, digital twin modeling, and high-precision simulation.

[0076] The proposed method for 3D reconstruction and pose estimation based on binocular structured light relies on the principle of binocular structured light 3D imaging, combined with a modular assembly structure and a three-degree-of-freedom attitude adjustment platform, to achieve complete modeling and high-precision attitude inference of complex targets under multi-angle and multi-pose conditions. It has the advantages of convenient deployment, strong adaptability and high robustness.

[0077] Experimental verification To verify the practicality and effectiveness of this invention, an experimental platform was constructed, and multi-angle modeling experiments were conducted using a typical irregularly shaped part—an industrial irregularly shaped joint—as the test object. Industrial irregularly shaped joints exhibit typical complex boundaries, irregular internal cavity structures, and local occlusion features, which can effectively reflect the modeling accuracy and robustness of the 3D reconstruction and pose estimation system under highly complex working conditions.

[0078] During the experiment, the 3D reconstruction and pose estimation system first selected the appropriate settings based on the structure of the industrial irregular joint and the acquisition line of sight to assemble the left camera, right camera, and structured light projector in the binocular multi-angle pose image acquisition module. After assembly, the built-in program stored in the system control computer automatically loaded the matching intrinsic parameters and baseline configuration, thus eliminating the need for manual calibration and significantly improving deployment efficiency.

[0079] During the testing phase, the industrial irregular-shaped connector was fixed to the center of a three-degree-of-freedom rotating platform using a quick-clamping device, and its attitude was adjusted sequentially in the pitch, roll, and yaw directions. The 3D reconstruction and pose estimation system acquired binocular structured light image pairs for each attitude and recorded the attitude angle data. A timestamp mechanism was used to achieve high-precision synchronization between the image pairs and the attitude angle data, ensuring the accuracy and traceability of the attitude calculation and point cloud registration process.

[0080] The acquired binocular structured light images are reconstructed into dense point clouds using a structured light phase inversion algorithm. Subsequently, combined with the acquired attitude angle information, an improved EPNP algorithm incorporating attitude priors is used for initial pose estimation. Furthermore, a multi-scale geometric constraint ICP algorithm is employed to complete high-precision registration of the point clouds across multiple frames. To eliminate platform interference, the 3D reconstruction and pose estimation system automatically removes non-target point clouds such as fixture edges and reflective surfaces of the platform based on rotation trajectory and template modeling.

[0081] Based on point cloud fusion, a deep learning completion module based on Point Completion Network (PCN) is further introduced to automatically recover the structural information of the inner cavity corners and occluded areas, significantly improving the integrity and boundary continuity of the final obtained 3D digital model.

[0082] The final reconstruction results are compared with the CAD benchmark model of industrial irregular joints, as shown in Table 1. Figure 5 and Figure 6 As shown, the reconstruction error of the 3D reconstruction and pose estimation system is controlled within ±0.5 mm on the main structural parameters (such as maximum outer diameter, inner cavity diameter, vertical height, etc.), the pose error is less than ±0.02°, the point cloud coverage reaches 98.9%, and the details of the obtained 3D digital model are well preserved. This verifies the high accuracy and high robustness of the 3D reconstruction and pose estimation system in the reconstruction of highly complex targets.

[0083] Figure 5 The physical appearance of the tested part (left) and its CAD baseline model (right) are shown to provide an intuitive reference for the actual part shape and design model. Figure 6 The 3D point cloud results obtained by the system are compared with... Figure 5 By comparing the physical object and the CAD benchmark model, it can be seen that the point cloud is highly consistent with the physical object and the CAD model in terms of overall outline, key structure and detailed features, which further illustrates that the system can achieve high precision and high robustness in the 3D reconstruction of complex irregular parts.

[0084] Table 1. Comparison of 3D Reconstruction Results of Industrial Irregular Joints with CAD Benchmark As shown in Table 2, a comparative analysis of different 3D reconstruction methods is presented. The point cloud completion capability is subjectively evaluated based on the completeness and boundary continuity of the obtained 3D digital model, with the processing time being the average processing time per unit. Overall test results demonstrate that the 3D reconstruction and pose estimation method proposed in this invention possesses advantages such as rapid deployment, stable attitude synchronization, accurate registration, and intelligent point cloud completion. It is capable of handling high-precision modeling and digital inspection tasks for typical non-standard irregular parts, such as industrial irregular joints, in industrial settings, and has significant engineering application value.

[0085] Table 2 Comparative Analysis of Different 3D Reconstruction Methods This invention addresses the high-precision perception requirements of non-standard irregular parts in industrial inspection and 3D reconstruction. It proposes a 3D reconstruction and pose estimation system and method based on binocular structured light. This method innovatively integrates a modular structured light assembly mechanism, a three-degree-of-freedom rotation platform control system, and a gyroscope attitude information feedback mechanism to construct a highly flexible and robust 3D perception system.

[0086] This invention enables equipment deployment without manual calibration, significantly shortening measurement preparation time. Combining structured light coding projection and binocular stereo matching technology, it can stably acquire dense point cloud information on complex lighting conditions and irregular curved surfaces. By fusing real-time attitude data from gyroscopes, it guides the rapid registration and stitching of multiple frames of point clouds. The improved algorithm combination of attitude-guided EPNP and multi-scale geometric constraint ICP effectively improves the robustness of pose estimation and the accuracy of point cloud fusion, solving the problems of accumulated error and registration failure commonly found in multi-view modeling in traditional methods.

[0087] In addition, to address the issue of missing points in the point cloud caused by self-occlusion and limited viewpoint, the system introduces a deep learning-based Point Completion Network (PCN) completion algorithm to automatically restore the geometric structure of invisible areas, enhancing the integrity and detail continuity of the 3D model. Meanwhile, the platform rotation trajectory analysis and background template culling mechanism can effectively eliminate interference points introduced by the platform structure and fixtures, improving the purity of the point cloud and post-processing efficiency.

[0088] The resulting 3D model exhibits excellent performance in surface detail restoration, boundary contour preservation, and attitude accuracy control, and can be widely applied to various tasks in industrial settings, such as structural inspection, defect analysis, assembly guidance, and reverse modeling. In summary, the method described in this invention offers advantages such as rapid deployment, stable data acquisition, accurate registration, intelligent completion, and strong adaptability. It provides a complete, efficient, and scalable solution for high-precision modeling and intelligent identification of non-standard and irregularly shaped parts, possessing significant engineering transformation value and promising industrial application prospects.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for 3D reconstruction and pose estimation based on binocular structured light, characterized in that, The steps are as follows: S1. Acquire binocular structured light image pairs and attitude angles of the part under test at different preset attitude angles, and synchronously bind the attitude angles with the binocular structured light images. S2. Based on the acquired binocular structured light image pairs, the disparity information between the binocular images is used to reconstruct the dense three-dimensional point cloud of the surface of the measured part in each pose by combining structured light coding and triangulation principles. S3. Based on the dense 3D point cloud under each pose and the pose angle obtained in step S1, the initial pose estimation is performed using an efficient multi-viewpoint algorithm. Based on the initial pose estimation result and the pose angle, the dense 3D point cloud under each pose is registered using the multi-scale geometrically constrained ICP algorithm to obtain the registered complete point cloud. S4. Using the PCN model, geometric completion is performed on the sparse point cloud in the obtained registered complete point cloud to generate a global complete 3D point cloud model of the tested part.

2. The method for 3D reconstruction and pose estimation based on binocular structured light according to claim 1, characterized in that, The three-dimensional attitude estimation results are corrected by calculating the pose deviation between the measured part and the reference part, and the pose deviation between the measured part and the real CAD model, and taking the difference between the two pose deviations as the measurement error and the attitude deviation difference.

3. The method for 3D reconstruction and pose estimation based on binocular structured light according to claim 1 or 2, characterized in that, The system for acquiring binocular structured light image pairs and attitude angles of the tested part under different preset attitude angles includes: The binocular multi-angle attitude image acquisition module is used to perform coded projection on the part under test and acquire binocular structured light image pairs of the part under test at each attitude angle under the coded projection state. A three-degree-of-freedom rotation platform with gyroscope attitude feedback includes a three-degree-of-freedom rotation platform and an IMU sensor module. The three-degree-of-freedom rotation platform is connected to the IMU sensor module. The three-degree-of-freedom rotation platform is used to adjust the attitude of the tested part by rotating around the X, Y, and Z axes, so that the tested part presents a preset attitude relative to the binocular multi-angle attitude image acquisition module. The IMU sensor module is used to record the attitude angle of the three-degree-of-freedom rotation platform in real time, and bind the attitude angle with the binocular structured light image acquired by the binocular multi-angle attitude image acquisition module through a timestamp mechanism. The 3D reconstruction and pose estimation module is used to receive and perform 3D reconstruction and pose estimation of the tested part based on attitude angles and binocular structured light image pairs. The IMU sensor module is connected to the 3D reconstruction and pose estimation module, the binocular multi-angle pose image acquisition module is connected to the 3D reconstruction and pose estimation module, and the 3D reconstruction and pose estimation module is located in the main control unit of the host computer.

4. The method for 3D reconstruction and pose estimation based on binocular structured light according to claim 3, characterized in that, The binocular multi-angle pose image acquisition module includes a housing and a left camera, a structured light projector, and a right camera disposed inside the housing; inside the housing, the left camera, the structured light projector, and the right camera are arranged sequentially in the horizontal direction; The outer shell includes a lower shell and an upper cover, with the upper cover hinged to one side of the lower shell; the lower shell has three through holes on the side facing the three-degree-of-freedom rotating platform, and the size and shape of the three through holes are respectively matched with the front end shapes of the left camera, the structured light projector and the right camera; The bottom of the lower housing is provided with multiple slots, which can be removably used to accommodate the left camera, the structured light projector, and the right camera.

5. The three-dimensional reconstruction and pose estimation method based on binocular structured light according to claim 4, characterized in that, The method for synchronously binding the attitude angle with the binocular structured light image is as follows: During the acquisition process, the main control unit sends a unified hardware trigger command to the left camera, right camera and structured light projector of the binocular multi-angle attitude image acquisition module: After receiving the trigger signal, the left camera and right camera start the exposure in strict synchronization to ensure that the acquisition time of the binocular image pair is completely consistent with the exposure window. The structured light projector outputs corresponding patterns during camera exposure according to a preset coding sequence; the main control unit assigns a unified timestamp to each frame of acquired data and synchronously binds it to the real-time attitude angle recorded by the IMU sensor module, forming a three-dimensional data association between image-attitude-projection sequence; The card slot includes a first card slot and a second card slot for placing the structured light projector. The first card slot is located at the center of the bottom of the lower housing. Three second card slots at different distances are provided on each side of the first card slot for placing the left camera and the right camera, respectively. The three different distances include three sets of settings: close distance, medium distance, and long distance. The baseline parameter configuration files of the cameras placed in the three second card slots at different distances are stored in the main control unit. The distance measurement ranges for the three gear settings are 300-600 mm, 600-1000 mm, and 1000-1500 mm, respectively. The three-degree-of-freedom rotating platform is equipped with a detachable buckle mechanism that matches the part being tested.

6. The method for three-dimensional reconstruction and pose estimation based on binocular structured light according to claim 4 or 5, characterized in that, Based on the geometry of the part under test and the required distance measurement range, select the appropriate gear and assemble the left camera, right camera, and structured light projector. Fix the part under test to the center of the three-degree-of-freedom rotating platform and start the IMU sensor module. Driven by the three-degree-of-freedom rotating platform, adjust the attitude of the part under test to different preset attitude angles in a preset order. After each part under test is adjusted to a preset attitude angle, record the corresponding attitude angle through the IMU sensor module. The main control unit sends a unified hardware trigger pulse: the left and right cameras are synchronously exposed based on the pulse signal to acquire binocular structured light image pairs under structured light encoded projection. The structured light projector outputs a preset coded pattern sequence during the camera's exposure window. The method for reconstructing the dense three-dimensional point cloud of the surface of the tested part in each pose is as follows: a stereo matching algorithm is used, and depth recovery is performed independently for each pair of binocular structured light images to obtain the dense three-dimensional point cloud.

7. The method for three-dimensional reconstruction and pose estimation based on binocular structured light according to any one of claims 1, 4, and 5, characterized in that, The efficient multi-view algorithm introduces gyroscope attitude priors to construct attitude constraint terms, thereby improving the stability of initial pose estimation under occlusion or viewpoint shift conditions. The ICP algorithm adopts a multi-scale iterative constraint strategy for multi-stage optimization, integrating feature point matching, point cloud normal direction consistency constraints, and attitude guidance brought by attitude angles to optimize point alignment accuracy.

8. The method for 3D reconstruction and pose estimation based on binocular structured light according to claim 7, characterized in that, In the initial pose estimation, the efficient multi-viewpoint algorithm represents each 3D point as a linear combination of four virtual control points: ;in, For the first i A three-dimensional point, For the first j One virtual control point, For the first j The linear coefficients of the virtual control points satisfy: =1; After mapping the virtual control points to the camera coordinate system, and combining them with the camera intrinsic parameter matrix K, the following projection relationship is formed: ； In the formula, For the first i The scale factor of a three-dimensional point , The first i The two-dimensional pixel coordinates of a three-dimensional point on the image plane For the first j The three-dimensional coordinate vectors of each virtual control point in the camera coordinate system; An attitude-guided optimization model is constructed by minimizing the attitude deviation as a regularization term: ； In the formula, Let be the vector of pose parameters to be optimized; This represents the output of the EPNP pose estimation algorithm, which takes the pose parameter vector θ as input. θ gyro For attitude priors, it refers to the attitude angle vector that is acquired and synchronously bound in real time in step S1; The objective function of the ICP algorithm with multi-scale geometric constraints is: ； in, and These are the corresponding points in the target point cloud and the source point cloud, respectively. It is a rigid transformation matrix. For matching point sets.

9. The method for three-dimensional reconstruction and pose estimation based on binocular structured light according to any one of claims 1, 4, 5, and 7, characterized in that, The PCN model is built on an encoder-decoder architecture. The workflow includes: generating a coarse point cloud through a global feature extraction step; then restoring the missing regions based on local folding operations, and finally outputting a complete point cloud with continuous structure and complete details. In the encoder, global feature extraction is achieved through a combination of multilayer perceptron and max pooling operations; in the decoder, the FoldingNet architecture is used to map the two-dimensional mesh to three-dimensional space to restore the point cloud morphology and fill in the missing regions.

10. The method for three-dimensional reconstruction and pose estimation based on binocular structured light according to claim 9, characterized in that, The training phase of the PCN model uses Chamfer distance as the main loss function to measure the generated point cloud. Alignment degree with the target point cloud P: ； In the formula, Represents the target point cloud P and the generated point cloud Q̂ Chamfer distance between them This represents a single point in the target point cloud P. Indicates the generation of point clouds Q̂ A single point in; The encoder takes an incomplete 3D point cloud X as its input, which is a sparse point cloud in the registered complete point cloud. Global features are extracted by stacking two PointNet layers. A shared multilayer perceptron is used to map each point in the 3D point cloud X to a high-dimensional point feature F1. Max pooling is then performed on all high-dimensional point features F1 to obtain a global feature vector g. The global feature vector g is broadcast and concatenated to each high-dimensional point feature F1 to obtain the enhanced feature F2; then, through another set of shared multilayer perceptron and max pooling operations, the final global feature vector v is extracted as the encoder output and used as the input of the downstream decoder. The decoder maps the global feature vector v to an s×3 three-dimensional point coordinate matrix through a multilayer perceptron and a reconstruction layer, generating a coarse point cloud Ycoarse. A two-dimensional grid with a resolution of u×u is constructed around each point in the coarse point cloud Ycoarse, where u is the resolution parameter of the two-dimensional grid. The coordinates of this two-dimensional grid, the global feature vector v, and the three-dimensional coordinates of the point are concatenated along the feature dimension to obtain the input vector. The input vector undergoes a nonlinear transformation through a shared multilayer perceptron to generate patch coordinates of the three-dimensional surface corresponding to the coarse point. These patch coordinates are added to the corresponding coarse point coordinates and converted into global coordinates. All global coordinates are merged to obtain the finely completed point cloud Ydetail, which serves as the final output of the decoder. The difference between the generated point cloud and the real point cloud is measured by the chamfer distance loss and the earth-movement distance loss between the coarse point cloud Ycoarse and the ground truth point cloud. The fine point cloud is generated by the decoder through local folding operations based on the coarse point cloud Ycoarse. The difference between the generated point cloud and the real complete point cloud is measured by the chamfer distance loss.

Images