Fixed and handheld integrated laser scanning method and system

Abstract

The invention relates to the field of three-dimensional scanning, and particularly discloses a fixed and handheld integrated laser scanning method and system, and the method comprises the steps: building a high-precision global coordinate system based on the mechanical and magnetic field reference of a detachable fixed module in a fixed mode, and obtaining an initial point cloud; in a hand-held mode, aligning hand-held data to the global coordinate system by identifying space-time cohesion features common in view with fixed scanning; by taking fixed data as an optimization anchor point, constructing a hierarchical factor graph model fused with an uncertainty weight, and carrying out constraint optimization on the aligned handheld data to correct an accumulative error of the handheld data; and finally, outputting a complete and consistent three-dimensional model through closed-loop verification and parameter self-adaption steps. The system comprises a scanning host, a detachable fixing module and a processing unit. According to the method and the system, the architecture error problem of mode switching is fundamentally solved through a standard unification and anchor point optimization fusion architecture, and high-precision and high-efficiency scanning of single equipment in various complex scenes is realized.

Description

Technical Field

[0001] This invention relates to the field of three-dimensional scanning technology, specifically to a fixed and handheld integrated laser scanning method and system. Background Technology

[0002] 3D scanning technology is a key technology in fields such as industrial inspection, cultural relic digitization, and reverse engineering. Currently, mainstream 3D scanning equipment can be divided into two categories based on their working mode: fixed and handheld. The two types differ fundamentally in principle, architecture, and performance, leading to significant limitations in practical applications.

[0003] Fixed scanners, such as those based on high-precision turntables or photogrammetry systems, typically mount the scanning head on a rigid support or automated turntable. They achieve high-precision registration of multi-view point clouds by precisely controlling the scanning angle or camera position and relying on external calibration parameters, with stitching errors below 0.02mm. Their technological advantages lie in extremely high accuracy and good repeatability. However, these devices are generally bulky, complex to set up, and expensive. Furthermore, their scanning range is strictly limited by the mechanical movement trajectory, making it impossible to effectively acquire data from the back of large objects, internal cavities of equipment, narrow gaps, and other measurement blind spots, resulting in a severe lack of flexibility.

[0004] Handheld scanners, such as those based on visual SLAM or optical marker tracking, offer excellent portability and flexibility. Operators can scan complex curved surfaces and hidden areas freely with their mobile devices. The core technology lies in using built-in sensors, such as IMUs and cameras, to estimate the device's own motion in real time and simultaneously build an environmental map. However, this approach has inherent drawbacks: during long-term, large-area scanning, the cumulative error in pose estimation continuously increases, typically exceeding 0.1 mm / meter, causing point cloud model drift or distortion; furthermore, it faces a high risk of tracking failure and insufficient stability when features are missing, lighting conditions change, or rapid movement occurs.

[0005] Existing technologies attempt to address the needs by using two separate sets of equipment and software-stitching the data, but this does not resolve the fundamental contradiction. The technical problem lies in the fact that fixed-scan and handheld-scan data are generated from completely different spatiotemporal reference frames and processing algorithms. The former relies on an external absolute benchmark, while the latter relies on internal relative calculations, resulting in architectural incompatibility between the two types of data. Simple post-alignment cannot eliminate the systematic biases introduced by these different principles, i.e., architectural errors. Therefore, the market urgently needs an integrated solution that deeply integrates hardware design and data processing algorithms, seamlessly combining high precision and high flexibility on a single device, breaking through the application boundaries of existing technologies. Summary of the Invention

[0006] The purpose of this invention is to provide a fixed and handheld integrated laser scanning method and system to solve the core problem in the prior art where the fixed scanning mode and the handheld scanning mode have different data acquisition principles and processing architectures, resulting in architectural errors that prevent seamlessly achieving both high precision and high flexibility on the same device.

[0007] To solve the above-mentioned technical problems, the present invention specifically provides the following technical solution:

[0008] A fixed and handheld integrated laser scanning method, applied to scanning devices including a detachable fixing module, includes the following steps:

[0009] S1. Based on a fixed scanning mode, construct a high-precision global reference coordinate system to describe the point cloud of the first region of the target object;

[0010] S2. In handheld scanning mode, acquire point cloud data of the second region of the target object, align it with the global reference coordinate system, and generate incremental point cloud data in the reference coordinate system;

[0011] S3. Using the point cloud data in the high-precision global reference coordinate system as a fixed constraint, optimize the incremental point cloud data in the reference coordinate system to correct its cumulative error;

[0012] S4. The optimized incremental point cloud data is fused with the point cloud data in the high-precision global reference coordinate system to output a complete three-dimensional model.

[0013] As a preferred embodiment of the present invention, the scanning device includes a scanner host and a detachable fixing module, which supports quick switching between fixed scanning mode and handheld scanning mode;

[0014] The scanner host includes a laser emitter, a binocular camera, an IMU, an encoder, and a main control chip;

[0015] The detachable fixing module includes a quick-release interface, a magnetic positioning pin, and a tilt sensor.

[0016] As a preferred embodiment of the present invention, the quick-installation interface is a mechanical connection mechanism between the scanner host and the detachable fixed module, such as a dovetail groove + locking mechanism, which supports disassembly and assembly within 1 second.

[0017] The magnetic positioning pin is a strong magnetic adsorption unit set at the bottom of the detachable fixing module, such as a neodymium iron boron magnet, which can be adsorbed onto a metal bracket or a preset magnetic adsorption point, such as a wall or the ground.

[0018] The tilt sensor is an attitude sensor built into the detachable fixed module, which monitors the pitch and yaw angles of the scanning equipment in real time and assists in the calibration of the viewing angle in the fixed scanning mode.

[0019] As a preferred embodiment of the present invention, S1 specifically includes:

[0020] S11. Install the scanner host onto the detachable fixing module via the quick-release interface, and use the magnetic positioning pins at the bottom of the detachable fixing module to magnetically fix the scanning device onto the support surface;

[0021] S12. Start the turntable and control the scanning device to rotate along a preset trajectory. The rotation angle sequence of the turntable is collected in real time through the encoder, and the attitude angle sequence of the scanning device is collected at the same time through the tilt sensor.

[0022] S13. Determine the horizontal reference plane based on the adsorption position provided by the magnetic positioning pin, and establish the initial device coordinate system in combination with the gravity direction in the attitude angle sequence;

[0023] S14. Based on the rotation angle sequence and the initial device coordinate system, generate point cloud data of the first region of the target object through multi-view geometric triangulation calculation, and define the coordinate system of the first region point cloud data as a high-precision global reference coordinate system.

[0024] As a preferred embodiment of the present invention, S2 specifically includes:

[0025] S21. The scanner host is detached from the detachable fixing module and enters the handheld scanning mode. Motion pre-integration is performed through the IMU integrated in the scanner host, and real-time pose tracking is performed in combination with the visual features extracted by the binocular camera. The original point cloud data of the second region of the target object is acquired simultaneously.

[0026] S22. In the initial stage of handheld scanning, the binocular camera identifies mode switching markers, i.e., specific optical markers or stable natural feature points, that have been observed in the previously fixed scanning field of view, as spatiotemporal connection features.

[0027] S23. Based on the spatiotemporal connection features in the three-dimensional coordinates of the high-precision global reference coordinate system and the current handheld local coordinate system, calculate the coordinate transformation matrix from the handheld local coordinate system to the global reference coordinate system;

[0028] S24. Apply the coordinate transformation matrix to the original point cloud data, transform it to the global reference coordinate system, and generate incremental point cloud data in the reference coordinate system.

[0029] As a preferred embodiment of the present invention, S3 specifically includes:

[0030] S31. Construct a hierarchical factor graph optimization model, wherein the scanning pose corresponding to the first region point cloud data in the high-precision global reference coordinate system is set as a fixed node as an optimization anchor point, and the scanning pose corresponding to the incremental point cloud data under the reference coordinate system is set as a variable node to be optimized.

[0031] S32. In the hierarchical factor graph optimization model, add sequential motion constraint edges generated by IMU pre-integration and co-visual geometric constraint edges generated by visual feature matching to the node of the variable to be optimized;

[0032] S33. Add a point-to-surface distance constraint edge based on point cloud overlapping region matching between the fixed node and the variable node to be optimized, so that the optimization anchor point provides global position constraints for the variable node to be optimized;

[0033] S34. Fix the parameters of the fixed node, solve the hierarchical factor graph optimization model, optimize the pose of the variable node to be optimized, and correct the cumulative error of the incremental point cloud data under the reference coordinate system caused by motion estimation.

[0034] As a preferred embodiment of the present invention, in step S3, when constructing the hierarchical factor graph optimization model, different confidence weights are assigned to different types of constraint edges based on the stability of the sensor during fixed scanning and the uncertainty of motion estimation during handheld scanning. Specifically, this includes:

[0035] Based on the reading variance of the tilt sensor during the fixed scanning process, the stability index of the fixed scanning posture is calculated; based on the angle measurement error of the encoder, the motion accuracy index of the turntable is calculated; and the first weighting coefficient of the fixed scanning constraint edge is generated according to the stability index and the motion accuracy index.

[0036] Based on the covariance matrix of the IMU pre-integration, the uncertainty index of handheld scanning motion estimation is calculated, and the second weight coefficient of the handheld scanning constraint edge is generated.

[0037] After normalizing the first and second weight coefficients, the sequence motion constraint edges, common-view geometric constraint edges, and point-to-surface distance constraint edges are respectively assigned to the hierarchical factor graph optimization model, so that the model applies differentiated confidence levels to constraints from different sources during the optimization process.

[0038] As a preferred embodiment of the present invention, S4 specifically includes:

[0039] S41. The hierarchical factor graph optimization model is solved using a nonlinear optimization algorithm to obtain the optimal pose estimates of all nodes of the variable to be optimized;

[0040] S42. Based on the optimal pose estimation, perform coordinate remapping on each point in the incremental point cloud data under the reference coordinate system to obtain the corrected incremental point cloud data and eliminate the cumulative error caused by handheld scanning.

[0041] S43. The corrected incremental point cloud data and the first region point cloud data in the high-precision global reference coordinate system are fused together in a common global reference coordinate system;

[0042] S44. Perform redundant point removal and surface smoothing on the fused point cloud in overlapping areas to generate and output a complete and globally consistent 3D model of the target object.

[0043] As a preferred embodiment of the present invention, the fixed and handheld integrated laser scanning method further includes: closed-loop verification and parameter adaptation, specifically:

[0044] S51. Under the global reference coordinate system, calculate the geometric consistency error index of the fused point cloud in the area where the data overlaps with the fixed and handheld mode data;

[0045] S52. Determine whether the geometric consistency error index exceeds a preset threshold; if it does, generate an adjustment instruction to correct the matching strategy of the spatiotemporal connection feature or adjust the confidence weight allocation in the optimization model.

[0046] S53. Update the relevant parameters based on the adjustment instructions, and re-execute the incremental data association, optimization and fusion process;

[0047] S54. If the geometric consistency error index meets the accuracy requirements, the spatiotemporal connection feature matching strategy and confidence weight allocation scheme finally adopted in this scanning task will be recorded as a set of successful fusion parameters in the adaptive parameter library.

[0048] S55. When starting a new scanning task, retrieve and load matching initial fusion parameters from the adaptive parameter library based on the current scene characteristics to optimize the initialization configuration for subsequent scans.

[0049] A fixed and handheld integrated laser scanning system, used to realize a fixed and handheld integrated laser scanning method, including:

[0050] The hardware platform includes a scanner host integrating a laser emitter, binocular camera, IMU, encoder and main control chip, and a detachable fixing module that can be quickly connected or separated from the scanner host via a quick-release interface. The detachable fixing module is also equipped with an angle sensor and a magnetic positioning pin for providing an external reference. The quick-release interface is a mechanical structure with guiding and self-locking functions to ensure that the scanner host and the detachable fixing module have sub-millimeter repeatability accuracy after repeated disassembly and assembly.

[0051] The processing unit is configured to automatically trigger a fixed scanning mode or a handheld scanning mode in response to the connection status of the hardware platform; it is used to execute the steps of the fixed and handheld integrated laser scanning method to realize the fusion processing of dual-modal data.

[0052] Compared with the prior art, the present invention has the following advantages:

[0053] 1. By creating a unified global reference coordinate system and a hierarchical fusion model with fixed data as the optimization anchor point, the data of high-precision fixed scanning and flexible handheld scanning are deeply integrated at the algorithm level, fundamentally eliminating the systematic deviation caused by mode switching in traditional solutions, and achieving a balance between accuracy and flexibility on a single device.

[0054] 2. Employing uncertainty-aware weight allocation and closed-loop self-verification strategies, the system can dynamically evaluate and optimize the reliability of data from different sources, significantly enhancing reconstruction robustness and the global consistency of the final model in complex environments. Combined with a quick-assembly / disassembly structure, a single device can seamlessly adapt to scanning needs in large-size, high-precision environments, confined spaces, and complex outdoor scenarios.

[0055] 3. Through the co-design of hardware and software, a single device can combine and surpass the functions of two traditional systems. This not only saves on purchase and maintenance costs, but also significantly reduces the operating threshold and time consumption through adaptive parameter learning and fast switching without calibration, achieving a simultaneous improvement in efficiency and ease of use. Attached Figure Description

[0056] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings in the following description are merely exemplary, and those skilled in the art can derive other embodiments based on the provided drawings without creative effort.

[0057] Figure 1 This is a flowchart of the method described in Embodiment 1 of the present invention.

[0058] Figure 2 This is a framework diagram of the system described in Embodiment 2 of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The concepts involved in this application will first be described with reference to the accompanying drawings. It should be noted that the following descriptions of various concepts are only for the purpose of making the content of this application easier to understand and do not constitute a limitation on the scope of protection of this application; furthermore, the embodiments and features in the embodiments of this application can be combined with each other unless otherwise specified. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0061] Example 1

[0062] like Figure 1 As shown, the present invention provides a fixed and handheld integrated laser scanning method, which is applied to a scanning device. The scanning device adopts a fixed and handheld integrated architecture, consisting of two parts: a scanner host and a detachable fixed module. The scanner host includes a laser emitter, a binocular camera, an IMU, an encoder, and a main control chip; the detachable fixed module includes a quick-release interface, a magnetic positioning pin, and a tilt sensor.

[0063] The scanning device achieves rapid switching between fixed scanning mode and handheld scanning mode through the physical connection and separation of the detachable fixing module and the scanner main unit. When the detachable fixing module is combined with the scanner main unit and installed on an external support platform, the scanning device is in fixed scanning mode, relying on a mechanical positioning reference to construct a high-precision global coordinate system. When the detachable fixing module is separated from the scanner main unit, the scanning device switches to handheld scanning mode, relying on the built-in inertial measurement unit to achieve flexible mobile scanning. The two modes share the same optical and computing core, ensuring data homogeneity and fusion consistency.

[0064] The method includes the following steps:

[0065] S1. Based on a fixed scanning mode, construct a high-precision global reference coordinate system to describe the point cloud of the first region of the target object; specifically including:

[0066] S11. Mechanical installation and rigid fixation, specifically:

[0067] S111. The operator holds the scanner main unit and mechanically connects it to the detachable fixing module via the quick-connect interface. The dovetail guide structure of the quick-connect interface ensures that the scanner main unit slides into the socket position of the fixing module in a predetermined direction. When the two are fully engaged, the locking mechanism automatically engages, completing the mechanical locking. The entire installation process is completed within 1 second without the need for auxiliary tools. At this point, the scanner main unit and the detachable fixing module form a rigid integrated structure.

[0068] S112. Using magnetic positioning pins located at the bottom of the detachable fixing module, the assembled scanning equipment is magnetically fixed to the external support surface. The magnetic positioning pins are made of neodymium iron boron strong magnetic material, which can firmly adhere to the metal bracket, pre-embedded magnetic points, or iron worktable surface, forming a stable mechanical positioning reference. This magnetic fixing method not only allows for rapid deployment but also provides the scanning equipment with horizontal shear stiffness and vertical stable support, ensuring that the scanning equipment does not shift or vibrate during subsequent turntable rotation, providing a physical basis for high-precision data acquisition.

[0069] S12. Turntable motion control and sensor data acquisition, specifically:

[0070] After mechanical fixation is completed, the system starts the turntable drive device to control the scanning equipment to rotate according to the preset scanning trajectory.

[0071] During the rotation of the turntable, the encoder monitors and collects the rotation angle sequence of the turntable in real time. The encoder accurately records the instantaneous angle position of the turntable at every moment with an angular resolution of 0.01°, forming a continuous angle-coded data stream.

[0072] Meanwhile, the tilt sensor built into the detachable fixing module synchronously collects the attitude angle sequence of the scanning device. The tilt sensor monitors the pitch and yaw angles of the scanning device relative to the direction of gravity in real time, and detects the tilt state of the device caused by uneven support surface or slight magnetic positioning offset.

[0073] The rotation angle sequence and attitude angle sequence are transmitted to the main control chip in real time through the data synchronization interface, providing complete kinematic parameters for subsequent geometric calculations.

[0074] S13. Establishment of the initial equipment coordinate system based on physical reference and gravity direction, specifically:

[0075] S131. The system determines the horizontal reference plane by the adsorption contact position between the magnetic positioning pin and the support surface. The magnetic positioning pin is set at the bottom of the detachable fixing module, and its bottom surface is a precision-machined flat contact surface. When the magnetic positioning pin is firmly adsorbed on the metal support surface, the contact surface is in close contact with the support surface, forming a physical positioning reference between the scanning equipment and the external environment.

[0076] The system uses the plane containing the bottom surface of the magnetic positioning pin as an approximate representation of the horizontal reference plane. This plane is parallel to the horizontal plane of the earth under ideal installation conditions, providing a two-dimensional reference datum in the horizontal direction for the establishment of the coordinate system.

[0077] S132. The system extracts the attitude angle sequence collected by the tilt sensor in step S12 and analyzes the gravity direction information contained therein. The tilt sensor is built into a detachable fixed module, and its sensitive axis is rigidly connected to the main axis of the scanning device. By measuring the component projection of gravitational acceleration on its sensitive axis, the system outputs the pitch angle and yaw angle of the scanning device relative to the gravity vector in real time.

[0078] The system extracts the average or steady-state value of the gravity direction from the attitude angle sequence, filters out transient vibration noise, and determines the specific directional representation of the gravity vector in the coordinate system of the scanning device body under the current installation state.

[0079] S133. Based on the above physical reference and gravity direction, the system establishes an initial equipment coordinate system. Specifically:

[0080] The projection point of the geometric center of the magnetic positioning pin onto the horizontal reference plane is taken as the origin of the coordinate system. This origin defines the planar position reference of the scanning device in the horizontal plane.

[0081] The opposite direction of the gravity direction obtained by the tilt sensor is used as the positive direction of the Z-axis to ensure that the Z-axis is strictly aligned with the actual gravity direction and to eliminate the uncertainty in the vertical direction caused by the installation posture deviation.

[0082] The X-axis direction is the projection direction of the main optical axis or structural longitudinal axis of the scanning device onto the horizontal reference plane. This direction usually maintains a known geometric relationship with the output optical axis of the laser emitter or the baseline direction of the binocular camera.

[0083] Finally, based on the right-hand rule, the direction of the Y-axis is determined by the cross product of the Z-axis and the X-axis, thus completing the establishment of the three-dimensional right-hand rectangular coordinate system.

[0084] S134. The initial equipment coordinate system fully considers the attitude uncertainty of the actual installation environment. The attitude is corrected by using the absolute gravity reference provided by the tilt sensor to define the physical benchmark of the magnetic positioning pin. This ensures that even if there is a slight tilt on the support surface or a slight deviation in the magnetic positioning pin's adsorption position, the Z-axis of the coordinate system can still be accurately aligned with the gravity direction.

[0085] This coordinate system serves as a temporary reference frame for multi-view geometric triangulation calculations in step S14, providing a unified metric for the generation of point cloud data in the first region and laying the geometric foundation for the subsequent definition of a high-precision global reference coordinate system.

[0086] S14. Multi-view geometric triangulation calculation and high-precision global reference coordinate system definition: After establishing the initial device coordinate system, the system initiates the multi-view geometric triangulation calculation process to generate high-precision point cloud data of the first region of the target object; specifically:

[0087] S141. The main control chip retrieves the rotation angle sequence recorded by the encoder in step S12, and combines it with the initial device coordinate system established in step S13 to calculate the precise rigid body transformation matrix of the scanning device relative to the initial device coordinate system at each sampling moment. Specifically:

[0088] For each angle value in the rotation angle sequence, the system calculates the three-dimensional coordinate positions of the laser emitter optical center and the binocular camera projection center in the initial device coordinate system at that moment, based on the known spatial position and direction vector of the turntable rotation axis in the initial device coordinate system. At the same time, it calculates the attitude angle of the laser plane or structured light pattern relative to the initial device coordinate system, thereby completely describing the external orientation parameters of the scanning device at each viewpoint.

[0089] S142. Based on the image sequence acquired by the binocular camera and the structured light pattern projected by the laser emitter, the system performs multi-view geometric triangulation calculation. In each viewpoint, the laser emitter projects a specifically coded optical pattern onto the surface of the first region of the target object using a multi-line blue laser or a speckle structured light mode, and the binocular camera synchronously acquires the image of the deformed pattern modulated by the object surface.

[0090] The main control chip first performs epipolar correction and stereo matching on the binocular image to extract the two-dimensional pixel coordinates of laser lines or speckle feature points in the image; then, combined with the external orientation parameters under this viewpoint, it uses the principle of spatial forward intersection to calculate the three-dimensional spatial coordinates of each feature point in the initial device coordinate system.

[0091] For surface points with overlapping areas between adjacent viewpoints, the system uses multi-view bundle adjustment optimization to jointly optimize the 3D coordinates of feature points and the external orientation parameters of each viewpoint, minimizing reprojection error and improving point cloud accuracy.

[0092] S143. As the turntable rotates along the preset trajectory, the system iteratively executes the above triangulation calculation from different perspectives, gradually accumulating complete surface information covering the first region of the target object. The main control chip performs rigid body transformation and fusion on the point cloud fragments from each perspective, performs precise registration between perspectives based on the 0.01-degree precision angle information provided by the encoder, eliminates accumulated errors, and generates high-density, high-precision point cloud data for the first region of the target object. The point cloud data for the first region retains sub-millimeter-level geometric details, and all point cloud coordinates are uniformly expressed in the initial device coordinate system.

[0093] S144. The system formally defines the coordinate system containing the point cloud data of the first region as a high-precision global reference coordinate system. This coordinate system inherits from the initial device coordinate system established in step S13, but after refinement and global optimization through multi-view geometric constraints in fixed scanning mode, its origin, coordinate axis directions, and scale units are all determined based on actual physical measurements, possessing clear geometric meaning and high-precision characteristics. Specifically:

[0094] This coordinate system uses the intersection of the turntable's rotation axis and the horizontal reference plane of the initial equipment coordinate system, or the projection point of the geometric center of the magnetic positioning pin, as its origin to ensure correspondence with the mechanical positioning reference. The Z-axis is perpendicular to the horizontal reference plane, i.e., the direction of gravity, to ensure alignment with the gravitational field. The XY plane direction of the initial equipment coordinate system is used as a reference to maintain consistency with the scanning equipment structure. Through this definition, the high-precision first-region point cloud data generated in step S1 becomes the absolute geometric anchor point for associating the handheld scanning data in the subsequent step S2, providing a globally consistent spatial reference framework for the entire scanning process and ensuring coordinate uniformity and measurement accuracy traceability during the fusion of fixed and handheld mode data.

[0095] S2. In handheld scanning mode, acquire point cloud data of the second region of the target object, align it with the global reference coordinate system, and generate incremental point cloud data in the reference coordinate system; specifically including:

[0096] S21. Handheld mode activation and raw point cloud data acquisition, specifically:

[0097] S211. The operator unlocks the locking mechanism of the quick-release interface and smoothly detaches the scanner main unit from the detachable fixed module along the dovetail groove. The scanning device then switches from fixed scanning mode to handheld scanning mode.

[0098] S212. At this time, the scanner host is moved by hand by the operator. The nine-axis IMU integrated inside starts high-frequency data acquisition, performs pre-integration calculation on the movement of the scanner host, and predicts the motion trajectory and attitude change of the device in three-dimensional space in real time, providing high-frequency and low-latency initial values ​​for pose estimation.

[0099] Meanwhile, the binocular camera uses a global shutter CMOS sensor to continuously acquire images of the second region of the target object, extracting texture and geometric features such as edges, corners, and texture gradient points.

[0100] The main control chip integrates the IMU pre-integration results with the visual features extracted by the binocular camera, executes the visual inertial odometry calculation method, and performs real-time pose tracking to correct the cumulative drift error of pure inertial navigation.

[0101] S213. During this process, the laser emitter switches to line laser mode or speckle structured light mode according to the scene requirements, projects an optical pattern onto the second region of the target object, and the binocular camera captures the deformed image modulated by the object surface. The parallax is calculated through a stereo matching algorithm, and then the original point cloud data of the second region of the target object is generated. This data is initially expressed in a handheld local coordinate system that moves with the device in real time.

[0102] S22. Mode switching marker recognition and spatiotemporal connection feature extraction, specifically:

[0103] S221. In the initial stage of handheld scanning, immediately after the scanner main unit has been unlocked via the locking mechanism of the quick-release interface and detached from the detachable fixing module along the dovetail groove, the operator aligns the scanner main unit with the overlapping area between the first region of the target object already covered during the previous fixed scanning process and the second region to be scanned. At this time, the scanner main unit's built-in binocular camera actively searches for and identifies mode switching markers usable for coordinate system one within a wide field of view using a global shutter CMOS sensor. Mode switching markers include two types:

[0104] The first type consists of specific optical markers pre-arranged on the surface of the target object. These markers are strategically pasted or placed on the surface of the target object before fixed scanning. They include reflective markers with high reflectivity under laser irradiation, circular coded markers with unique two-dimensional coding patterns, or LED luminescent markers that actively emit light of specific wavelengths.

[0105] The second category consists of stable natural feature points inherent to the surface of the target object. These feature points have had their three-dimensional coordinates recorded during the previous fixed scanning process. They include sharp edges and corners of the object's surface, edge corners formed by depth discontinuities, centers of high-contrast texture patterns, or other local features with significant geometric salience or texture uniqueness.

[0106] S222. After the binocular camera acquires the image of the current field of view, the main control chip first performs feature extraction and descriptor calculation on the image to generate feature description vectors of candidate feature points in the current field of view.

[0107] Subsequently, the system matches and compares the currently extracted candidate feature points with the feature point library stored in the data cache, which is associated with the first region point cloud data generated during the previous S14 step fixed scan. The feature point library records the precise three-dimensional coordinates, surface normal vectors, and visual descriptor information of each marker point in a high-precision global reference coordinate system.

[0108] By employing nearest neighbor search based on descriptor similarity, geometric consistency verification, such as epipolar constraint checks, and RANSAC random sampling consistency algorithm to eliminate false matches, the system identifies common feature points that are observed in both the current handheld field of view and the previous fixed field of view.

[0109] S223. For each successfully identified mode switching marker, the system calculates its three-dimensional coordinates in the current handheld local coordinate system through binocular stereo vision triangulation, and establishes a correspondence between the known coordinates of the point in the high-precision global reference coordinate system.

[0110] These jointly observed mode-switching markers constitute the spatiotemporal connection features linking fixed scan data and handheld scan data, providing the necessary geometric constraints and corresponding point data pairs for the coordinate transformation matrix calculation in step S23. By completing the identification and matching of such markers in the initial stage of handheld scanning, the system ensures that the handheld scan data can be accurately correlated to the established high-precision global reference coordinate system, avoiding coordinate drift problems caused by the lack of a global reference in subsequent scanning processes.

[0111] S23. Calculation of coordinate transformation matrix, specifically:

[0112] S231. The main control chip extracts the three-dimensional spatial coordinates of each feature point in a high-precision global reference coordinate system from the previously established spatiotemporal connection feature correspondence. These coordinates originate from the mode switching marker position data accurately determined through multi-view geometric triangulation in fixed scanning mode. Simultaneously, it extracts the three-dimensional spatial coordinates of the spatiotemporal connection features in the current handheld local coordinate system. These coordinates are obtained from the current observation position data through real-time triangulation calculation using binocular stereo vision. The system requires that the number of successfully identified common feature points satisfy the minimum mathematical constraint condition for rigid body transformation solution to ensure the geometric determinism of the transformation solution.

[0113] S232. The main control chip executes a rigid body transformation solution algorithm based on singular value decomposition. Specifically:

[0114] First, calculate the center point positions of all feature point sets in the two coordinate systems respectively. Data is digitized by subtracting the corresponding center point from each coordinate to eliminate the influence of translation components on rotation estimation. Then, calculate the cross-covariance matrix of the two centered point sets, which represents the statistical correlation between the two point sets.

[0115] Singular value decomposition is performed on the covariance matrix to obtain orthogonal and diagonal matrices. The optimal rotation matrix is ​​then calculated based on the decomposition results, and the rotation matrix is ​​corrected by a determinant to ensure that it satisfies the normal rotation characteristics of the right-hand coordinate system. The optimal translation vector is then calculated based on the center point deviation and the rotation results. This vector represents the spatial offset between the origins of the two coordinate systems.

[0116] S233. To improve the robustness of the transformation estimation, the main control chip employs a random sampling consensus algorithm framework during the solution process to eliminate potential mismatch points. Specifically:

[0117] The candidate transformation matrix is ​​calculated by randomly selecting the smallest subset of points from all spatiotemporal connectivity features. The reprojection error or 3D coordinate residual of the remaining feature points under the candidate transformation is verified. Points with errors less than a set threshold are identified as interior points, and points with excessive errors are identified as exterior points. The above process is iterated until the optimal transformation estimate with the maximum interior point support is obtained.

[0118] Finally, the rotation matrix and translation vector are refined using the least squares optimization method based on all interior points, minimizing the 3D coordinate transformation residuals of all interior points.

[0119] S234. The main control chip combines the obtained rotation matrix and translation vector to construct a homogeneous transformation matrix. This matrix accurately describes the rigid body transformation relationship from the handheld local coordinate system to the high-precision global reference coordinate system, and fully characterizes the spatial rotation and translation mapping between the two coordinate systems.

[0120] The main control chip further verifies the accuracy and reliability of the coordinate transformation matrix by calculating the reprojection error or Euclidean distance residual of the spatiotemporal connection features after transformation. If the mean residual is less than the preset accuracy threshold, the transformation estimation is confirmed to be effective and applied to the coordinate transformation in subsequent steps. If the residual is too large, the operator is prompted to rescan the overlapping area to obtain more spatiotemporal connection features.

[0121] S24. Original Point Cloud Coordinate Transformation and Incremental Data Generation: After obtaining the coordinate transformation matrix, the main control chip applies this matrix to the original point cloud data generated in step S21. Specifically:

[0122] A homogeneous coordinate transformation is performed on the coordinates of each 3D point in the original point cloud data. This involves converting the point coordinates from the handheld local coordinate system to a high-precision global reference coordinate system using matrix multiplication. After this transformation, the original point cloud data is re-expressed within a globally unified metric framework, generating incremental point cloud data in the same reference coordinate system as the first region's point cloud data. This incremental point cloud data inherits the geometric information of the second region obtained through the flexibility of handheld scanning, and simultaneously achieves high-precision alignment with the fixed scan data through spatiotemporal connectivity features. This provides a unified coordinate system data foundation for the anchor point constraint optimization in the subsequent S3 step.

[0123] S3. Using point cloud data in a high-precision global reference coordinate system as a fixed constraint, optimize the incremental point cloud data in the reference coordinate system to correct its accumulated error; specifically including:

[0124] S31. Construction of hierarchical factor graph optimization model and assignment of confidence weights, specifically:

[0125] S311. The main control chip initializes the graph structure of the hierarchical factor graph optimization model, explicitly defining two types of nodes:

[0126] The first type is fixed nodes, which correspond to the scanning poses of each viewpoint in the first region of the high-precision global reference coordinate system during the generation process of the point cloud data in step S14. These nodes originate from the geometric calculations based on the encoder's high-precision angle measurement and tilt sensor attitude correction in the fixed scanning mode. They have known and accurate spatial coordinates and attitude parameters, and remain fixed in the subsequent optimization process, serving as optimization anchor points to provide an absolute geometric reference for the entire factor map.

[0127] The second category consists of nodes that are variables to be optimized. These nodes correspond to the handheld scanning poses of the incremental point cloud data in the reference coordinate system at each moment during the acquisition process in step S21. These nodes currently carry accumulated uncertainties caused by IMU drift and visual tracking errors, and their pose parameters will be adjusted as optimization variables during the iteration process.

[0128] S312. The core calculation for the main control chip to implement the hierarchical weight allocation strategy is to generate differentiated confidence weight coefficients based on the difference in the reliability of sensor data between fixed scanning and handheld scanning modes:

[0129] a. For fixed-scan related constraints, the system retrieves the attitude angle reading sequence collected by the tilt sensor during the entire turntable rotation process in step S12, calculates the statistical variance of this sequence as a stability index of the fixed-scan attitude, and the smaller the variance, the smoother the turntable rotation and the less vibration interference. At the same time, the nominal angle measurement error parameter of the encoder is retrieved, and combined with the mechanical accuracy of the turntable transmission mechanism, the turntable motion accuracy index is calculated to reflect the system accuracy of angle measurement. Based on the above stability index and motion accuracy index, the main control chip generates a first weighting coefficient through a preset mapping relationship. This coefficient represents the high confidence level of the fixed-scan geometric data.

[0130] b. For handheld scanning-related constraints, the system extracts the covariance matrix estimated in real time during the IMU pre-integration process in step S21. This matrix gradually increases with the pre-integration time, intuitively reflecting the degree of uncertainty accumulation in handheld motion estimation. The main control chip calculates the uncertainty index of handheld scanning motion estimation based on this covariance matrix. The larger the covariance, the less reliable the motion estimation, thus generating a relatively small second weighting coefficient to characterize the lower confidence level of the handheld scanning kinematic data.

[0131] S313. The main control chip normalizes the first and second weighting coefficients to ensure they are on a uniform numerical scale. During factor graph construction, the normalized first weighting coefficient is assigned to the point-to-surface distance constraint edges involving the geometric relationship between fixed and variable nodes, ensuring that the high-precision point cloud data of the fixed scan exerts a strong constraint on the handheld scanning pose. The normalized second weighting coefficient is assigned to the sequence motion constraint edges and common-view geometric constraint edges within the handheld scan, allowing these internal constraints to have a certain degree of flexibility in optimization.

[0132] By using this hierarchical and differentiated weight allocation, a factor graph optimization model with distinct confidence levels is constructed. This ensures that the optimization process prioritizes respecting the high-precision geometric facts of fixed scanning while making reasonable use of the kinematic information from handheld scanning, thus achieving a balance between global consistency and local smoothness.

[0133] S32. Adding internal constraint edges for handheld scanning: Specifically, in the hierarchical factor graph optimization model, the main control chip adds two types of internal constraint edges between the nodes of the variable to be optimized to maintain the local consistency of the handheld scanning trajectory.

[0134] S321. The first type is the sequential motion constraint edge, which is generated based on the IMU pre-integration result in step S21. It connects the handheld scanning pose nodes at adjacent time points, encodes the kinematic constraints measured by the IMU, and reflects the relative rotation and translation relationship of the scanner host between adjacent time points. The weight of this constraint edge adopts the second weight coefficient and is adaptively adjusted according to the magnitude of the IMU pre-integration covariance. The longer the integration time, the larger the covariance, and the lower the constraint weight accordingly.

[0135] S322. The second type is the common-view geometric constraint edge, which is generated based on the visual feature matching results extracted by the binocular camera in step S21. It connects the pose nodes that observe the same visual features at different handheld scanning times, encodes the geometric observation constraints, and reflects the spatial positional relationship that the scanning device should be in at these times to meet the requirements of visual feature reprojection consistency. This constraint edge is also assigned a second weight coefficient, which is dynamically adjusted according to the magnitude of the reprojection error of feature matching. The higher the matching accuracy, the greater the weight.

[0136] By adding these two types of constraint edges, a rigid skeleton is constructed inside the handheld scanning trajectory to prevent non-rigid distortion of the handheld pose during the optimization process.

[0137] S33. Construction of cross-mode point-to-surface distance constraint edges and application of global position constraints. Specifically: between fixed nodes and nodes to be optimized, the main control chip constructs point-to-surface distance constraint edges based on the matching of overlapping areas of point clouds, thereby realizing the global position constraint of the optimized anchor point on the handheld scanning trajectory.

[0138] S331. The system identifies the spatial overlap region between the first region point cloud data and the incremental point cloud data in the reference coordinate system. This identification process takes the mode switching marker point identified in step S22 as the center, expands to the three-dimensional spatial neighborhood, and extracts the local point cloud subsets in the first region point cloud data located within a preset distance range around the marker point, as well as the local point cloud subsets in the corresponding region of the incremental point cloud data. These two subsets spatially cover the same surface area of ​​the target object, forming the overlapping region of cross-mode data fusion.

[0139] S332. The main control chip performs local geometric analysis on the point cloud data of the first region within the overlapping area. Since the point cloud data of the first region comes from high-precision multi-view triangulation of fixed scanning, it has high density and low noise characteristics. The system performs normal vector estimation and planarity detection on this subset of point clouds, and extracts local planar patches or smooth curved surface patches with stable normal vector directions as reference surfaces.

[0140] For regions that are approximately planar, the plane equation is fitted and its normal vector and center point position are recorded; for curved regions, a continuous geometric reference surface model is established using local quadratic surface fitting or implicit surface representation. These reference surfaces represent the true geometric shape of the target object's surface and serve as the benchmark for distance measurement.

[0141] S333. The system performs coordinate association on the point clouds of corresponding overlapping regions in the incremental point cloud data under the reference coordinate system. For each point in the incremental point cloud data, the main control chip calculates its vertical distance to the reference surface extracted from the first region's point cloud data, i.e., the point-to-surface distance. This distance reflects the geometric deviation between the incremental point cloud and the high-precision fixed scan data under the current handheld pose estimation. The smaller the distance, the more accurate the handheld pose estimation; the larger the distance, the more accumulated error or registration deviation exists. The system selects corresponding points with reliable normal vector consistency and generates a point-to-surface distance residual term.

[0142] S334. Based on the aforementioned point-to-surface distance residuals, the main control chip constructs constraint edges in the factor graph connecting fixed nodes and nodes of the variable to be optimized. Each point-to-surface distance constraint edge is associated with a specific node of the variable to be optimized, corresponding to the pose at the handheld scanning moment, and a fixed reference surface, which originates from the fixed node of the point cloud data in the first region.

[0143] In the optimization model, the constraint edge encodes the following geometric constraint relationship: when the pose parameters of the variable node to be optimized are adjusted, the coordinates of the associated incremental point cloud change accordingly, and its distance to the reference surface should be minimized. The system assigns the first weight coefficient generated in step S31 to these point-to-surface distance constraint edges. This coefficient is relatively large, reflecting a strong degree of confidence in the fixed scan high-precision data.

[0144] S335. By constructing this cross-modal point-to-surface distance constraint edge, the optimized anchor point provides a global positional constraint for the variable nodes to be optimized. Specifically, regardless of how the sequence motion constraints and common-view geometric constraints within the handheld scan adjust the local trajectory shape, the point-to-surface distance constraint forces the point cloud acquired by the handheld scan in the overlapping area to remain geometrically aligned with the point cloud data of the first region, preventing the overall handheld scan trajectory from shifting or rotating. This constraint rigidly anchors the local coordinate system of the handheld scan onto a high-precision global reference coordinate system, ensuring that the optimized handheld pose remains consistent with the fixed scan data in absolute space, laying the geometric foundation for the seamless fusion of the two types of point cloud data in the subsequent S4 step.

[0145] S34. Optimization and error correction of hierarchical factor graph, specifically:

[0146] S341. After completing the factor graph construction, the main control chip fixes the pose parameters of all fixed nodes, keeping them unchanged as a high-precision global reference; at the same time, it releases the pose parameters of the nodes to be optimized as optimization variables. The system uses a nonlinear least squares optimization algorithm based on the Gauss-Newton method or the Levenberg-Marquardt method to iteratively solve the hierarchical factor graph optimization model.

[0147] S342. During the optimization process, the algorithm calculates the contribution of different constraint residuals to the objective function based on the weight coefficients assigned to each constraint edge. It prioritizes satisfying the high-weight point-to-surface distance constraint (i.e., the first weight coefficient) to ensure alignment between handheld scan data and fixed scan data. Next, it satisfies the medium-weight co-view geometric constraints to maintain visual consistency. Finally, it satisfies the relatively low-weight sequence motion constraints to maintain motion smoothness as much as possible while ensuring overall alignment. The algorithm iteratively optimizes and adjusts the pose parameters of the nodes to be optimized, minimizing the weighted sum of squared residuals from all constraint edges.

[0148] S343. After optimization and convergence, the system recalculates the spatial coordinates of each point in the incremental point cloud data under the reference coordinate system based on the corrected handheld scanning pose, correcting the accumulated errors caused by IMU drift and visual feature tracking errors in step S21. The optimized incremental point cloud data is output, which has eliminated trajectory drift during the handheld scanning process and maintains geometric consistency with the first region point cloud data, laying the foundation for fusion in step S4.

[0149] S4. Fuse the optimized incremental point cloud data with the point cloud data in the high-precision global reference coordinate system to output a complete 3D model; specifically including:

[0150] S41. Solving the hierarchical factor graph optimization model and estimating the optimal pose, specifically:

[0151] The S411 main control chip employs a nonlinear least squares optimization algorithm to numerically solve the previously constructed hierarchical factor graph optimization model. Specifically, the Gauss-Newton method or the Levenberg-Marquardt method is selected as the core solver. During the optimization initialization phase, the system clearly distinguishes the processing methods for two types of nodes:

[0152] For fixed nodes representing the scanning pose of point cloud data in the first region, the system sets their pose parameters as strict constants, which remain unchanged in all subsequent iterations. These nodes serve as optimization anchors and continuously provide an absolute geometric reference for the entire optimization process.

[0153] For the variable node representing the pose at each moment of handheld scanning, the system uses its current pose estimate as the initial value for optimization and marks it as an adjustable variable, ready to enter the iterative optimization process.

[0154] S412. During the iterative optimization process, the main control chip first calculates the residual values ​​of all constraint edges in the factor graph:

[0155] For the point-to-surface distance constraint edge connecting the fixed node and the variable node to be optimized, calculate the vertical distance residual from the observation point in the incremental point cloud to the reference surface of the first region;

[0156] For the sequential motion constraint edges connecting adjacent handheld moments, calculate the deviation between the relative transformation of the poses at two moments and the pre-integrated measurement values ​​of the inertial measurement unit;

[0157] For the common-view geometric constraint edges connecting common-view features, calculate the reprojection error or 3D position deviation of the visual features.

[0158] The system weights these residuals based on the first or second weight coefficients assigned to each constraint edge in the previous steps, and constructs an objective function in the form of a weighted residual sum of squares. This function quantitatively evaluates the overall consistency error under the current pose estimation.

[0159] S413. The solver calculates the gradient information of the objective function with respect to the parameters of the nodes to be optimized, and constructs an approximate Hessian matrix or information matrix, forming a linear system describing the local curvature of the optimization problem. By solving this linear system, the correction increment of the pose of the nodes to be optimized is calculated. The system applies this correction increment to the current pose estimate, updating the spatial position and attitude parameters of the nodes to be optimized. This process is executed iteratively, with each iteration shifting the objective function value in a decreasing direction, gradually improving the global consistency of the handheld scanning trajectory.

[0160] S414. The optimization process continues until the preset convergence criterion is met. The main control chip monitors the rate of change of the objective function value between two adjacent iterations. When the rate of change is less than a preset threshold, the optimization is considered to have converged to a local optimum; or when the number of iterations reaches a preset upper limit, the optimization is forcibly terminated to ensure real-time computation.

[0161] After convergence, the system outputs the optimal pose estimate of all variable nodes to be optimized. This result represents the optimal spatial trajectory of handheld scanning under the constraint of fixed high-precision anchor points, which takes into account both the kinematic consistency of the inertial measurement unit and the consistency of visual geometric observation. It eliminates the systematic bias of pure inertial navigation cumulative drift and visual tracking error, and provides an accurate geometric reference for subsequent coordinate remapping.

[0162] S42. Incremental point cloud coordinate remapping and cumulative error correction. Specifically: Based on the optimal pose estimation obtained in step S41, the main control chip performs coordinate remapping on the incremental point cloud data in the reference coordinate system.

[0163] For each 3D point in the incremental point cloud data, the system extracts the optimal pose estimation parameters at the corresponding acquisition time, constructs the rigid body transformation relationship from the handheld local coordinate system to the high-precision global reference coordinate system at that time, and applies this transformation relationship to the coordinate calculation of the point.

[0164] Through this coordinate remapping process, the positional deviations in the original incremental point cloud data caused by handheld motion estimation errors are systematically corrected, and the coordinates of each point are recalculated to a geometric framework that is strictly consistent with the point cloud data of the first region.

[0165] This step generates corrected incremental point cloud data, which eliminates the accumulated errors generated during handheld scanning and has the same spatial measurement benchmark and coordinate origin as the fixed scanning data.

[0166] S43. Dual-mode point cloud data fusion, specifically: Under a common high-precision global reference coordinate system, the main control chip fuses the corrected incremental point cloud data with the point cloud data of the first region:

[0167] The first region point cloud data comes from the fixed scanning mode and has the characteristics of high density, low noise and high precision; the corrected incremental point cloud data comes from the handheld scanning mode and covers the geometric information of the second region that cannot be reached by the fixed scanning.

[0168] The system integrates the two types of point cloud data into the same data structure for management, ensuring that the point cloud coordinates, normal vectors, and color information in three-dimensional space are all expressed using a unified coordinate system.

[0169] Through this fusion step, the geometric information of the first and second regions of the target object is seamlessly stitched together in the global coordinate system to form an original fused point cloud covering the entire surface of the object, preserving the advantages of each mode of scanning: the geometric accuracy of fixed scanning and the coverage integrity of handheld scanning.

[0170] S44. For point cloud post-processing and 3D model generation, specifically: the main control chip performs post-processing operations on the fused complete point cloud data to improve model quality:

[0171] S441. First, redundant points in the overlapping area are removed: Since the point cloud data of the first region and the corrected incremental point cloud data have spatial overlap in the overlapping area defined in step S33, the system uses voxel grid filtering or spatial hashing to identify and delete redundant points within the distance threshold in the overlapping area, retaining points with stronger geometric representativeness, reducing the data storage scale and eliminating potential noise superposition.

[0172] S442. Then, surface smoothing is performed: To address the local noise or uneven point cloud density that may be introduced by handheld scanning, moving least squares surface fitting, Laplacian smoothing, or a local neighborhood-based normal vector consistency filtering algorithm is used to smooth the point cloud surface, suppressing high-frequency noise while maintaining the sharpness of geometric feature edges.

[0173] S443. After post-processing, the system generates and outputs a complete and globally consistent 3D model of the target object. This model has a unified coordinate system, continuous surface representation, and consistent geometric accuracy, fully reflecting the 3D morphological information of the target object, and can be used for subsequent measurement analysis, reverse engineering, or visualization applications.

[0174] S5. Closed-loop verification and parameter self-adaptation, specifically:

[0175] S51. Calculation of geometric consistency error index in overlapping area: Specifically, under a high-precision global reference coordinate system, the main control chip performs quality assessment on the fused complete point cloud data, focusing on analyzing the geometric consistency of fixed scanning and handheld scanning data in the overlapping area;

[0176] S511. The system first identifies the spatially overlapping area between the first region point cloud data and the corrected incremental point cloud data. This area is the part of the target object surface covered by both scanning modes.

[0177] S512. For the point cloud data within the overlapping region, the main control chip calculates multi-dimensional geometric consistency error indicators, including:

[0178] Point-to-surface distance statistics, which are calculated as the vertical distance distribution from the observation point in the incremental point cloud to the fitting reference surface of the first region point cloud;

[0179] The normal vector consistency index compares the deviation of the angle between the normal vectors of two point clouds at corresponding surface positions to detect inconsistencies in surface orientation.

[0180] Point cloud density difference index, which assesses the difference in the uniformity of point cloud distribution between two data sources within an overlapping area;

[0181] The local surface roughness comparison index reflects the smoothness of the transition at the fusion boundary.

[0182] S513. The system integrates the above indicators to form a comprehensive geometric consistency error evaluation, which quantitatively represents the global consistency level of the current fusion results.

[0183] S52. Error threshold judgment and adjustment instruction generation, specifically:

[0184] S521. The main control chip compares the calculated geometric consistency error index with a preset accuracy threshold. If the error index exceeds the preset threshold, it indicates that there is a significant registration deviation or unresolved cumulative error in the current fusion result, and the system generates targeted adjustment instructions.

[0185] S522. Adjustment instructions include two types of parameter adjustment strategies:

[0186] The first category is the modification of spatiotemporal connection feature matching strategy, including adjusting the response threshold of the binocular camera feature extraction algorithm to obtain richer or more stable feature points, modifying the recognition and matching tolerance of mode switching marker points to relax or tighten the corresponding point screening conditions, or adjusting the sampling number and inlier determination threshold of the random sampling consistency algorithm to improve the ability to eliminate mismatches.

[0187] The second category is to optimize the confidence weight allocation of the model, including adjusting the relative ratio of the first weight coefficient and the second weight coefficient, increasing or decreasing the weight ratio of fixed scanning anchor point constraints relative to handheld scanning internal constraints, or dynamically adjusting the confidence parameters of specific constraint edges according to error distribution characteristics, so as to balance geometric fit and motion smoothness.

[0188] S53. Parameter update and process re-execution, specifically: based on the generated adjustment instructions, the main control chip updates the relevant algorithm parameters;

[0189] S531. If the adjustment instruction involves a spatiotemporal connection feature matching strategy, the system returns to step S2 and re-executes the mode switching marker recognition and coordinate transformation matrix calculation based on the new feature extraction parameters or matching threshold to generate incremental point cloud data in a new reference coordinate system.

[0190] S532. If the adjustment instruction involves confidence weight allocation, the system returns to step S3, reconstructs the hierarchical factor graph optimization model based on the new first weight coefficient or second weight coefficient, and performs optimization solution and error correction.

[0191] S533. After completing the parameter update and re-execution, the system re-enters step S4 to generate new fusion results, forming a closed-loop iterative optimization process until the geometric consistency meets the accuracy requirements.

[0192] S54. Successful parameter configuration record and adaptive parameter library update: Specifically, when the geometric consistency error index meets the preset accuracy requirements, the main control chip records the final parameter configuration used in this scanning task as a successful experience.

[0193] S541. The system extracts the final set of parameters for the spatiotemporal connectivity feature matching strategy, including the type and threshold of the feature extraction algorithm, the selection and tolerance settings of the matching algorithm, and the parameter combination of the mismatch elimination strategy; at the same time, it records the confidence weight allocation scheme in the hierarchical factor graph optimization model, including the specific values ​​of the first weight coefficient and the second weight coefficient, the weight normalization method, and the weight mapping relationship of the constraint edge type.

[0194] S542. The system associates the above parameter configurations with the scene feature description of the current scanning task, such as the surface material characteristics of the target object, ambient lighting conditions, geometric complexity level, and the timing characteristics of switching between fixed and handheld modes, and stores them as a set of successful fusion parameter entries in the adaptive parameter library to form a reusable knowledge accumulation.

[0195] S55. New task scene awareness and adaptive loading of initial parameters, specifically:

[0196] S551. When starting a new scanning task, the main control chip first analyzes the current scene characteristics, including sensing the ambient light intensity through sensors, assessing the geometric complexity and surface texture richness of the target object through pre-scanning, and obtaining application scene type information through the human-computer interaction interface.

[0197] S552. Based on these scene feature descriptions, the system performs a similarity search in the adaptive parameter library, matching the most similar successful fusion parameter entries in the historical records to the current scene features. The spatiotemporal connectivity feature matching strategy and confidence weight allocation scheme from the retrieved parameter entries are automatically loaded into the system configuration as initial fusion parameters, replacing the default initialization parameters.

[0198] By loading historically validated optimized parameters, new tasks can obtain near-optimal feature matching and optimized configurations in the initial stage, significantly reducing the number of iterations for closed-loop verification and parameter adjustment, accelerating the convergence of the scanning process, and improving overall job efficiency and user experience.

[0199] Example 2

[0200] like Figure 2 As shown, a fixed and handheld integrated laser scanning system is used to realize a fixed and handheld integrated laser scanning method, including:

[0201] The hardware platform includes a scanner main unit integrating a laser emitter, binocular camera, IMU, encoder, and main control chip. The scanner main unit is the core functional unit of the scanning device, integrating a multimodal sensing system and real-time computing capabilities. It also includes a detachable mounting module that can be quickly connected or disconnected from the scanner main unit via a quick-release interface. This detachable mounting module includes magnetic positioning pins, a tilt sensor, and a power supply interface. The detachable mounting module provides mechanical support, attitude reference, and expansion interfaces for the scanning device, enabling rapid assembly and disassembly with the scanner main unit and ensuring accurate and repeatable positioning.

[0202] A1. The laser emitter employs a multi-line blue laser source, capable of switching between line laser mode and speckle structured light dual-mode. In fixed scanning mode, the laser emitter, in conjunction with a rotating turntable, acquires high-density point cloud data of the first region of the target object through multi-line scanning. In handheld scanning mode, the laser emitter projects random speckle patterns in speckle structured light mode, enhancing the texture features of complex curved surfaces and narrow areas, and assisting the binocular camera in stereo matching. The blue laser wavelength is typically 405nm or 450nm, effectively suppressing ambient light interference and improving measurement accuracy on dark or highly reflective surfaces.

[0203] A2. The binocular camera is equipped with a global shutter CMOS image sensor, simultaneously acquiring images of the target surface illuminated by a laser emitter. Global shutter technology ensures distortion-free image capture even in high-speed motion scenarios, i.e., during handheld scanning. Texture features extracted by the binocular camera are used for spatiotemporal connectivity feature recognition and incremental data association in step S2, while extracted geometric features, such as edges and corners, are used for real-time pose tracking in handheld scanning mode. The binocular baseline length is optimized to balance near-range blind spots and long-range accuracy, adapting to multi-scale scanning needs from delicate parts to large objects.

[0204] The A3.IMU is a nine-axis inertial measurement unit, integrating a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, forming the core of motion sensing for the scanning device. In handheld scanning mode, the IMU performs pre-integration calculations to predict the scanner's trajectory and attitude changes in three-dimensional space in real time, providing initial pose estimation for incremental data association in step S2 and compensating for rapid perspective changes caused by handheld movements. The magnetometer provides an absolute orientation reference, suppressing long-term drift in pure inertial navigation. In fixed scanning mode, IMU data is used to monitor the turntable's rotational stability and external vibration interference, assisting the tilt sensor in attitude calibration.

[0205] A4. The encoder is a high-precision angle sensor, installed at the mechanical interface between the scanner host and the detachable fixed module, or integrated into the external turntable drive unit. It records the turntable rotation angle information in fixed scanning mode, achieving an angle recording accuracy of 0.01°. During the global reference establishment process in step S1, the angle encoding information provided by the encoder is combined with the marker point images acquired by the binocular camera. Through geometric relationships, the precise attitude of the turntable under each viewpoint is calculated, achieving high-precision stitching of multi-view point clouds with an error of <0.02mm. The encoder data serves as the key input for coordinate system transformation and point cloud fusion by the main control chip, forming the geometric basis of the high-precision reference coordinate system.

[0206] A5. The main control chip is an embedded high-performance computing unit responsible for real-time processing of laser point cloud data and multi-source sensor fusion data. In fixed scanning mode, the main control chip executes a feature matching algorithm based on turntable calibration and a graph optimization global correction algorithm; in handheld scanning mode, it executes a real-time positioning algorithm based on IMU pre-integration and geometric / texture feature tracking. The main control chip has a built-in hardware acceleration module that supports computationally intensive tasks such as point cloud registration and deep learning inference, ensuring rapid solution of large-scale optimization models in the S3 anchor point constraint optimization step, and achieving real-time fusion and error correction of fixed and handheld point cloud data.

[0207] The B1 quick-release interface is the mechanical connection mechanism between the scanner main unit and the detachable fixed module. It features guiding and self-locking functions, employing a dovetail groove and latch design. The dovetail groove provides high-precision mechanical guidance and positioning reference, ensuring repeatability of position when the scanner main unit and the detachable fixed module are engaged. The latch mechanism enables rapid locking and releasing, allowing the scanner main unit to be installed, fixed, or disassembled within one second. When the latch is locked, the scanner main unit and the fixed module form a rigid connection, ensuring structural stability in fixed scanning mode. When the latch is released, the scanner main unit can quickly detach from the fixed module, switching to handheld operation. The quick-release interface design eliminates the need for tools during mode switching and recalibration of the scanner main unit's internal sensors, ensuring seamless data transfer between dual modes and sub-millimeter repeatability after repeated assembly and disassembly of the scanner main unit and the detachable fixed module.

[0208] B2. The magnetic positioning pin, located at the bottom of the detachable fixing module, is a strong magnetic adsorption unit made of neodymium iron boron permanent magnet material, possessing high magnetic energy and strong adsorption force. The magnetic positioning pin firmly attaches the entire scanning device to the metal bracket. Pre-set magnetic adsorption points, such as embedded steel plates in walls or floors, or other iron support structures, allow for the rapid establishment of a fixed scanning workstation. Compared to traditional clamp fixing, the magnetic method offers advantages such as flexible deployment, strong adaptability, and no damage to the support surface, making it particularly suitable for temporary setups in industrial sites, complex outdoor terrain, or artifact scanning scenarios. The magnetic positioning pin works in conjunction with the quick-release interface to ensure the stability and anti-tipping capability of the fixing module in the adsorbed state, providing a reliable mechanical positioning reference for the high-precision benchmark establishment in step S1.

[0209] B3. The tilt sensor, integrated into the detachable mounting module, is an attitude monitoring sensor that monitors the pitch and yaw angles of the scanning equipment relative to the direction of gravity in real time. In fixed scanning mode, the tilt sensor data is used to assist in viewpoint calibration, detect the installation level of the mounting module and the perpendicularity of the turntable's rotation axis, and compensate for system attitude deviations caused by uneven support surfaces or slight tilting of the magnetic positioning. The tilt sensor and the IMU within the scanner host form redundant measurements, and data fusion improves the accuracy of attitude calculation. In step S1, the absolute angle reference provided by the tilt sensor is combined with the relative rotation angle recorded by the encoder to jointly construct a high-precision spatial attitude description, ensuring that the establishment of the reference coordinate system is unaffected by the installation attitude.

[0210] B4. The detachable mounting module features an external power interface, supporting connection to an external power adapter or battery pack. In fixed scanning mode, this power interface provides continuous power to the scanner, extending continuous working time and avoiding the battery life limitations of handheld mode. The power interface and quick-release interface are integrated, automatically establishing electrical connection when the scanner is installed and automatically disconnecting upon removal, eliminating the need for additional plugging and unplugging. This design ensures that fixed scanning mode is suitable for long-duration, multi-view scanning of complex objects, while handheld mode maintains lightweight and portability, only being activated when additional scanning is required.

[0211] The processing unit is configured to automatically trigger a fixed scanning mode or a handheld scanning mode in response to the connection status of the hardware platform; it is used to execute the steps of the fixed and handheld integrated laser scanning method to achieve the fusion processing of dual-modal data.

[0212] As can be seen from the above description, the embodiments of the present invention achieve the following technical effects:

[0213] This invention fundamentally eliminates architectural errors caused by mode switching, achieving true integrated fusion while ensuring ultra-high precision and high efficiency. Traditional solutions only mechanically combine two modes, leaving the data layer fragmented. This invention establishes a global reference coordinate system based on magnetic positioning and gravity sensing, and seamlessly links handheld data to this reference system using shared spatiotemporal connectivity features, ensuring coordinate system uniformity from the data source. Crucially, by constructing a hierarchical factor graph model with fixed data as immovable optimization anchor points, the high-precision information of the fixed mode is transformed into a spatial reference constraining the drift of handheld data. This achieves deep fusion of heterogeneous data from an algorithmic framework perspective, rather than simple splicing. This allows the fixed mode to guarantee global precision with an error of <0.03mm, while the handheld mode can focus on detailed scanning, improving efficiency by over 50%. It achieves the performance level previously required two separate systems on a single device.

[0214] Through intelligent optimization based on uncertainty awareness, the reconstruction accuracy, robustness, and ease of operation in complex scenarios are significantly improved. Addressing issues such as large cumulative errors in handheld scanning and the impact of posture stability on fixed scanning, this invention introduces an uncertainty-aware confidence weight allocation mechanism. The system dynamically assesses the uncertainty of data from different modes and applies differentiated constraints to the optimization model. This intelligent optimization strategy enables the system to automatically adjust the confidence level when facing complex conditions such as vibration, rapid movement, or feature loss, thereby significantly improving the global consistency accuracy of the final fused point cloud and the robustness of the overall reconstruction process. Combined with the feature of mode switching within one second without recalibration achieved through a quick-installation interface, the user experience is greatly optimized while maintaining accuracy, enabling the device to flexibly handle all scenario requirements, from high-precision scanning of large objects to flexible scanning in narrow spaces, and even scanning in complex outdoor environments.

[0215] Equipped with closed-loop self-verification and parameter evolution capabilities, this invention achieves system-level replacement with a single device, significantly reducing overall costs and improving adaptability and reliability. Going beyond the scope of a single scan, this invention incorporates closed-loop verification and parameter adaptation steps. The system automatically verifies the quality of the fusion results and performs parameter tuning and iterative reprocessing for tasks that fail to meet standards, ensuring consistently reliable output. Successful processing parameters are stored as experiential knowledge for learning and optimization of initialization settings for similar scenarios in the future. This enables the device to evolve and become increasingly intelligent with use. Ultimately, a single device can replace two independent systems—a traditional fixed scanner and a handheld scanner—not only saving significant procurement and maintenance costs but also overcoming the fundamental limitations of traditional devices—limited functionality and application scope—through its adaptive and all-scenario capabilities, providing a highly reliable one-stop solution.

[0216] The embodiments and / or implementation methods described above are merely preferred embodiments and / or implementation methods for implementing the technology of the present invention, and are not intended to limit the implementation methods of the technology of the present invention in any way. Any person skilled in the art can make some modifications or alterations to other equivalent embodiments without departing from the scope of the technical means disclosed in the present invention, but these should still be regarded as the technology or embodiments that are substantially the same as the present invention.

[0217] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. The above descriptions are only preferred embodiments of this application. It should be noted that due to the limitations of written expression, while there are objectively infinite specific structures, those skilled in the art can make several improvements, modifications, or changes without departing from the principles of this application, and can also combine the above technical features in an appropriate manner. These improvements, modifications, changes, or combinations, or the direct application of the inventive concept and technical solution to other situations without modification, should all be considered within the scope of protection of this application.

Claims

1. A fixed and handheld integrated laser scanning method, applied to a scanning device including a detachable fixing module, characterized in that, include: Based on a fixed scanning mode, a high-precision global reference coordinate system is constructed to describe the point cloud of the first region of the target object. In handheld scanning mode, point cloud data of the second region of the target object is acquired and aligned with the global reference coordinate system to generate incremental point cloud data in the reference coordinate system. Using the point cloud data in the high-precision global reference coordinate system as a fixed constraint, the incremental point cloud data in the reference coordinate system is optimized to correct its accumulated error, specifically including: A hierarchical factor graph optimization model is constructed. The scanning pose corresponding to the first region point cloud data in the high-precision global reference coordinate system is set as a fixed node as the optimization anchor point, and the scanning pose corresponding to the incremental point cloud data in the reference coordinate system is set as the variable node to be optimized. In the hierarchical factor graph optimization model, sequential motion constraint edges and common-view geometric constraint edges are added to the variable node to be optimized. Between the fixed node and the variable node to be optimized, a point-to-surface distance constraint edge based on point cloud overlapping region matching is added, so that the optimization anchor point provides global position constraints for the variable node to be optimized. The parameters of the fixed node are fixed, and the hierarchical factor graph optimization model is solved to optimize the pose of the variable node to be optimized and correct the cumulative error caused by motion estimation in the incremental point cloud data in the reference coordinate system. The optimized incremental point cloud data is fused with the point cloud data in the high-precision global reference coordinate system to output a complete 3D model.

2. The fixed and handheld integrated laser scanning method according to claim 1, characterized in that, The scanning device includes a scanner main unit and a detachable fixing module, supporting quick switching between fixed scanning mode and handheld scanning mode; The scanner host includes a laser emitter, a binocular camera, an IMU, an encoder, and a main control chip; The detachable fixing module includes a quick-release interface, a magnetic positioning pin, and a tilt sensor.

3. The fixed and handheld integrated laser scanning method according to claim 2, characterized in that, The quick-release interface is a mechanical connection mechanism between the scanner host and the detachable fixed module. The magnetic positioning pin is a strong magnetic adsorption unit set at the bottom of the detachable fixing module, which can be adsorbed onto the metal bracket or the preset magnetic adsorption point. The tilt sensor is an attitude sensor built into the detachable fixed module, which monitors the pitch and yaw angles of the scanning equipment in real time and assists in the calibration of the viewing angle in the fixed scanning mode.

4. The fixed and handheld integrated laser scanning method according to claim 3, characterized in that, The construction of a high-precision global reference coordinate system for describing the point cloud of the first region of the target object based on a fixed scanning mode specifically includes: The scanner host is installed on the detachable fixing module via a quick-release interface, and the scanning device is magnetically fixed to the support surface using the magnetic positioning pins at the bottom of the detachable fixing module. The turntable is started and the scanning device is controlled to rotate along a preset trajectory. The rotation angle sequence of the turntable is collected in real time through the encoder, and the attitude angle sequence of the scanning device is collected through the tilt sensor. The horizontal reference plane is determined based on the adsorption position provided by the magnetic positioning pin, and the initial device coordinate system is established by combining the gravity direction in the attitude angle sequence. Based on the rotation angle sequence and the initial device coordinate system, point cloud data of the first region of the target object is generated through multi-view geometric triangulation calculation, and the coordinate system of the first region point cloud data is defined as a high-precision global reference coordinate system.

5. The fixed and handheld integrated laser scanning method according to claim 4, characterized in that, In handheld scanning mode, acquiring point cloud data of the second region of the target object and aligning it with the global reference coordinate system to generate incremental point cloud data in the reference coordinate system specifically includes: The scanner host is detached from the detachable fixing module and enters the handheld scanning mode. Motion pre-integration is performed through the IMU integrated in the scanner host, and real-time pose tracking is performed in combination with the visual features extracted by the binocular camera, and the original point cloud data of the second region of the target object is acquired simultaneously. In the initial stage of handheld scanning, the binocular camera identifies mode switching markers that have been observed in the previously fixed scanning field of view, serving as spatiotemporal connection features. Based on the spatiotemporal connection features, the three-dimensional coordinates in the high-precision global reference coordinate system and the current handheld local coordinate system are used to calculate the coordinate transformation matrix from the handheld local coordinate system to the global reference coordinate system. The coordinate transformation matrix is ​​applied to the original point cloud data to transform it into the global reference coordinate system, thereby generating incremental point cloud data in the reference coordinate system.

6. The fixed and handheld integrated laser scanning method according to claim 5, characterized in that, When constructing the hierarchical factor graph optimization model, different confidence weights are assigned to different types of constraint edges based on the stability of the sensor during fixed scanning and the uncertainty of motion estimation during handheld scanning. Specifically, this includes: Based on the reading variance of the tilt sensor during the fixed scanning process, the stability index of the fixed scanning posture is calculated; based on the angle measurement error of the encoder, the motion accuracy index of the turntable is calculated; and the first weighting coefficient of the fixed scanning constraint edge is generated according to the stability index and the motion accuracy index. Based on the covariance matrix pre-integrated by the IMU, the uncertainty index of handheld scanning motion estimation is calculated, and the second weight coefficient of the handheld scanning constraint edge is generated. After normalizing the first and second weight coefficients, the sequence motion constraint edges, common-view geometric constraint edges, and point-to-surface distance constraint edges are respectively assigned to the hierarchical factor graph optimization model, so that the model applies differentiated confidence levels to constraints from different sources during the optimization process.

7. The fixed and handheld integrated laser scanning method according to claim 6, characterized in that, The process of fusing the optimized incremental point cloud data with the point cloud data in the high-precision global reference coordinate system to output a complete 3D model specifically includes: The hierarchical factor graph optimization model is solved using a nonlinear optimization algorithm to obtain the optimal pose estimates of all nodes of the variable to be optimized. Based on the optimal pose estimation, each point in the incremental point cloud data under the reference coordinate system is remapped to obtain the corrected incremental point cloud data, thus eliminating the cumulative error caused by handheld scanning. The corrected incremental point cloud data is fused with the first region point cloud data in the high-precision global reference coordinate system under a common global reference coordinate system. Redundant points in overlapping areas of the fused point cloud are removed and the surface is smoothed to generate and output a complete and globally consistent 3D model of the target object.

8. The fixed and handheld integrated laser scanning method according to claim 7, characterized in that, Also includes: Closed-loop verification and parameter adaptation are as follows: Under the global reference coordinate system, calculate the geometric consistency error index of the fused point cloud in the area overlapping with the fixed and handheld mode data; Determine whether the geometric consistency error index exceeds a preset threshold; If the value exceeds the limit, an adjustment instruction is generated to correct the matching strategy of the spatiotemporal connection feature or adjust the confidence weight allocation in the optimization model. Based on the adjustment instructions, update the relevant parameters and re-execute the incremental data association, optimization and fusion process; If the geometric consistency error index meets the accuracy requirements, the matching strategy and confidence weight allocation scheme of the spatiotemporal connection feature finally adopted in this scanning task will be recorded as a set of successful fusion parameters in the adaptive parameter library. When starting a new scanning task, matching initial fusion parameters are retrieved from the adaptive parameter library based on the current scene characteristics to optimize the initialization configuration for subsequent scans.

9. A fixed and handheld integrated laser scanning system, characterized in that, The method for implementing the fixed and handheld integrated laser scanning method according to any one of claims 1-8 includes: The hardware platform includes a scanner host integrating a laser emitter, a binocular camera, an IMU, an encoder, and a main control chip, as well as a detachable fixing module that can be quickly connected or separated from the scanner host via a quick-release interface. The detachable fixing module is also equipped with an tilt sensor and a magnetic positioning pin for providing an external reference. The quick-release interface is a mechanical structure with guiding and self-locking functions. The processing unit is configured to automatically trigger a fixed scanning mode or a handheld scanning mode in response to the connection status of the hardware platform; it is used to perform each step of the fixed and handheld integrated laser scanning method according to any one of claims 1-8 to realize the fusion processing of dual-modal data.

Images