Iterative calibration method for multi-view laser three-dimensional scanning system
By employing a multi-view iterative calibration method, utilizing a calibration board and auxiliary signals, and combining stereo vision algorithms, the position of the laser 3D scanning system is calculated and rendered in real time. This solves the problem of users being unable to move to a specified position in traditional calibration, and achieves efficient and accurate image data acquisition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HANGZHOU FEIBAI 3D TECH CO LTD
- Filing Date
- 2022-12-12
- Publication Date
- 2026-05-15
AI Technical Summary
In the calibration process of traditional laser 3D scanning systems, it is difficult for users to guide the device to a designated position to complete the acquisition of image data using a single-viewpoint guide.
A multi-view iterative calibration method is adopted, which uses a calibration board to construct a world coordinate system. Combined with auxiliary pointers, first adjustment signals and second adjustment signals, the position and attitude of the laser 3D scanning system are rendered and calculated in real time through binocular stereo vision relationship and corresponding point matching algorithm, so as to assist users to move accurately to the designated position for image acquisition.
It achieves user-friendly multi-view guidance, assists users in accurately moving mobile devices through various methods, improves the efficiency and accuracy of image data acquisition, supports 2D and 3D guidance, and simplifies the calibration process.
Smart Images
Figure CN116245957B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the technical field of laser three-dimensional scanning systems, specifically relating to an iterative calibration method for a multi-view laser three-dimensional scanning system. Background Technology
[0002] Currently, laser 3D scanning systems require the device to establish a specific posture relationship with a known calibration object during calibration, constrained within a threshold range. Once the threshold constraint is reached, the corresponding image data is acquired for subsequent calculations. However, because handheld laser 3D scanners are in a 6DOF (6 Degrees of Freedom) state, constraints on the XYZ axes and the angles along each XYZ axis are necessary. Traditional guidance methods often employ 2D approaches and single-viewpoint guidance, making it very difficult for users to move the device to the designated position to acquire the corresponding image data. Summary of the Invention
[0003] To overcome the shortcomings of existing technologies, this invention provides an iterative calibration method for a multi-view laser 3D scanning system, which solves the problem that traditional guidance methods mostly use 2D approaches and single-view guidance, making it very difficult for users to move the device to the designated position to complete the acquisition of the corresponding image data.
[0004] One embodiment of the present invention provides an iterative calibration method for a multi-view laser three-dimensional scanning system, comprising:
[0005] A calibration board is used by the user to construct a world coordinate system. The center of the world coordinate system renders an image calibration board according to the known real calibration board, the proportional relationship, and the calibration threshold that the user expects to reach.
[0006] A laser 3D scanning system is used to constrain the angles corresponding to the X, Y, Z axes and the three axes to which the user is expected to move, and to display the scanned 3D model in the world coordinate system.
[0007] The laser three-dimensional scanning system includes an auxiliary pointer, a first adjustment signal, and a second adjustment signal;
[0008] The auxiliary pointer is used to help determine the left and right rotation angles and guides the user to center the device.
[0009] The first adjustment signal is used to assist the user in aligning the height and angle;
[0010] The second adjustment signal is used to assist the user in aligning the calibration plate center;
[0011] The colors of the first adjustment signal and the second adjustment signal are different.
[0012] In one embodiment, the laser three-dimensional scanning system further includes:
[0013] Binocular stereo vision is used to obtain the precise 2D and 3D coordinates of the Mark points on the calibration board. If the Mark coordinates on the calibration board are known, the known Mark point information can be substituted into the planar simulation legal or PNP algorithm to obtain the coordinates.
[0014] The RT matrix and distortion coefficients of a single camera relative to the world coordinate system can be determined by observing the same calibration plate through the binocular stereo vision relationship formed by two cameras.
[0015] In one embodiment, the laser three-dimensional scanning system further includes:
[0016] The corresponding point matching algorithm can be used to determine the spatial positioning relationship of the Mark points, thereby obtaining the attitude and position information of the real 3D scanner system in space and displaying it in the world coordinate system.
[0017] In one embodiment, when the user presses the UI interaction start calibration button to start calibration, the laser 3D scanning system calculates the position of the calibration plate and the 2D coordinate point information of the prior calibration plate by using the image matching algorithm and the 3D stereo vision reconstruction algorithm.
[0018] In this process, the RT transformation matrix between the two can be solved by the rigid body transformation of the former, the actual attitude solved, and the previously known position information of the calibration plate. After obtaining the attitude matrix, the coordinate system can be unified and aligned so as to guide the user to the expected specified position in the same coordinate system.
[0019] By continuously performing real-time calculations, the positions of the real laser 3D scanning system, the ideal laser 3D scanning system, and the calibration plate in the same world coordinate system are rendered in real time.
[0020] In one embodiment, the auxiliary pointer and other icons assist the user in constraining 6DOF.
[0021] In one embodiment, when a user's mobile device causes the actual position and the ideal position of the 3D model of the laser 3D scanning system to coincide within the expected threshold, and the actual position and the ideal position of the 3D model coincide with the expected threshold by more than 50%, it is determined that the actual scanner has entered the ideal acquisition position, and image capture is performed.
[0022] The threshold can be set arbitrarily.
[0023] In one embodiment, during the image capture process, the standard model does not move on the display. The user only needs to place the real model calculated in real time onto the standard position model, where the ICON assists the user to reach the desired position.
[0024] In one embodiment, once the guidance for a location is completed and one acquisition is finished, the standard model for the next location is refreshed and rendered to continue guiding the user to complete the acquisition of the next pose.
[0025] In one embodiment, the guidance process is repeated until N images are acquired, fine calibration calculations are performed, and the intrinsic and extrinsic parameters of the left and right cameras of the laser 3D scanning system, as well as the RT matrix between the two cameras, are updated.
[0026] In one embodiment, the laser 3D scanning system uses a 3D engine as its core to render the calibration plate and the 3D model scanned by the laser 3D scanning system in the ideal position in space.
[0027] The iterative calibration method for a multi-view laser 3D scanning system provided in the above embodiments has the following beneficial effects:
[0028] 1. The user constructs a world coordinate system using a calibration board and displays the scanned 3D model and calibration board in an ideal position in space using a laser 3D scanning system. The 3D model can be displayed from multiple 3D perspectives. An auxiliary pointer is used to help determine the left and right rotation angles of the 3D model and guides the user to center it. The first adjustment signal is used to help the user align the height and angle, and the second adjustment signal is the lower red auxiliary crosshair to help the user align the center of the calibration board. Multi-angle constraint of the target point ensures that the position where the device needs to be moved is clearly presented. Furthermore, the iterative calibration method of the multi-view laser 3D scanning system of this invention can also be used in 2D mode, using a single perspective for guidance, giving the user multiple choices.
[0029] 2. When the user presses the "Start Calibration" button in the UI interaction to begin calibration, the laser 3D scanning system calculates the position of the calibration board and its 2D coordinates using image matching and 3D stereoscopic vision reconstruction algorithms. Since these are rigid body transformations, the RT transformation matrix between them can be solved using the actual solved posture and the previously known position information of the calibration board. After obtaining the posture matrix, the coordinate system can be unified and aligned, guiding the user to the desired location within the same coordinate system. Through continuous real-time calculation and rendering of the real handheld laser 3D scanning system, the ideal handheld laser 3D scanning system, and the calibration board's position in the same world coordinate system, this invention calculates and displays the device's current position in real time, showing the desired location for the user. This user-friendly guidance allows the user to move the 3D scanning device to the designated location to complete data acquisition. Attached Figure Description
[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.
[0031] Figure 1 This is a flowchart of an iterative calibration method for a multi-view laser three-dimensional scanning system according to the present invention;
[0032] Figure 2 This is a schematic diagram of the structure of the laser three-dimensional scanning system of the present invention;
[0033] Figure 3 This is a schematic diagram of the ideal spatial position of the 3D model of the present invention;
[0034] Figure 4 This is a schematic diagram showing the overlap between the real model and the ideal model of the present invention;
[0035] Figure 5 This is a schematic diagram of image capture according to the present invention. Detailed Implementation
[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.
[0037] It should be noted that if the embodiments of the present invention involve directional indications (such as up, down, left, right, front, back, etc.), the directional indications are only used to explain the relative positional relationship and movement of the components in a specific posture. If the specific posture changes, the directional indications will also change accordingly.
[0038] Furthermore, if the embodiments of this invention involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the use of "and / or" or "and / or" throughout the text includes three parallel solutions. For example, "A and / or B" includes solution A, solution B, or a solution where both A and B are satisfied simultaneously. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed by this invention.
[0039] Please see Figure 1-5 As shown, one embodiment of the present invention provides an iterative calibration method for a multi-view laser 3D scanning system, comprising:
[0040] A calibration board is used by the user to construct a world coordinate system. The center of the world coordinate system renders an image calibration board according to the known real calibration board, the proportional relationship, and the calibration threshold that the user expects to reach.
[0041] A laser 3D scanning system is used to constrain the angles corresponding to the X, Y, Z axes and the three axes to which the user is expected to move, and to display the scanned 3D model in the world coordinate system.
[0042] The laser three-dimensional scanning system includes an auxiliary pointer, a first adjustment signal, and a second adjustment signal;
[0043] The auxiliary pointer is used to help determine the left and right rotation angles and guides the user to center the device.
[0044] The first adjustment signal is used to assist the user in aligning the height and angle;
[0045] The second adjustment signal is used to assist the user in aligning the calibration plate center;
[0046] The colors of the first adjustment signal and the second adjustment signal are different.
[0047] In one embodiment, the laser three-dimensional scanning system further includes:
[0048] Binocular stereo vision is used to obtain the precise 2D and 3D coordinates of the Mark points on the calibration board. If the Mark coordinates on the calibration board are known, the known Mark point information can be substituted into the planar simulation legal or PNP algorithm to obtain the coordinates.
[0049] The RT matrix and distortion coefficients of a single camera relative to the world coordinate system can be determined by observing the same calibration plate through the binocular stereo vision relationship formed by two cameras.
[0050] In one embodiment, the laser three-dimensional scanning system further includes:
[0051] The corresponding point matching algorithm can be used to determine the spatial positioning relationship of the Mark points, thereby obtaining the attitude and position information of the real 3D scanner system in space and displaying it in the world coordinate system.
[0052] In one embodiment, when the user presses the UI interaction start calibration button to start calibration, the laser 3D scanning system calculates the position of the calibration plate and the 2D coordinate point information of the prior calibration plate by using the image matching algorithm and the 3D stereo vision reconstruction algorithm.
[0053] In this process, the RT transformation matrix between the two can be solved by the rigid body transformation of the former, the actual attitude solved, and the previously known position information of the calibration plate. After obtaining the attitude matrix, the coordinate system can be unified and aligned so as to guide the user to the expected specified position in the same coordinate system.
[0054] By continuously performing real-time calculations, the positions of the real laser 3D scanning system, the ideal laser 3D scanning system, and the calibration plate in the same world coordinate system are rendered in real time.
[0055] In one embodiment, the auxiliary pointer and other icons assist the user in constraining 6DOF.
[0056] In one embodiment, when a user's mobile device causes the actual position and the ideal position of the 3D model of the laser 3D scanning system to coincide within the expected threshold, and the actual position and the ideal position of the 3D model coincide with the expected threshold by more than 50%, it is determined that the actual scanner has entered the ideal acquisition position, and image capture is performed.
[0057] The threshold can be set arbitrarily.
[0058] In one embodiment, during the image capture process, the standard model does not move on the display. The user only needs to place the real model calculated in real time onto the standard position model, where the ICON assists the user to reach the desired position.
[0059] In one embodiment, once the guidance for a location is completed and one acquisition is finished, the standard model for the next location is refreshed and rendered to continue guiding the user to complete the acquisition of the next pose.
[0060] In one embodiment, the guidance process is repeated until N images are acquired, fine calibration calculations are performed, and the intrinsic and extrinsic parameters of the left and right cameras of the laser 3D scanning system, as well as the RT matrix between the two cameras, are updated.
[0061] In one embodiment, the laser 3D scanning system uses a 3D engine as its core to render the calibration plate and the 3D model scanned by the laser 3D scanning system in the ideal position in space.
[0062] The above description is merely a preferred embodiment of the present invention and does not limit the patent scope of the present invention. Any equivalent structural transformations made using the contents of the present invention's specification and drawings under the inventive concept of the present invention, or direct / indirect applications in other related technical fields, are included within the patent protection scope of the present invention.
Claims
1. An iterative calibration method for a multi-view laser three-dimensional scanning system, characterized in that, include: A calibration board is used by the user to construct a world coordinate system. The center of the world coordinate system renders an image calibration board according to the known real calibration board, the proportional relationship, and the calibration threshold that the user expects to reach. The laser 3D scanning system is used to constrain the angles corresponding to the X, Y, Z and three axes that the user is expected to move to, and to display the scanned 3D model in the world coordinate system. The laser 3D scanning system is based on a 3D engine and renders the 3D model of the calibration plate and the laser 3D scanning system in the ideal position in space. The laser three-dimensional scanning system includes an auxiliary pointer, a first adjustment signal, and a second adjustment signal; The auxiliary pointer is used to help determine the left and right rotation angles and guides the user to center the device. The first adjustment signal is used to assist the user in aligning the height and angle; The second adjustment signal is used to assist the user in aligning the calibration plate center; Wherein, the colors of the first adjustment signal and the second adjustment signal are different from each other; The laser 3D scanning system also includes: a binocular stereo vision system, used to obtain the precise 2D and 3D coordinates of the Mark points on the calibration board. If the Mark coordinates on the calibration board are known, the known Mark point information can be substituted into the plane simulation legal or PNP algorithm to obtain the coordinates. The RT matrix and distortion coefficients of a single camera relative to the world coordinate system can be determined by observing the same calibration plate through the binocular stereo vision relationship formed by two cameras.
2. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 1, characterized in that, The laser three-dimensional scanning system also includes: The corresponding point matching algorithm can be used to determine the spatial positioning relationship of the Mark points, thereby obtaining the attitude and position information of the real 3D scanner system in space and displaying it in the world coordinate system.
3. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 2, characterized in that, When the user presses the UI interaction start calibration button to start calibration, the laser 3D scanning system calculates the position of the calibration board and the 2D coordinate point information of the prior calibration board by using the image matching algorithm and 3D stereo vision reconstruction algorithm. In this process, the RT transformation matrix between the two can be solved by the rigid body transformation of the former, the actual attitude solved, and the previously known position information of the calibration plate. After obtaining the attitude matrix, the coordinate system can be unified and aligned so as to guide the user to the expected specified position in the same coordinate system. By continuously performing real-time calculations, the positions of the real laser 3D scanning system, the ideal laser 3D scanning system, and the calibration plate in the same world coordinate system are rendered in real time.
4. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 1, characterized in that, The auxiliary pointer and other icons are used to assist the user in constraining 6DOF.
5. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 1, characterized in that, When the user's mobile device causes the actual position and the ideal position of the 3D model of the laser 3D scanning system to coincide within the expected threshold, and the actual position and the ideal position of the 3D model coincide with the expected threshold by more than 50%, it is determined that the real scanner has entered the ideal acquisition position, and image capture is performed. The threshold can be set arbitrarily.
6. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 2, characterized in that, During the image capture process, the standard model remains stationary on the display. Users only need to place the real-time calculated model onto the standard position model, where the ICON assists the user in reaching the desired location.
7. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 3, characterized in that, Once the guidance for one location is complete and one data acquisition is finished, the standard model for the next location is refreshed and rendered to continue guiding the user to complete the acquisition of the next pose.
8. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 2, characterized in that, Repeat the guidance process until N images are acquired, perform fine calibration calculations, and update the intrinsic and extrinsic parameters of the left and right cameras and the RT matrix between the two cameras of the laser 3D scanning system.
9. The iterative calibration method for a multi-view laser three-dimensional scanning system as described in claim 1, characterized in that, The laser three-dimensional scanning system also includes: 3D scanners, display devices, and image acquisition devices; The 3D scanner is simultaneously connected to the display device and the image acquisition device, and interacts with both the display device and the image acquisition device via signals. The 3D scanner also includes a processor and storage; The processor executes the steps of the method as described in any one of claims 1 to 8 by invoking programs or instructions stored in memory.