Three-dimensional scanning method, three-dimensional scanning device, three-dimensional scanning system, and storage medium

Abstract

The application relates to a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system and a storage medium, wherein the three-dimensional scanning method comprises the following steps: obtaining a first pose of a scanning device in a tracking device camera coordinate system when the scanning device performs scanning, the first pose being obtained by tracking the scanning device in real time by a tracking device; obtaining a conversion relationship between a camera coordinate system of the scanning device and a scanning device coordinate system; the conversion relationship is determined based on observation results and the first pose of the scanning device and the tracking device on a target at the same time when the scanning device performs scanning; determining a camera relative pose between the scanning device and the tracking device according to the conversion relationship and the first pose; and converting scanning data collected by the scanning device to the camera coordinate system of the tracking device according to the camera relative pose. The joint calibration before scanning is omitted, so that the overall scanning efficiency of the tracking type three-dimensional scanning is improved.

Description

Technical Field

[0001] This application relates to the field of 3D scanning, and in particular to 3D scanning methods, 3D scanning devices, 3D scanning systems and storage media. Background Technology

[0002] Tracking-type 3D scanning systems combine optical, mechanical, and electronic technologies to achieve precise measurement and digital reconstruction of the 3D shape of objects. A tracking-type 3D scanning system consists of a scanning device and a tracking device. The scanning device moves to different positions to scan the object under test, while the tracking device continuously tracks the scanning device to determine its spatial position after movement.

[0003] Currently, in tracking-based scanning systems, joint calibration of the tracking and scanning devices is often required before the scanning device performs the scan. This joint calibration process involves placing a calibration board in the tracking device's field of view, ensuring that both the tracking and scanning devices can simultaneously observe the calibration board. This determines the transformation relationship between the camera coordinate system and the scanning device's coordinate system. The result of this joint calibration depends on the accuracy of the tracking device. When the tracking device is changed to track the scanning device, joint calibration needs to be performed again based on the new tracking device, resulting in low overall scanning efficiency for tracking-based 3D scanning.

[0004] There is currently no effective solution to the problem that the overall scanning efficiency of tracking-type 3D scanning is low due to the need for joint calibration before scanning. Summary of the Invention

[0005] This embodiment provides a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system, and a storage medium to solve the problem in related technologies that require joint calibration before scanning, resulting in low overall scanning efficiency of tracking-type three-dimensional scanning.

[0006] In a first aspect, this embodiment provides a three-dimensional scanning method for a three-dimensional scanning system, the three-dimensional scanning system including a scanning device, a tracking device, and a target, the method comprising:

[0007] When the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device is obtained by the tracking device tracking the scanning device in real time.

[0008] The transformation relationship between the camera coordinate system and the coordinate system of the scanning device is obtained; the transformation relationship is determined by the observation results obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, and the first pose is determined.

[0009] Based on the transformation relationship and the first pose, the relative camera pose between the scanning device and the tracking device is determined;

[0010] Based on the relative pose of the camera, the scanning data collected by the scanning device is converted to the camera coordinate system of the tracking device.

[0011] In some embodiments, the conversion relationship is determined based on the observation results obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, and the first pose determination, including:

[0012] The target is observed by the scanning device at a first moment, and the position and orientation of the target in the camera coordinate system of the scanning device are determined to obtain the first observation result;

[0013] The tracking device observes the target at a first moment, determines the position and orientation of the target in the camera coordinate system of the tracking device, and obtains a second observation result.

[0014] The tracking device observes the scanning device at a first moment, and determines the position and orientation of the scanning device in the camera coordinate system of the tracking device to obtain the first pose;

[0015] The transformation relationship is determined based on the first observation result, the second observation result, and the first pose.

[0016] In some embodiments, the method further includes:

[0017] If a prior transformation relationship already exists, the transformation relationship is updated based on the latest observation and the latest first pose.

[0018] In some embodiments, the method further includes:

[0019] If the transformation relationship cannot be obtained and the observation result cannot be obtained, the transformation processing of the scanning data currently acquired by the scanning device shall be terminated.

[0020] In some embodiments, if the scanning device and the tracking device cannot observe the target features set on the target at the same time, or if the observed target features do not meet the preset feature splicing conditions, it is determined that the acquisition of the observation result has failed.

[0021] In some embodiments, the method further includes:

[0022] When the first pose at the current moment cannot be obtained, the first observation result of the scanning device on the target and the second observation result obtained by the tracking device synchronously observing the target are obtained; the relative pose of the camera between the scanning device and the tracking device is determined based on the first observation result and the second observation result.

[0023] In some embodiments, after converting the scanning data acquired by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera, the method further includes:

[0024] The observation results obtained from the first pose and the observations of the target by the scanning device and the tracking device at the same time are jointly optimized.

[0025] In some embodiments, the method further includes:

[0026] Based on the scanning data in the camera coordinate system of the tracking device, the three-dimensional reconstruction of the object under test is completed.

[0027] Secondly, this embodiment provides a three-dimensional scanning device for a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device, a tracking device, and a target. The three-dimensional scanning device includes: a first acquisition module, a second acquisition module, a solving module, and a conversion module; wherein:

[0028] The first acquisition module is used to acquire, when the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device, obtained by the tracking device tracking the scanning device in real time.

[0029] The second acquisition module is used to acquire the transformation relationship between the camera coordinate system and the coordinate system of the scanning device; the transformation relationship is based on the observation results obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning and the first pose determination.

[0030] The solving module is used to determine the camera relative pose between the scanning device and the tracking device based on the transformation relationship and the first pose;

[0031] The conversion module is used to convert the scanning data collected by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera.

[0032] Thirdly, this embodiment provides a three-dimensional scanning system, including: a scanning device, a tracking device, a target, and a processor; wherein the target is disposed within the tracking range of the tracking device; the scanning device and the tracking device are respectively connected to the processor;

[0033] The scanning device is used to perform scanning on the object to be tested;

[0034] The tracking device is used to observe the scanning device and the target respectively when the scanning device performs a scan;

[0035] The processor is used to execute the three-dimensional scanning method described in the first aspect.

[0036] Fourthly, this embodiment provides a storage medium storing a computer program that, when executed by a processor, implements the three-dimensional scanning method described in the first aspect above.

[0037] Compared with related technologies, this embodiment provides a 3D scanning method, a 3D scanning device, a 3D scanning system, and a storage medium. The 3D scanning method, during scanning, acquires the first pose of the scanning device in the camera coordinate system of the tracking device, obtained by real-time tracking of the scanning device by the tracking device; acquires the transformation relationship between the camera coordinate system and the scanning device coordinate system; the transformation relationship is determined based on the observation results and first pose obtained by the scanning device and the tracking device observing the target at the same moment during scanning; based on the transformation relationship and the first pose, the relative camera pose between the scanning device and the tracking device is determined; based on the relative camera pose, the scanning data acquired by the scanning device is transformed to the camera coordinate system of the tracking device. This method enables the solution of the relative camera pose between the scanning device and the tracking device during scanning, using a target, eliminating the need for joint calibration before scanning, thereby improving the overall scanning efficiency of tracking-based 3D scanning.

[0038] Details of one or more embodiments of this application are set forth in the following drawings and description to make other features, objects and advantages of this application more readily apparent. Attached Figure Description

[0039] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0040] Figure 1 This is a flowchart of a three-dimensional scanning method according to an embodiment of this application;

[0041] Figure 2This is a flowchart of a three-dimensional scanning method according to an embodiment of this application;

[0042] Figure 3 This is a flowchart of a three-dimensional scanning method according to some embodiments of this application;

[0043] Figure 4 This is a schematic diagram of the coordinate system of a scanning device according to an embodiment of this application;

[0044] Figure 5 This is a schematic diagram of the camera coordinate system of a scanning device according to an embodiment of this application;

[0045] Figure 6 This is a schematic diagram of the camera coordinate system of a tracking device according to an embodiment of this application;

[0046] Figure 7 This is a structural block diagram of a three-dimensional scanning device according to an embodiment of this application;

[0047] Figure 8 This is a schematic diagram of the structure of a three-dimensional scanning system according to an embodiment of this application. Detailed Implementation

[0048] To better understand the purpose, technical solution, and advantages of this application, the application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, the technical or scientific terms used in this application shall have the general meaning understood by one of ordinary skill in the art to which this application pertains. Words such as “a,” “an,” “an,” “the,” “the,” and “these” used in this application do not indicate quantitative limitation and may be singular or plural. The terms “comprising,” “including,” “having,” and any variations thereof used in this application are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include steps or modules (units) not listed, or may include other steps or modules (units) inherent to these processes, methods, products, or devices. Words such as “connected,” “linked,” and “coupled” used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. “Multiple” used in this application refers to two or more. “And / or” describes the relationship between related objects, indicating that three relationships may exist; for example, “A and / or B” can represent: A alone, A and B simultaneously, and B alone. Normally, the character " / " indicates that the objects before and after it are in an "or" relationship. The terms "first," "second," "third," etc., used in this application are merely to distinguish similar objects and do not represent a specific order of objects.

[0050] The method embodiments provided in this example can be executed on a terminal, computer, or similar computing device. For example, it can run on a terminal. Figure 1 This is a hardware structure block diagram of the terminal for the 3D scanning method in this embodiment. For example... Figure 1 As shown, a terminal may include one or more ( Figure 1 Only one is shown in the diagram. A processor 102 and a memory 104 for storing data are also included. The processor 102 may be, but is not limited to, a microprocessor (MCU) or a programmable logic device (FPGA). The terminal may also include a transmission device 106 for communication functions and an input / output device 108. Those skilled in the art will understand that… Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the terminal described above. For example, the terminal may also include components that are larger than... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown are illustrated.

[0051] The memory 104 can be used to store computer programs, such as application software programs and modules, like the computer program corresponding to the 3D scanning method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, thereby implementing the aforementioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] The transmission device 106 is used to receive or send data via a network. This network includes a wireless network provided by the terminal's communication provider. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 can be a Radio Frequency (RF) module used for wireless communication with the Internet.

[0053] This embodiment provides a three-dimensional scanning method for a three-dimensional scanning system, which includes a scanning device, a tracking device, and a target. Figure 2 This is a flowchart of the three-dimensional scanning method in this embodiment, as follows: Figure 2As shown, the process includes the following steps:

[0054] Step S210: When the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device is obtained by the tracking device tracking the scanning device in real time.

[0055] First, a 3D scanning system can be built using a scanning device, a tracking device, and a target. The target is placed within the field of view of the tracking device, and the tracking device tracks the scanning device while the scanning device scans the object under test. The scanning device can be any optical scanning device suitable for 3D scanning, and its surface can be fixed with a structure of captureable marker points. The tracking device is a device capable of capturing images of the marker point structure on the surface of the scanning device, such as using a binocular vision sensor or a multi-view vision sensor as the main component, combined with other components such as a laser trigger, laser sensor, and inertial measurement unit (IMU). The target can be a reference object used for spatial positioning in the 3D scanning system. In this embodiment, the target can be a single piece or an object composed of multiple connecting parts. This embodiment does not specifically limit the shape of the target; it can be a planar object or a three-dimensional object. The target has target features, which can be reflective marking features or actively emitting marking features that are recognized by the tracking device. During the scanning process, the target can remain stationary or move as needed. Furthermore, this embodiment does not limit the number of scanning devices, tracking devices, and targets. For ease of description, the following explanation will primarily use one tracking device, one scanning device, and one target as an example.

[0056] When the scanning device scans the object under test, the tracking device tracks the scanning device by observing the marker point structure set on the surface of the scanning device, thus determining the first pose of the scanning device in the camera coordinate system of the tracking device. Specifically, when the tracking device observes the marker point structure of the scanning device, it can perform feature stitching with the observed data and a pre-generated scanning device model to determine the current position and pose of the scanning device in the camera coordinate system of the tracking device, thereby obtaining the aforementioned first pose. Taking point features as an example, when the following conditions are met:

[0057] ;

[0058] The solution can be obtained. and Among them, the above This represents the coordinates of the marked point features on the surface of the scanning device in the camera coordinate system of the tracking device; and This indicates the relative pose of the scanning device in the camera coordinate system of the tracking device (first pose). This indicates the coordinates of the marker point features of the scanning device in the coordinate system of the scanning device.

[0059] The scanning device model is a preset 3D layout of the scanning device's geometric features or marker points. It is a 3D CAD model or simplified geometric representation of the scanning device itself, used to describe the shape of the scanning device and the position of the sensors. The position and attitude of the scanning device in the camera coordinate system of the tracking device can be determined by stitching the collected marker point features of the scanning device with the scanning device model. If the tracking device cannot observe the scanning device, or if the number of observed feature points of the scanning device is insufficient to meet the stitching conditions, it is considered that the tracking device has failed to track the scanning device, and the aforementioned first pose cannot be obtained.

[0060] Step S220: Obtain the transformation relationship between the camera coordinate system and the coordinate system of the scanning device; the transformation relationship is based on the observation results and first pose determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning.

[0061] The transformation relationship can be obtained either previously or currently. It can be continuously solved and updated during the scanning process, or it can be directly used in subsequent 3D scanning calculations. Specifically, the transformation relationship can be determined by combining the observation results obtained from the scanning and tracking devices observing the target at a specific moment during the scanning process with the aforementioned first pose:

[0062] ;

[0063] ;

[0064] in, and This indicates the transformation relationship between the camera coordinate system and the scanning device coordinate system. , This indicates the pose of the target in the camera coordinate system of the tracking device (the observation result obtained by the tracking device observing the target); and This represents the target's pose in the camera coordinate system of the scanning device (the observation result obtained by the scanning device observing the target). Understandably, as long as the above transformation relationship can be solved once based on a frame of data acquired by the tracking and scanning devices, it can be directly read and used in subsequent 3D scanning calculations. Therefore, the transformation relationship can be solved as long as the scanning and tracking devices can simultaneously observe the target at a certain moment, without needing to maintain the continuity of the target observations by the scanning and tracking devices.

[0065] Step S230: Determine the relative pose of the cameras between the scanning device and the tracking device based on the transformation relationship and the first pose.

[0066] After obtaining the above transformation relationship and the first pose, we can then solve for the relative camera pose between the scanning device and the tracking device:

[0067] ;

[0068] ;

[0069] in, and This refers to the relative pose of the cameras between the scanning and tracking devices.

[0070] Step S240: Based on the relative pose of the camera, the scanning data collected by the scanning device is converted to the camera coordinate system of the tracking device.

[0071] It can also acquire the scanning data collected by the scanning device and cache the scanning data of the current frame. After obtaining the relative pose of the camera, the scanning data (point cloud data) collected by the scanning device in the camera coordinate system of the scanning device can be transformed into the camera coordinate system of the tracking device to complete the scanning of the current frame.

[0072] ;

[0073] in, This indicates the coordinates of the scan data in the camera coordinate system of the tracking device; This indicates the coordinates of the scanned data in the camera coordinate system of the scanning device.

[0074] Steps S210 to S240 can be repeated to convert the scanning data obtained by the scanning device from scanning the object under test at different scanning positions into the camera coordinate system of the tracking device, thereby realizing the three-dimensional reconstruction of the object under test.

[0075] In related technologies, a joint calibration of the tracking and scanning devices is often performed before scanning to determine the transformation relationship between the camera coordinate system and the scanning coordinate system. During this joint calibration, a calibration board needs to be placed in the field of view of the tracking device, ensuring that both devices can see it simultaneously. This can be constrained by usage scenarios (e.g., the tracking instrument is fixed at a high angle, making disassembly inconvenient), making calibration difficult. Furthermore, in scenarios with multiple tracking devices or when changing tracking devices, the transformation relationship between the camera and scanning coordinate systems is affected by the accuracy of the tracking device during the previous joint calibration. If a new tracking device is used for scanning, the parameters of the scanning device need to be recalibrated, which is detrimental to accuracy assurance in scenarios with multiple tracking devices or when changing tracking devices. Additionally, the overall scanning efficiency of tracking-based 3D scanning is relatively low.

[0076] In this embodiment, before the scanning device performs scanning, it is no longer necessary to jointly calibrate the tracking device and the scanning device; only the scanning device and the tracking device need to be calibrated separately. This improves the overall scanning efficiency of the tracking-based 3D scanning. Furthermore, in this embodiment, the accuracy of the transformation relationship between the camera coordinate system and the scanning device coordinate system is no longer limited by the accuracy of a single tracking device. Therefore, the tracking device or the scanning device can be arbitrarily replaced during the scanning process to build the 3D scanning system.

[0077] Therefore, through the above steps S210 to S240, when the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device is obtained by the tracking device tracking the scanning device in real time; the transformation relationship between the camera coordinate system of the scanning device and the coordinate system of the scanning device is obtained; the transformation relationship is determined based on the observation results and first pose obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning; the relative pose of the cameras between the scanning device and the tracking device is determined according to the transformation relationship and the first pose; and the scanning data collected by the scanning device is transformed into the camera coordinate system of the tracking device according to the relative pose of the cameras. This allows the relative pose of the cameras between the scanning device and the tracking device to be solved using the target during the scanning process, eliminating the need for joint calibration before scanning, thereby improving the overall scanning efficiency of tracking-type 3D scanning.

[0078] In one embodiment, the conversion relationship is determined based on the observation results and first pose obtained by the scanning device and the tracking device observing the target at the same moment when the scanning device performs scanning, which may include:

[0079] The first observation result is obtained by acquiring the target observed by the scanning device at the first moment and determining the position and orientation of the target in the camera coordinate system of the scanning device; the second observation result is obtained by acquiring the target observed by the tracking device at the first moment and determining the position and orientation of the target in the camera coordinate system of the tracking device; the first pose is obtained by acquiring the scanning device observed by the tracking device at the first moment and determining the position and orientation of the scanning device in the camera coordinate system of the tracking device; and the transformation relationship is determined based on the first observation result, the second observation result, and the first pose.

[0080] A target model can be pre-constructed. This model can be a preset 3D layout of the target's geometric features or marker points, representing a 3D CAD model or simplified geometric representation of the target features. At the first instant, both the scanning and tracking devices can observe the target, and the tracking device can observe the scanning device. When the scanning device observes the target features, it can stitch these observed features with the pre-constructed target model to obtain the target's position and orientation in the camera coordinate system of the scanning device, thus obtaining the first observation result. Specifically (taking point features as an example):

[0081] ;

[0082] in, and P represents the first observation result. R This represents the coordinates of the feature points of the target model. This indicates the coordinates of the feature points on the target in the camera coordinate system of the scanning device.

[0083] Accordingly, when the tracking device observes target features on the target, it can stitch the observed target features with a pre-built target model to obtain the target's position and orientation in the camera coordinate system of the tracking device, thus obtaining the second observation result. Specifically (taking point features as an example):

[0084] ;

[0085] in, and This indicates the second observation result. This indicates the coordinates of the feature points on the target in the camera coordinate system of the tracking device.

[0086] After obtaining the first and second observation results, and combining the position and attitude of the scanning device in the camera coordinate system of the tracking device at the first moment (i.e., the first pose), the transformation relationship between the camera coordinate system of the scanning device and the coordinate system of the scanning device is determined.

[0087] It should be noted that when the scanning device observes the target, it must be able to successfully observe the features on the target, and the number of observed features must be sufficient to successfully calculate the first observation result mentioned above. Similarly, when the tracking device observes the target, it must also be able to successfully observe the features on the target, and the number of observed features must be sufficient to successfully calculate the second observation result mentioned above.

[0088] In this embodiment, during the scanning process, a target can be used to help determine the transformation relationship between the coordinate system of the scanning device and the camera coordinate system of the scanning device. This transformation relationship does not depend on the accuracy of the tracking device, and the scanning device or tracking device can be replaced midway without recalibration, thus improving the overall scanning efficiency of tracking scanning.

[0089] In another embodiment, the above-described three-dimensional scanning method may further include:

[0090] If a prior transformation relation already exists, update the transformation relation based on the latest observation results and the latest first pose.

[0091] After the above transformation relationship has been solved, at a new moment, the observation results of the scanning device and the tracking device on the target can be combined with the first pose of the scanning device in the camera coordinate system of the tracking device at that moment to solve the transformation relationship again, and the existing transformation relationship can be updated using the newly solved transformation relationship.

[0092] For example, the newly solved transformation relation can directly replace the existing transformation relation, or a weighted average can be calculated from multiple separately solved transformation relations, or a threshold can be set according to an empirical value to eliminate calculated noise. The specific update method is not limited here.

[0093] In this embodiment, by using the latest observation results and the latest first pose to update the transformation relationship, the accuracy of the transformation relationship calculation results can be improved, thereby improving the accuracy of 3D reconstruction.

[0094] In one embodiment, the above-described three-dimensional scanning method may further include:

[0095] If the conversion relationship cannot be obtained and the acquisition of observation results fails, the conversion processing of the scanning data currently acquired by the scanning device is terminated. In some embodiments, if the scanning device and the tracking device cannot observe the target features set on the target at the same time, or if the observed target features do not meet the preset feature stitching conditions, the acquisition of observation results is determined to have failed.

[0096] If the transformation relationship has not been solved beforehand, it cannot be obtained. For example, during the scanning process, if the scanning and tracking devices have not yet synchronously observed the target at a certain moment, the transformation relationship cannot be solved. If the scanning device cannot observe the target, or the observed target features cannot be successfully stitched with the target model, the scanning device is deemed to have failed to observe the target. If the tracking device cannot observe the target, or the observed target features cannot be successfully stitched with the target model, the tracking device is deemed to have failed to observe the target. If the tracking device and / or the scanning device fails to observe the target, the acquisition of the observation result is deemed to have failed. If there is no previously solved transformation relationship, and the current scanning device and / or the tracking device fails to observe the target, it is confirmed that the scanning data currently acquired by the scanning device cannot be successfully transformed into the camera coordinate system of the tracking device; therefore, the scanning data of the current frame can be discarded.

[0097] In one embodiment, the above-described three-dimensional scanning method may further include:

[0098] When the first pose at the current moment cannot be obtained, the first observation result of the scanning device on the target and the second observation result obtained by the tracking device synchronously observing the target are obtained; based on the first observation result and the second observation result, the relative pose of the camera between the scanning device and the tracking device is determined.

[0099] If, at the current moment, the tracking device cannot observe the scanning device, or if the marker features set on the observed scanning device cannot be stitched together with the scanning device model, then acquiring the first pose at the current moment is determined to have failed. In this case, the relative camera pose between the scanning and tracking devices can be determined based on their observations of the target:

[0100] ;

[0101] ;

[0102] in, and This indicates the relative pose of the cameras between the scanning device and the tracking device.

[0103] Therefore, even if the tracking device fails to observe the scanning device, this embodiment can still determine the relative pose of the camera between the tracking device and the scanning device based on the aid of the target, thereby improving the stability in the tracking-type 3D scanning scenario.

[0104] In another embodiment, after converting the scanning data acquired by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera, the above-mentioned three-dimensional scanning method may further include:

[0105] The observation results obtained by the scanning and tracking devices simultaneously observing the target in the first pose are jointly optimized.

[0106] Specifically, the first pose and observation results can be jointly optimized based on the two-dimensional coordinates of the features on the target observed simultaneously by the scanning and tracking devices, the intrinsic and extrinsic parameters of the tracking device, and the intrinsic and extrinsic parameters of the scanning device.

[0107] In this embodiment, by jointly optimizing the first pose and the observation results, the accuracy of the 3D reconstruction results of the tracking 3D scan can be improved.

[0108] Furthermore, in one embodiment, the above-described three-dimensional scanning method may further include:

[0109] The 3D reconstruction of the object under test is completed based on the scanning data in the camera coordinate system of the tracking device. Specifically, scanning data of the object under test collected by the scanning device at different scanning positions can be converted to the camera coordinate system of the tracking device, thus obtaining multi-frame scanning data in a unified coordinate system. This allows for the 3D reconstruction of the object under test, and finally, the 3D reconstruction result can be output, for example, by displaying the reconstructed 3D model of the object on a screen.

[0110] Figure 3 These are flowcharts of some embodiments of the 3D scanning method, such as... Figure 3 As shown, the three-dimensional scanning method includes the following steps:

[0111] Step S301: The scanning device performs a scan.

[0112] Step S302: Determine whether the tracking device has observed the scanning device; if yes, proceed to steps S304 and S308 respectively; otherwise, proceed to step S303.

[0113] Step S303: Determine whether the tracking device has observed the target; if yes, proceed to step S305; otherwise, proceed to step S311.

[0114] Step S304: Obtain the position and orientation of the scanning device in the camera coordinate system of the tracking device.

[0115] Step S305: Obtain the position and orientation of the target in the camera coordinate system of the tracking device.

[0116] Step S306: Determine whether the scanning device can observe the target; if yes, proceed to steps S307 and S308; otherwise, proceed to step S311.

[0117] Step S307: Obtain the position and orientation of the target in the camera coordinate system of the scanning device; execute step S310.

[0118] Step S308: Solve and update the transformation relationship between the camera coordinate system and the scanning device coordinate system.

[0119] Step S309: Determine whether the transformation relationship between the camera coordinate system and the scanning device coordinate system has been obtained; if yes, proceed to step S310; otherwise, proceed to step S311.

[0120] Step S310: Solve the relative pose of the cameras of the scanning device and the tracking device. Execute step S312.

[0121] Step S311: Discard the scan data of the current frame.

[0122] Step S312: Obtain the scanning data of the current frame of the scanning device; Steps S313 and S315 are executed respectively.

[0123] Step S313: Convert the scan data of the current frame to the camera coordinate system of the tracking device.

[0124] Step S314: Complete the scanning of the current frame and return to step S301.

[0125] Step S315: Cache the scan data of the current frame.

[0126] Step S316, joint optimization; wherein, the position and attitude of the scanning device under the tracking device, and the observation results of the tracking device and the scanning device on the target at the same time are jointly optimized.

[0127] Step S317: Output the scan results. Steps S301 to S310 and S312 to S314 can be real-time processes, while steps S311 and S315 to S317 can be post-processing processes.

[0128] Steps S301 to S317 form a tracking-type 3D scanning system consisting of a tracking device, a scanning device, and a target. Before use, only the tracking device and the scanning device need to be calibrated individually, eliminating the need for joint calibration of the two devices and improving the overall efficiency and convenience of the tracking-type 3D scanning. Furthermore, since joint calibration of the tracking device and the scanning device is unnecessary, their accuracy remains independent, allowing for the replacement of either the tracking device or the scanning device.

[0129] Figure 4 This is a schematic diagram of the coordinate system of a scanning device according to this embodiment. Figure 4As shown, the center of the scanning device's own structure, or a predetermined reference point, can be used as the origin of the coordinate system to establish the scanning device's coordinate system. The scanning device's coordinate system is a three-dimensional coordinate system and can include the X-axis, Y-axis, and Z-axis. For example... Figure 4 The scanning equipment coordinate system is constructed with the center of the mechanical structure of the scanning equipment as the origin. The scanning equipment coordinate system describes the local reference coordinate system of the scanning equipment itself.

[0130] Figure 5 This is a schematic diagram of the camera coordinate system of a scanning device according to this embodiment. Figure 5 As shown, the camera coordinate system of the scanning device is a three-dimensional coordinate system of one of the camera sensors of the scanning device, which may include the X-axis, Y-axis, and Z-axis. Specifically, the optical center (optical center of the lens) of one of the camera sensors of the scanning device can be used as the origin to establish the camera coordinate system of the scanning device. In this embodiment, the camera coordinate system of the scanning device can be the coordinate system to which the coordinates of the scanned data belong when the scanning device acquires scanned data. It can be understood that the camera coordinate system of the scanning device is different at different positions, therefore, the coordinate system to which different frames of scanned data acquired by the scanning device at different positions belong is also different.

[0131] Figure 6 This is a schematic diagram of the camera coordinate system of a tracking device according to this embodiment. Figure 6 As shown, the camera coordinate system of the tracking device is a three-dimensional coordinate system of one camera sensor of the tracking device, which may include the X-axis, Y-axis, and Z-axis. Specifically, the optical center of one camera sensor of the tracking device can be used as the origin to establish the camera coordinate system of the tracking device. In this embodiment, the camera coordinate system of the tracking device can be a unified coordinate system obtained by unifying the coordinate systems of different frame scan data.

[0132] This embodiment also provides a three-dimensional scanning device for implementing the above embodiments and preferred embodiments; details already described will not be repeated. The terms "module," "unit," "subunit," etc., used below refer to combinations of software and / or hardware that perform a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0133] Figure 7 This is a structural block diagram of the three-dimensional scanning device 70 of this embodiment, used in a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device, a tracking device, and a target; as shown... Figure 7 As shown, the 3D scanning device 70 includes: a first acquisition module 72, a second acquisition module 74, a solution module 76, and a conversion module 78; wherein:

[0134] The first acquisition module 72 is used to acquire the first pose of the scanning device in the camera coordinate system of the tracking device, obtained by the tracking device tracking the scanning device in real time when the scanning device performs scanning; the second acquisition module 74 is used to acquire the transformation relationship between the camera coordinate system of the scanning device and the coordinate system of the scanning device; the transformation relationship is determined by the observation results and the first pose obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning; the solution module 76 is used to determine the relative pose of the camera between the scanning device and the tracking device according to the transformation relationship and the first pose; the conversion module 78 is used to convert the scanning data collected by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera.

[0135] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can reside in the same processor; or the above modules can be located in different processors in any combination.

[0136] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated in this embodiment.

[0137] In addition, a three-dimensional scanning system is also provided in this embodiment. Figure 8 This is a schematic diagram of the structure of the three-dimensional scanning system 80 in this embodiment, as shown below. Figure 8 As shown, the three-dimensional scanning system includes a scanning device 82, a tracking device 84, a target 86, and a processor 88; wherein, the target 86 is disposed within the tracking range of the tracking device 84; the scanning device 82 and the tracking device 84 are respectively connected to the processor 88;

[0138] The scanning device 82 is used to perform scanning on the object to be tested; the tracking device 84 is used to observe the scanning device 82 and the target 86 respectively when the scanning device 82 performs scanning; the processor 88 is used to execute the three-dimensional scanning method provided in any of the above embodiments.

[0139] In conjunction with the three-dimensional scanning methods provided in the above embodiments, this embodiment can also provide a storage medium for implementation. The storage medium stores a computer program; when executed by a processor, the computer program implements any of the three-dimensional scanning methods described in the above embodiments.

[0140] It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. All other embodiments derived by those skilled in the art based on the embodiments provided in this application without inventive effort are within the scope of protection of this application.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0142] Obviously, the accompanying drawings are merely some examples or embodiments of this application. Those skilled in the art can apply this application to other similar situations based on these drawings without any creative effort. Furthermore, it is understood that although the work done in this development process may be complex and lengthy, for those skilled in the art, certain design, manufacturing, or production modifications made based on the technical content disclosed in this application are merely conventional technical means and should not be considered as insufficient disclosure of this application.

[0143] The term "embodiment" in this application refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily imply the same embodiment, nor does it imply that it is mutually exclusive with or independent of other embodiments. It will be clearly or implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0144] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the appended claims.

Claims

1. A three-dimensional scanning method, characterized in that, For a three-dimensional scanning system, the three-dimensional scanning system including a scanning device, a tracking device, and a target, the method includes: When the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device is obtained by the tracking device tracking the scanning device in real time. Obtain the transformation relationship between the camera coordinate system and the coordinate system of the scanning device; the transformation relationship is based on the observation results obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, and the determination of the first pose; including: The following steps are performed: First, the target is observed by the scanning device at a first moment, and the position and orientation of the target in the camera coordinate system of the scanning device are determined to obtain a first observation result; second, the target is observed by the tracking device at a first moment, and the position and orientation of the scanning device in the camera coordinate system of the tracking device are determined to obtain a second observation result; third, the scanning device is observed by the tracking device at a first moment, and the position and orientation of the scanning device in the camera coordinate system of the tracking device are determined to obtain a first pose; and finally, the transformation relationship is determined based on the first observation result, the second observation result, and the first pose. Based on the transformation relationship and the first pose, the relative camera pose between the scanning device and the tracking device is determined; Based on the relative pose of the camera, the scanning data collected by the scanning device is converted to the camera coordinate system of the tracking device.

2. The three-dimensional scanning method according to claim 1, characterized in that, The method further includes: If a prior transformation relationship already exists, the transformation relationship is updated based on the latest observation and the latest first pose.

3. The three-dimensional scanning method according to claim 1, characterized in that, The method further includes: If the transformation relationship cannot be obtained and the observation result cannot be obtained, the transformation processing of the scanning data currently acquired by the scanning device shall be terminated.

4. The three-dimensional scanning method according to claim 3, characterized in that: If the scanning device and the tracking device cannot observe the target features set on the target at the same time, or if the observed target features do not meet the preset feature splicing conditions, it is determined that the acquisition of the observation result has failed.

5. The three-dimensional scanning method according to claim 1, characterized in that, The method further includes: When the first pose at the current moment cannot be obtained, the first observation result of the scanning device on the target and the second observation result obtained by the tracking device synchronously observing the target are obtained; the relative pose of the camera between the scanning device and the tracking device is determined based on the first observation result and the second observation result.

6. The three-dimensional scanning method according to claim 1, characterized in that, After converting the scanning data acquired by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera, the method further includes: The observation results obtained from the first pose and the observations of the target by the scanning device and the tracking device at the same time are jointly optimized.

7. The three-dimensional scanning method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the scanning data in the camera coordinate system of the tracking device, the three-dimensional reconstruction of the object under test is completed.

8. A three-dimensional scanning device, characterized in that, A 3D scanning system is used, the 3D scanning system including a scanning device, a tracking device, and a target, the 3D scanning device including: a first acquisition module, a second acquisition module, a solving module, and a conversion module; wherein: The first acquisition module is used to acquire, when the scanning device performs scanning, the first pose of the scanning device in the camera coordinate system of the tracking device, obtained by the tracking device tracking the scanning device in real time. The second acquisition module is used to acquire the transformation relationship between the camera coordinate system and the coordinate system of the scanning device; the transformation relationship is based on the observation results obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, and the first pose determination; including: The following steps are performed: First, the target is observed by the scanning device at a first moment, and the position and orientation of the target in the camera coordinate system of the scanning device are determined to obtain a first observation result; second, the target is observed by the tracking device at a first moment, and the position and orientation of the scanning device in the camera coordinate system of the tracking device are determined to obtain a second observation result; third, the scanning device is observed by the tracking device at a first moment, and the position and orientation of the scanning device in the camera coordinate system of the tracking device are determined to obtain a first pose; and finally, the transformation relationship is determined based on the first observation result, the second observation result, and the first pose. The solving module is used to determine the camera relative pose between the scanning device and the tracking device based on the transformation relationship and the first pose; The conversion module is used to convert the scanning data collected by the scanning device to the camera coordinate system of the tracking device according to the relative pose of the camera.

9. A three-dimensional scanning system, characterized in that, include: The system includes a scanning device, a tracking device, a target, and a processor; wherein the target is positioned within the tracking range of the tracking device. The scanning device and the tracking device are respectively connected to the processor; The scanning device is used to perform scanning on the object to be tested; The tracking device is used to observe the scanning device and the target respectively when the scanning device performs a scan; The processor is used to execute the three-dimensional scanning method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the three-dimensional scanning method according to any one of claims 1 to 7.

Images