Three-dimensional scanning method, three-dimensional scanning device, three-dimensional scanning system and storage medium

Abstract

The invention relates to a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system and a storage medium, and the three-dimensional scanning method comprises the steps: obtaining a first pose of scanning equipment in a camera coordinate system of the tracking equipment, which is obtained through the real-time tracking of the scanning equipment by the tracking equipment when the scanning equipment executes scanning; acquiring a conversion relation between a camera coordinate system of the scanning equipment and a coordinate system of the scanning equipment; the conversion relation is determined based on an observation result obtained by observing the target by the scanning equipment and the tracking equipment at the same moment and the first pose when the scanning equipment executes scanning; determining a camera relative pose between the scanning device and the tracking device according to the conversion relation and the first pose; and according to the relative pose of the camera, converting scanning data collected by the scanning device to a camera coordinate system of the tracking device. According to the method, joint calibration before scanning is omitted, so that the overall scanning efficiency of tracking type three-dimensional scanning is improved.

Description

Technical Field

[0001] The present application relates to the field of three-dimensional scanning, and in particular to a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system and a storage medium. Background Art

[0002] Tracking 3D scanning systems combine optical, mechanical, and electronic technologies to accurately measure and digitally reconstruct an object's three-dimensional shape. These systems consist of a scanning device and a tracking device. The scanning device moves to different locations to scan the object, while the tracking device continuously tracks the scanning device to determine its spatial position after movement.

[0003] Currently, in tracking scanning systems, it is often necessary to jointly calibrate the tracking device and the scanning device before the scanning device performs a scan. During this joint calibration process, a calibration plate must be placed in the tracking device's field of view, ensuring that both the tracking device and the scanning device can observe the plate simultaneously. This is used to determine the transformation relationship between the scanning device's camera coordinate system and the scanning device's coordinate system. The results of this joint calibration depend on the accuracy of the tracking device. When the tracking device is replaced to track the scanning device, the joint calibration must be repeated based on the replaced tracking device, resulting in lower overall scanning efficiency for tracking 3D scanning.

[0004] Currently, no effective solution has been proposed to the problem that the overall scanning efficiency of tracking 3D scanning is low due to the need for joint calibration before scanning in related technologies. Summary of the Invention

[0005] In this embodiment, a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system and a storage medium are provided to solve the problem in related technologies that joint calibration is required before scanning, resulting in low overall scanning efficiency of tracking three-dimensional scanning.

[0006] In a first aspect, a three-dimensional scanning method is provided in this embodiment for use in a three-dimensional scanning system, wherein the three-dimensional scanning system includes a scanning device, a tracking device, and a target. The method includes:

[0007] When the scanning device performs scanning, obtaining the first pose of the scanning device in the camera coordinate system of the tracking device obtained by the tracking device tracking the scanning device in real time;

[0008] Obtaining a conversion relationship between a camera coordinate system of the scanning device and a coordinate system of the scanning device; the conversion relationship is based on observation results and the first posture determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning;

[0009] Determining a relative camera pose between the scanning device and the tracking device according to the conversion relationship and the first pose;

[0010] The scanning data collected by the scanning device is converted into the camera coordinate system of the tracking device according to the relative position of the camera.

[0011] In some embodiments, the conversion relationship is based on observation results and the first posture determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, including:

[0012] Obtaining the scanning device observing the target at a first moment, determining the position and posture of the target in the camera coordinate system of the scanning device, and obtaining a first observation result;

[0013] Obtaining the target observed by the tracking device at a first moment, and determining the position and posture of the target in the camera coordinate system of the tracking device to obtain a second observation result;

[0014] Obtaining a position and posture of the scanning device in a camera coordinate system of the tracking device determined by the tracking device when the tracking device observes the scanning device at a first moment, to obtain the first posture;

[0015] The conversion relationship is determined based on the first observation result, the second observation result and the first posture.

[0016] In some embodiments, the method further comprises:

[0017] When a previous conversion relationship already exists, the conversion relationship is updated according to the latest observation result and the latest first posture.

[0018] In some embodiments, the method further comprises:

[0019] In the case that the conversion relationship cannot be obtained and the acquisition of the observation result fails, the conversion process of the scanning data currently acquired by the scanning device is terminated.

[0020] In some embodiments, when the scanning device and the tracking device cannot observe the target features set on the target at the same time, or the observed target features do not meet the preset feature splicing conditions, it is determined that the acquisition of the observation result has failed.

[0021] In some embodiments, the method further comprises:

[0022] When the first posture at the current moment cannot be obtained, a first observation result of the scanning device on the target is obtained, and a second observation result of the tracking device synchronously observing the target is obtained; based on the first observation result and the second observation result, the relative posture of the camera between the scanning device and the tracking device is determined.

[0023] In some embodiments, after converting the scan data collected by the scanning device into a camera coordinate system of the tracking device according to the relative position of the camera, the method further includes:

[0024] The first posture and observation results obtained by the scanning device and the tracking device observing the target at the same time are jointly optimized.

[0025] In some embodiments, the method further comprises:

[0026] The three-dimensional reconstruction of the object to be measured is completed according to the scanning data in the camera coordinate system of the tracking device.

[0027] In a second aspect, this embodiment provides a three-dimensional scanning device for use in a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device, a tracking device, and a target. The three-dimensional scanning device includes: a first acquisition module, a second acquisition module, a solution module, and a conversion module; wherein:

[0028] The first acquisition module is configured to acquire, when the scanning device performs scanning, a first pose of the scanning device in the camera coordinate system of the tracking device, obtained by the tracking device tracking the scanning device in real time;

[0029] The second acquisition module is configured to acquire a conversion relationship between a camera coordinate system of the scanning device and a coordinate system of the scanning device; the conversion relationship is based on observation results of the target observed by the scanning device and the tracking device at the same time when the scanning device performs scanning and the first posture determination;

[0030] The solving module is configured to determine a relative camera pose between the scanning device and the tracking device based on the conversion relationship and the first pose;

[0031] The conversion module is used to convert the scanning data collected by the scanning device into the camera coordinate system of the tracking device according to the relative position of the camera.

[0032] In a third aspect, this embodiment provides a three-dimensional scanning system, comprising: a scanning device, a tracking device, a target, and a processor; wherein the target is set within a tracking range of the tracking device; the scanning device and the tracking device are respectively connected to the processor;

[0033] The scanning device is used to scan the object to be measured;

[0034] The tracking device is used to observe the scanning device and the target respectively when the scanning device performs scanning;

[0035] The processor is used to execute the three-dimensional scanning method described in the first aspect.

[0036] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored. When the program is executed by a processor, the three-dimensional scanning method described in the first aspect is implemented.

[0037] Compared with related technologies, this embodiment provides a three-dimensional scanning method, a three-dimensional scanning device, a three-dimensional scanning system, and a storage medium. The three-dimensional scanning method, when the scanning device performs a scan, obtains the first position of the scanning device in the tracking device camera coordinate system obtained by real-time tracking of the scanning device by the tracking device; obtains the conversion relationship between the scanning device camera coordinate system and the scanning device coordinate system; the conversion relationship is determined based on the observation results and the first position obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs a scan; the relative position of the camera between the scanning device and the tracking device is determined based on the conversion relationship and the first position; the scanning data collected by the scanning device is converted to the camera coordinate system of the tracking device based on the relative position of the camera. It can solve the relative position of the camera between the scanning device and the tracking device with the help of the target during the scanning process of the scanning device, eliminating the need for joint calibration before scanning, thereby improving the overall scanning efficiency of tracking-type three-dimensional scanning.

[0038] The details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0040] Figure 1 is a flow chart of a three-dimensional scanning method according to an embodiment of the present application;

[0041] Figure 2is a flow chart of a three-dimensional scanning method according to an embodiment of the present application;

[0042] Figure 3 is a flow chart of a three-dimensional scanning method according to some embodiments of the present application;

[0043] Figure 4 This is a schematic diagram of a scanning device coordinate system according to an embodiment of the present application;

[0044] Figure 5 This is a schematic diagram of a camera coordinate system for a scanning device according to an embodiment of the present application;

[0045] Figure 6 This is a schematic diagram of a camera coordinate system for a tracking device according to an embodiment of the present application;

[0046] Figure 7 is a structural block diagram of a three-dimensional scanning device according to an embodiment of the present application;

[0047] Figure 8 It is a structural diagram of a three-dimensional scanning system according to an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0049] Unless otherwise defined, technical or scientific terms used in this application shall have the ordinary meanings as understood by persons of ordinary skill in the art to which this application belongs. The terms "a," "an," "the," "these," and similar expressions in this application do not denote limitations on quantity and may be singular or plural. The terms "comprise," "include," "have," and any variations thereof, as used in this application, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units) but may include unlisted steps or modules (units) or other steps or modules (units) inherent to the process, method, product, or device. The terms "connected," "connected," "coupled," and similar expressions used in this application are not limited to physical or mechanical connections but may include electrical connections, whether direct or indirect. As used in this application, "plurality" means two or more. "And / or" describes an association between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone; A and B exist simultaneously; or B exists alone. Generally, the character " / " indicates that the objects in the preceding and following relationship are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0050] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG is a block diagram of the hardware structure of the terminal of the three-dimensional scanning method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 The processor 102 (only one is shown) and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input / output device 108. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0051] The memory 104 can be used to store computer programs, such as software programs and modules of application software, such as the computer program corresponding to the three-dimensional scanning method in this embodiment. The processor 102 executes the computer program stored in the memory 104 to perform various functional applications and data processing, thereby implementing the above-mentioned method. The memory 104 may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0052] Transmission device 106 is used to receive or transmit data via a network. This network may include a wireless network provided by the terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0053] In this embodiment, a three-dimensional scanning method is provided for a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device, a tracking device, and a target. Figure 2 is a flow chart of the three-dimensional scanning method of this embodiment. Figure 2As shown, the process includes the following steps:

[0054] Step S210 , when the scanning device performs scanning, obtaining the first position of the scanning device in the tracking device camera coordinate system obtained by real-time tracking of the scanning device by the tracking device.

[0055] First, a 3D scanning system can be constructed using a scanning device, a tracking device, and a target. The target is placed within the tracking device's field of view. As the scanning device scans the object to be measured, the tracking device tracks the scanning device. The scanning device can be any optical scanning device suitable for 3D scanning, and its surface can have a capturable marker structure fixed to it. The tracking device is a device capable of capturing images of the marker structure on the scanning device's surface. For example, it can use a binocular or multi-lens vision sensor as its primary component, along with other components such as a laser trigger, a laser sensor, and an inertial measurement unit (IMU). The target can be a reference object used for spatial positioning in the 3D scanning system. In this embodiment, the target can be a single piece or comprised of multiple connected components. This embodiment does not specifically limit the shape of the target; it can be either a flat or three-dimensional object. The target is provided with a target feature, which can be a reflective marker or an active luminous marker recognized by the tracking device. During the scanning process, the target can remain stationary or move as needed. In addition, this embodiment does not limit the number of scanning devices, tracking devices, and targets. For the sake of convenience, the following description will mainly take one tracking device, one scanning device, and one target as an example.

[0056] When the scanning device scans the object to be measured, the tracking device tracks the scanning device by observing the structure of the marker points set on the surface of the scanning device, and determines the first position of the scanning device in the tracking device camera coordinate system. When the tracking device observes the marker point structure of the scanning device, it can perform feature splicing on the observed data with the scanning device model pre-generated by the scanning device to determine the current position and posture of the scanning device in the tracking device camera coordinate system, thereby obtaining the above-mentioned first position. Taking the point feature as an example, when the following is satisfied:

[0057] ;

[0058] You can solve it and Among them, the above Represents the coordinates of the marker features on the surface of the scanning device in the camera coordinate system of the tracking device; and Represents the relative pose of the scanning device in the camera coordinate system of the tracking device (first pose); Represents the coordinates of the marker feature of the scanning device in the coordinate system of the scanning device.

[0059] The scanning device model is a preset 3D layout of the scanning device's geometric features or markers, and is a 3D CAD model or simplified geometric representation of the scanning device itself, used to describe the scanning device's shape and sensor position. The position and posture of the scanning device in the tracking device's camera coordinate system can be determined by feature splicing the captured scanning device's marker features with the scanning device model. If the tracking device cannot observe the scanning device, or the number of observed feature points of the scanning device is insufficient to meet the splicing conditions, the tracking device is deemed to have failed to track the scanning device, and the aforementioned first pose cannot be obtained.

[0060] Step S220, obtaining the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system; the conversion relationship is based on the observation results and the first pose determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs the scan.

[0061] This transformation relationship can be obtained by solving it previously or solving it currently. The transformation relationship can be solved and updated continuously during the scanning process. Alternatively, the transformation relationship obtained by the initial calculation can be directly used in subsequent three-dimensional scanning calculations. Specifically, the transformation relationship can be determined by combining the observation results obtained by the scanning device and the tracking device observing the target at the same time during the scanning process of the scanning device with the above-mentioned first pose:

[0062] ;

[0063] ;

[0064] in, and Indicates the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system. 、 Represents the target's position in the tracking device's camera coordinate system (the observation result obtained by the tracking device observing the target); and Represents the target's pose in the scanning device's camera coordinate system (the result of the scanning device observing the target). It can be understood that once this transformation relationship is solved based on a frame of data acquired by the tracking and scanning devices, it can be directly read and used in subsequent 3D scanning calculations. Therefore, the transformation relationship can be solved simply by the scanning and tracking devices observing the target simultaneously at a certain moment, without the need for the scanning and tracking devices to maintain continuous observation of the target.

[0065] Step S230: Determine the relative camera pose between the scanning device and the tracking device according to the conversion relationship and the first pose.

[0066] After obtaining the above transformation relationship and the first pose, we can continue to solve the relative pose of the camera between the scanning device and the tracking device:

[0067] ;

[0068] ;

[0069] in, and is the relative camera pose between the scanning device and the tracking device.

[0070] Step S240 : converting the scan data collected by the scanning device into the camera coordinate system of the tracking device according to the relative position of the camera.

[0071] You can also obtain the scan data collected by the scanning device and cache the current frame scan data. After obtaining the above-mentioned relative camera pose, you can convert the scan data (point cloud data) in the camera coordinate system of the scanning device collected by the scanning device when scanning the object to be measured into the camera coordinate system of the tracking device to complete the scan of the current frame:

[0072] ;

[0073] in, Represents the coordinates of the scan data in the camera coordinate system of the tracking device; Represents the coordinates of the scanned data in the camera coordinate system of the scanning device.

[0074] Steps S210 to S240 may be repeatedly performed to uniformly convert scanning data obtained by scanning the object under test by the scanning device at different scanning positions into the camera coordinate system of the tracking device, thereby achieving three-dimensional reconstruction of the object under test.

[0075] In the related art, the tracking device and the scanning device are often jointly calibrated in advance before the scanning device performs scanning to determine the conversion relationship between the camera coordinate system of the scanning device and the coordinate system of the scanning device. When performing the joint calibration, it is necessary to place a calibration plate in the field of view of the tracking device, and ensure that the tracking device and the scanning device can see the calibration plate at the same time. At this time, it may be constrained by the usage scenario (for example, the tracking instrument is fixed at a high viewing angle and is not convenient to disassemble), making the calibration operation inconvenient. Moreover, for scenarios with multiple tracking devices or scenarios where the tracking device is replaced, since the conversion relationship between the camera coordinate system of the scanning device and the coordinate system of the scanning device is affected by the accuracy of the tracking device during the previous joint calibration, if the tracking device is replaced to perform the scanning operation at this time, the parameters of the scanning device need to be recalibrated. Therefore, it is not conducive to accuracy assurance in scenarios with multiple tracking devices or replacement of tracking devices. In addition, the overall scanning efficiency of tracking-type three-dimensional scanning is low.

[0076] In this embodiment, prior to scanning, the tracking device and the scanning device need no longer be jointly calibrated; only the scanning device and the tracking device need to be calibrated separately, thereby improving the overall efficiency of tracking-based 3D scanning. Furthermore, the accuracy of the transformation between the camera coordinate system and the scanning device coordinate system is no longer limited by the accuracy of a single tracking device. Therefore, the tracking device or the scanning device can be replaced at will during the scanning process to establish a 3D scanning system.

[0077] Therefore, through the above steps S210 to S240, when the scanning device performs a scan, the first position of the scanning device in the tracking device camera coordinate system is obtained by the tracking device tracking the scanning device in real time; the conversion relationship between the scanning device camera coordinate system and the scanning device coordinate system is obtained; the conversion relationship is determined based on the observation results and the first position obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs a scan; based on the conversion relationship and the first position, the relative position of the camera between the scanning device and the tracking device is determined; based on the relative position of the camera, the scanning data collected by the scanning device is converted to the camera coordinate system of the tracking device. This can solve the relative position of the camera between the scanning device and the tracking device with the help of the target during the scanning process of the scanning device, eliminating the need for joint calibration before scanning, thereby improving the overall scanning efficiency of tracking-type three-dimensional scanning.

[0078] In one embodiment, the conversion relationship is based on the observation results and the first pose determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, and can include:

[0079] Obtain the scanning device observing the target at the first moment, and determine the position and posture of the target in the camera coordinate system of the scanning device to obtain a first observation result; obtain the tracking device observing the target at the first moment, and determine the position and posture of the target in the camera coordinate system of the tracking device to obtain a second observation result; obtain the tracking device observing the scanning device at the first moment, and determine the position and posture of the scanning device in the camera coordinate system of the tracking device to obtain a first posture; determine the conversion relationship according to the first observation result, the second observation result and the first posture.

[0080] A target model can be pre-constructed. The target model can be a preset three-dimensional layout of the target's geometric features or marking points, and is a three-dimensional CAD model or simplified geometric representation of the target features set on the target. At the first moment, both the scanning device and the tracking device can observe the target, and the tracking device can observe the scanning device. When the scanning device observes the target features on the target, the observed target features can be feature-joined with the pre-constructed target model to obtain the position and posture of the target in the camera coordinate system of the scanning device, thereby obtaining the first observation result. Specifically (taking the target features as point features as an example):

[0081] ;

[0082] in, and represents the first observation result, P R Represents the feature point coordinates of the target model, Indicates the coordinates of the feature points on the target in the camera coordinate system of the scanning device.

[0083] Accordingly, when the tracking device observes the target features on the target, it can perform feature splicing on the observed target features and the pre-built target model to obtain the position and posture of the target in the camera coordinate system of the tracking device, and obtain the second observation result. Specifically (taking the target features as point features as an example):

[0084] ;

[0085] in, and represents the second observation, Indicates the coordinates of the feature points on the target in the camera coordinate system of the tracking device.

[0086] After obtaining the above-mentioned first observation results and second observation results, the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system is determined in combination with the position and posture of the scanning device in the camera coordinate system of the tracking device at the first moment (that is, the first posture).

[0087] It should be noted that when the scanning device observes the target, it must be able to successfully observe the features on the target, and the number of observed features must be sufficient to successfully calculate the first observation result. Similarly, when the tracking device observes the target, it must also be able to successfully observe the features on the target, and the number of observed features must be sufficient to successfully calculate the second observation result.

[0088] In this embodiment, during the scanning process performed by the scanning device, the target can be used to assist in determining the conversion relationship between the scanning device coordinate system and the camera coordinate system of the scanning device, so that this conversion relationship does not need to rely on the accuracy of the tracking device. The scanning device or tracking device can be replaced in the middle without recalibration, thereby improving the overall scanning efficiency of the tracking scanning.

[0089] Additionally, in one embodiment, the three-dimensional scanning method may further include:

[0090] When a previous transformation relationship already exists, the transformation relationship is updated based on the latest observation results and the latest first pose.

[0091] Among them, after the above-mentioned conversion relationship has been solved, the observation results of the scanning device and the tracking device on the target at a new moment can be combined with the first position of the scanning device in the camera coordinate system of the tracking device at that moment to solve the conversion relationship again, and the existing conversion relationship can be updated using the newly solved conversion relationship.

[0092] For example, the newly solved transformation relationship may be directly used to replace the existing transformation relationship, or a weighted average of multiple solved transformation relationships may be performed, or a threshold value may be set based on empirical values ​​to eliminate calculated noise, etc. The specific updating method is not limited here.

[0093] In this embodiment, the latest observation result and the latest first pose are used to update the transformation relationship, which can improve the accuracy of the transformation relationship calculation result and thus improve the accuracy of three-dimensional reconstruction.

[0094] In one embodiment, the three-dimensional scanning method may further include:

[0095] If the conversion relationship cannot be obtained and the acquisition of the observation result fails, the conversion processing of the scan data currently acquired by the scanning device is terminated. In some embodiments, if the scanning device and the tracking device cannot observe the target features set on the target at the same time, or the observed target features do not meet the preset feature splicing conditions, it is determined that the acquisition of the observation result has failed.

[0096] Among them, if the conversion relationship has not been solved before, the conversion relationship cannot be obtained. For example, during the scanning process of the scanning device, the scanning device and the tracking device have not been able to synchronously observe the target at a certain moment, so that the above conversion relationship cannot be solved. When the scanning device cannot observe the target, or the observed target features cannot be successfully spliced ​​with the target model, it is determined that the scanning device has failed to observe the target. When the tracking device cannot observe the target, or the observed target features cannot be successfully spliced ​​with the target model, it is determined that the tracking device has failed to observe the target. When the tracking device and / or the scanning device fails to observe the target, it is determined that the acquisition of the observation result has failed. When there is no conversion relationship solved before, and the current scanning device, and / or the tracking device fails to observe the target, it is confirmed that the scanning data currently obtained by the scanning device cannot be successfully converted to the camera coordinate system of the tracking device, so the scanning data of the current frame can be discarded.

[0097] In one embodiment, the three-dimensional scanning method may further include:

[0098] When the first pose at the current moment cannot be obtained, the first observation result of the scanning device on the target is obtained, and the second observation result of the tracking device is obtained by synchronously observing the target; based on the first observation result and the second observation result, the relative pose of the camera between the scanning device and the tracking device is determined.

[0099] If the tracking device cannot observe the scanning device at the current moment, or the features of the markers set on the observed scanning device cannot be spliced ​​with the scanning device model, it is determined that the acquisition of the above-mentioned first pose at the current moment has failed. In this case, the relative pose of the cameras between the scanning device and the tracking device can be determined based on the observation results of the scanning device and the tracking device on the target:

[0100] ;

[0101] ;

[0102] in, and Represents the relative camera pose between the scanning device and the tracking device.

[0103] Therefore, in the case where the tracking device fails to observe the scanning device, this embodiment can still determine the relative position of the cameras between the tracking device and the scanning device based on the assistance of the target, thereby improving the stability in the tracking three-dimensional scanning scenario.

[0104] Additionally, in one embodiment, after converting the scan data collected by the scanning device into a camera coordinate system of the tracking device according to the relative position of the camera, the three-dimensional scanning method may further include:

[0105] The observation results obtained by the scanning device and the tracking device observing the target simultaneously at the first pose and at the same time are jointly optimized.

[0106] The first pose and the observation results can be jointly optimized based on the two-dimensional coordinates of the features on the target simultaneously observed by the scanning device and the tracking device, the internal and external parameters of the tracking device, and the internal and external parameters of the scanning device.

[0107] In this embodiment, by jointly optimizing the first pose and the observation result, the accuracy of the three-dimensional reconstruction result of the tracking three-dimensional scanning can be improved.

[0108] In addition, in one embodiment, the three-dimensional scanning method may further include:

[0109] The object to be measured is reconstructed in three dimensions based on the scan data in the tracking device's camera coordinate system. This can be accomplished by converting the scan data of the object to be measured, collected by the scanning device at different scanning positions, into the tracking device's camera coordinate system. This allows for the acquisition of multiple frames of scan data in a unified coordinate system, enabling the object to be reconstructed in three dimensions. Finally, the 3D reconstruction results of the object to be measured can be output, for example, as a visual display of the reconstructed 3D model of the object to be measured on a display screen.

[0110] Figure 3 is a flow chart of a three-dimensional scanning method in some embodiments, such as Figure 3 As shown, the three-dimensional scanning method includes the following steps:

[0111] Step S301: The scanning device performs scanning.

[0112] Step S302 , determining whether the tracking device observes the scanning device; if so, executing steps S304 and S308 respectively; otherwise, executing step S303 .

[0113] Step S303, determine whether the tracking device observes the target; if so, execute step S305; otherwise, execute step S311.

[0114] Step S304: Acquire the position and posture of the scanning device in the camera coordinate system of the tracking device.

[0115] Step S305: Acquire the position and posture of the target in the camera coordinate system of the tracking device.

[0116] Step S306 , determining whether the scanning device can observe the target; if so, executing steps S307 and S308 ; otherwise, executing step S311 .

[0117] Step S307 , obtaining the position and posture of the target in the camera coordinate system of the scanning device; and executing step S310 .

[0118] Step S308 , solving and updating the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system.

[0119] Step S309 , determining whether the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system is obtained; if so, executing step S310 ; otherwise, executing step S311 .

[0120] Step S310: Calculate the relative pose of the cameras of the scanning device and the tracking device. Execute step S312.

[0121] Step S311: discard the scan data of the current frame.

[0122] Step S312, obtaining the scanning data of the current frame of the scanning device; and executing steps S313 and S315 respectively.

[0123] Step S313: convert the scan data of the current frame into the camera coordinate system of the tracking device.

[0124] Step S314, completing the scanning of the current frame, and returning to step S301.

[0125] Step S315: Buffer the scan data of the current frame.

[0126] Step S316, joint optimization; wherein, the position and posture of the scanning device under the tracking device and the observation results of the tracking device and the scanning device on the target at the same time are jointly optimized.

[0127] Step S317: Output the scanning result. Steps S301 to S310 and steps S312 to S314 may all be real-time processes, and steps S311 and S315 to S317 may be post-processing processes.

[0128] Steps S301 to S317 establish a tracking-based 3D scanning system constructed using the tracking device, scanning device, and target. Prior to use, only the tracking device and scanning device require separate calibration, eliminating the need for joint calibration of the tracking and scanning devices. This improves the efficiency and convenience of overall tracking-based 3D scanning. Furthermore, since joint calibration of the tracking and scanning devices is unnecessary, the accuracy of the tracking and scanning devices remains unchanged, allowing for easy replacement of either device.

[0129] Figure 4 FIG. 1 is a schematic diagram of a scanning device coordinate system in this embodiment. Figure 4As shown in the figure, the center of the scanning device structure or a certain reference point can be used as the origin of the coordinate system to establish the scanning device coordinate system. The scanning device coordinate system is a three-dimensional coordinate system that can include the X axis, Y axis and Z axis. Figure 4 The scanning device coordinate system is constructed with the center of the scanning device's mechanical structure as the origin of the coordinate system. The scanning device coordinate system describes the local reference coordinate system of the scanning device itself.

[0130] Figure 5 Schematic diagram of the camera coordinate system of a scanning device in this embodiment. Figure 5 As shown, the scanning device camera coordinate system is the three-dimensional coordinate system of a camera sensor of the scanning device, which may include the X-axis, the Y-axis, and the Z-axis. Specifically, the scanning device camera coordinate system may be established using the optical center of a camera sensor of the scanning device (the optical center of the lens) as the origin. In this embodiment, the scanning device camera coordinate system may be the coordinate system in which the coordinates of the scan data are captured by the scanning device. It will be understood that the scanning device camera coordinate system is different at different positions of the scanning device. Therefore, different frames of scan data captured by the scanning device at different positions will also have different coordinate systems.

[0131] Figure 6 Schematic diagram of the camera coordinate system of a tracking device in this embodiment. Figure 6 As shown, the tracking device camera coordinate system is the three-dimensional coordinate system of a camera sensor of the tracking device, which may include the X-axis, the Y-axis, and the Z-axis. Specifically, the tracking device camera coordinate system may be established by using the optical center of a camera sensor of the tracking device as the origin. In this embodiment, the tracking device camera coordinate system may be a unified coordinate system obtained by unifying the coordinate systems of different frames of scan data.

[0132] This embodiment also provides a three-dimensional scanning device for implementing the above-mentioned embodiments and preferred embodiments. Details already described will not be repeated. The terms "module," "unit," "subunit," etc. used below may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0133] Figure 7 70 is a structural block diagram of the three-dimensional scanning device of this embodiment, which is used in a three-dimensional scanning system. The three-dimensional scanning system includes a scanning device, a tracking device, and a target; Figure 7 As shown, the three-dimensional scanning device 70 includes: a first acquisition module 72, a second acquisition module 74, a solution module 76 and a conversion module 78; wherein:

[0134] The first acquisition module 72 is used to obtain the first pose of the scanning device in the camera coordinate system of the tracking device, which is obtained by the tracking device performing real-time tracking of the scanning device when the scanning device performs scanning; the second acquisition module 74 is used to obtain the conversion relationship between the camera coordinate system of the scanning device and the scanning device coordinate system; the conversion relationship is determined based on the observation results and the first pose obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning; the solution module 76 is used to determine the relative camera pose between the scanning device and the tracking device based on the conversion relationship and the first pose; the conversion module 78 is used to convert the scanning data collected by the scanning device to the camera coordinate system of the tracking device according to the relative camera pose.

[0135] It should be noted that the above modules can be functional modules or program modules, and can be implemented through software or hardware. For modules implemented through hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0136] It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0137] In addition, a three-dimensional scanning system is also provided in this embodiment. Figure 8 is a schematic structural diagram of the three-dimensional scanning system 80 of this embodiment, as shown in FIG. Figure 8 As shown, the three-dimensional scanning system includes a scanning device 82, a tracking device 84, a target 86 and a processor 88; wherein the target 86 is set within the tracking range of the tracking device 84; the scanning device 82 and the tracking device 84 are respectively connected to the processor 88;

[0138] The scanning device 82 is used to scan the object to be measured; the tracking device 84 is used to observe the scanning device 82 and the target 86 respectively when the scanning device 82 performs the scanning; the processor 88 is used to execute the three-dimensional scanning method provided by any of the above embodiments.

[0139] In conjunction with the three-dimensional scanning method provided in the above embodiments, a storage medium may also be provided in this embodiment to implement the three-dimensional scanning method. The storage medium stores a computer program that, when executed by a processor, implements any one of the three-dimensional scanning methods in the above embodiments.

[0140] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0142] Obviously, the accompanying drawings are merely examples or embodiments of the present application. A person skilled in the art can also apply the present application to other similar situations based on these drawings without inventive effort. Furthermore, it is understandable that, although the work involved in this development process may be complex and lengthy, certain design, manufacturing, or production changes based on the technical content disclosed in this application are merely routine technical means for a person skilled in the art and should not be considered to constitute a deficiency in the disclosure of the present application.

[0143] The term "embodiment" as used in this application refers to specific features, structures, or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of this application. The appearance of this phrase in various places in the specification does not necessarily mean that the embodiment is the same, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is understood, either explicitly or implicitly, by those skilled in the art that the embodiments described in this application can be combined with other embodiments when there is no conflict.

[0144] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of patent protection. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A three-dimensional scanning method, characterized in that: For a three-dimensional scanning system, the three-dimensional scanning system includes a scanning device, a tracking device, and a target, the method comprising: When the scanning device performs scanning, obtaining the first pose of the scanning device in the camera coordinate system of the tracking device obtained by the tracking device tracking the scanning device in real time; Obtaining a conversion relationship between a camera coordinate system of the scanning device and a coordinate system of the scanning device; the conversion relationship is based on observation results and the first posture determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning; Determining a relative camera pose between the scanning device and the tracking device according to the conversion relationship and the first pose; The scanning data collected by the scanning device is converted into the camera coordinate system of the tracking device according to the relative position of the camera.

2. The three-dimensional scanning method according to claim 1, characterized in that: The conversion relationship is based on the observation results and the first posture determination obtained by the scanning device and the tracking device observing the target at the same time when the scanning device performs scanning, including: Obtaining the scanning device observing the target at a first moment, determining the position and posture of the target in the camera coordinate system of the scanning device, and obtaining a first observation result; Obtaining the target observed by the tracking device at a first moment, and determining the position and posture of the target in the camera coordinate system of the tracking device to obtain a second observation result; Obtaining a position and posture of the scanning device in a camera coordinate system of the tracking device determined by the tracking device when the tracking device observes the scanning device at a first moment, to obtain the first posture; The conversion relationship is determined based on the first observation result, the second observation result and the first posture.

3. The three-dimensional scanning method according to claim 1, characterized in that: The method further comprises: When a previous conversion relationship already exists, the conversion relationship is updated according to the latest observation result and the latest first posture.

4. The three-dimensional scanning method according to claim 1, characterized in that: The method further comprises: In the case that the conversion relationship cannot be obtained and the acquisition of the observation result fails, the conversion process of the scanning data currently acquired by the scanning device is terminated.

5. The three-dimensional scanning method according to claim 4, wherein: When the scanning device and the tracking device cannot observe the target features set on the target at the same time, or the observed target features do not meet the preset feature splicing conditions, it is determined that the acquisition of the observation result has failed.

6. The three-dimensional scanning method according to claim 1, characterized in that: The method further comprises: When the first posture at the current moment cannot be obtained, a first observation result of the scanning device on the target is obtained, and a second observation result of the tracking device synchronously observing the target is obtained; based on the first observation result and the second observation result, the relative posture of the camera between the scanning device and the tracking device is determined.

7. The three-dimensional scanning method according to claim 1, characterized in that: After converting the scan data collected by the scanning device into the camera coordinate system of the tracking device according to the relative position of the camera, the method further includes: The first posture and observation results obtained by the scanning device and the tracking device observing the target at the same time are jointly optimized.

8. The three-dimensional scanning method according to any one of claims 1 to 7, characterized in that: The method further comprises: The three-dimensional reconstruction of the object to be measured is completed according to the scanning data in the camera coordinate system of the tracking device.

9. A three-dimensional scanning device, characterized in that: Used in a three-dimensional scanning system, the three-dimensional scanning system includes a scanning device, a tracking device and a target, the three-dimensional scanning device includes: a first acquisition module, a second acquisition module, a solution module and a conversion module; wherein: The first acquisition module is configured to acquire, when the scanning device performs scanning, a first pose of the scanning device in the camera coordinate system of the tracking device, obtained by the tracking device tracking the scanning device in real time; The second acquisition module is configured to acquire a conversion relationship between a camera coordinate system of the scanning device and a coordinate system of the scanning device; the conversion relationship is based on observation results of the target observed by the scanning device and the tracking device at the same time when the scanning device performs scanning and the first posture determination; The solving module is configured to determine a relative camera pose between the scanning device and the tracking device based on the conversion relationship and the first pose; The conversion module is used to convert the scanning data collected by the scanning device into the camera coordinate system of the tracking device according to the relative position of the camera.

10. A three-dimensional scanning system, characterized in that: include: A scanning device, a tracking device, a target, and a processor; wherein the target is set within the tracking range of the tracking device; The scanning device and the tracking device are respectively connected to the processor; The scanning device is used to scan the object to be measured; The tracking device is used to observe the scanning device and the target respectively when the scanning device performs scanning; The processor is configured to execute the three-dimensional scanning method according to any one of claims 1 to 8.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the three-dimensional scanning method according to any one of claims 1 to 8 are implemented.

Images