Three-dimensional scanning method for transferring station by using marker ball and storage medium

Abstract

The invention discloses a three-dimensional scanning method using a marker ball to transfer a station and a storage medium, and relates to the technical field of computers, a target is scanned through a scanner, and first three-dimensional information of the target under a scanner coordinate system is obtained. And synchronously tracking the scanner through the tracker to obtain second three-dimensional information of the marking ball under the tracker coordinate system. The method comprises the following steps: modeling a marker ball based on a spatial position relationship between the marker ball and a scanner to obtain a three-dimensional model of the marker ball, and determining a first conversion matrix for converting a model coordinate system used for modeling into a scanner coordinate system; based on the three-dimensional model and the second three-dimensional information, determining a second conversion matrix for converting the model coordinate system into the tracker coordinate system; and converting the first three-dimensional information to a tracker coordinate system based on the first conversion matrix and the second conversion matrix to obtain third three-dimensional information for performing three-dimensional reconstruction on the target. The technical problem that in the prior art, the scanning precision is low when a mark point is used for station transferring is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a three-dimensional scanning method and storage medium using a marker ball for station transfer. Background Art

[0002] A scanner is a device that reconstructs the three-dimensional information of an object's surface through binocular reconstruction. There are mainly handheld scanners and tracking scanners, and both directly or indirectly use marker points to achieve the pose determination of the object.

[0003] The handheld scanner realizes station transfer by pasting marker points in various directions on the object's surface. The problems are that pasting marker points is time-consuming, and when the camera's line of sight moves to the side position of the marker point, only the edge of the marker point can be seen, which cannot guarantee the accuracy. That is, the accuracy will fluctuate after being tilted by anisotropy in space when using the method of pasting marker points, and there are stitching errors.

[0004] The station transfer of the tracking scanner relies on a target ball marker point to achieve. The target ball is actually a polyhedron with multiple marker point surfaces formed by splicing multiple marker points. Compared with pasting marker points on the object's surface, using the target ball can reduce the number of marker points. However, since the target ball is a collection of multiple marker points, the edge of the tilted marker point can always be seen during station transfer, and the accuracy cannot be guaranteed either.

[0005] In the prior art, whether it is a marker point or a target ball, there is a problem that due to anisotropy in space, only the edge of the plane can be seen, resulting in stitching errors in the three-dimensional reconstruction of the object. Summary of the Invention

[0006] In an embodiment of this application, a three-dimensional scanning method and storage medium using a marker ball for station transfer are provided to solve the technical problem of stitching errors existing in the prior art when using marker points or target balls for marking.

[0007] To achieve the above object, the embodiments of this application adopt the following technical solutions:

[0008] In a first aspect, an embodiment of this application provides a three-dimensional scanning method using a marker ball for station transfer, and the method includes:

[0009] Scanning a target through a scanner to obtain first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marker balls are provided on the scanner;

[0010] When the scanner scans the target, synchronously tracking the scanner through a tracker to obtain second three-dimensional information of the plurality of marker balls in the tracker coordinate system;

[0011] Model the multiple marker balls based on the spatial position relationship between the multiple marker balls and the scanner to obtain a three-dimensional model of the multiple marker balls, and determine a first transformation matrix for transforming the model coordinate system used for modeling to the scanner coordinate system;

[0012] Based on the three-dimensional model of the multiple marker balls and the second three-dimensional information, determine a second transformation matrix for transforming the model coordinate system to the tracker coordinate system;

[0013] Based on the first transformation matrix and the second transformation matrix, transform the first three-dimensional information to the tracker coordinate system to obtain third three-dimensional information, and the third three-dimensional information is used for three-dimensional reconstruction of the target.

[0014] Combined with the first aspect, in a possible design, the tracker includes multiple cameras, and synchronously tracking the scanner through the tracker to obtain the second three-dimensional information of the multiple marker balls in the tracker coordinate system includes:

[0015] Synchronously track the scanner through multiple cameras of the tracker to obtain a first image of the scanner collected by each camera;

[0016] Based on the first image collected by each camera, determine the two-dimensional ellipse information of each marker ball among the multiple marker balls;

[0017] Based on the two-dimensional ellipse information of each marker ball among the multiple marker balls, construct the second three-dimensional information of the multiple marker balls in the tracker coordinate system.

[0018] Combined with the first aspect, in a possible design, the two-dimensional ellipse information includes the centroid, major axis, and minor axis of the marker ball. Based on the first image collected by each camera, determining the two-dimensional ellipse information of each marker ball among the multiple marker balls includes:

[0019] Obtain a second image corresponding to each marker ball from the first image collected by each camera;

[0020] Determine the center of the second image as the centroid of the marker ball;

[0021] Determine the major axis and minor axis of the marker ball in the second image.

[0022] Combined with the first aspect, in a possible design, obtaining a second image corresponding to each marker ball from the first image collected by each camera includes:

[0023] Obtain the edge information of each marker ball in the first image;

[0024] The second image is cropped from the first image based on the edge information of each of the marked balls.

[0025] Combined with the first aspect, in a possible design, the first transformation matrix is represented by RT m and the second transformation matrix is represented by RT n ; the first three-dimensional information is represented by object XYZ and the third three-dimensional information is represented by world XYZ .

[0026] The third three-dimensional information is obtained by using the first relational expression:

[0027] world XYZ = object XYZ × RT m × RT n .

[0028] In a second aspect, an embodiment of the present application provides a three-dimensional scanning method using a marked ball transfer station. The method includes:

[0029] Scanning a target by a scanner to obtain first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marked balls are provided on the target;

[0030] Based on the spatial position relationship between the plurality of marked balls and the target, modeling the plurality of marked balls to obtain three-dimensional models of the plurality of marked balls, and determining a first transformation matrix for converting the model coordinate system used for modeling to the scanner coordinate system;

[0031] Based on the first transformation matrix, converting the first three-dimensional information to the scanner coordinate system to obtain fourth three-dimensional information, and the fourth three-dimensional information is used for three-dimensional reconstruction of the target.

[0032] Combined with the second aspect, in a possible design, the first transformation matrix is represented by RT m ; the fourth three-dimensional information is represented by benchmark XYZ ; and the first three-dimensional information is represented by object XYZ .

[0033] The fourth three-dimensional information is obtained by using the second relational expression:

[0034] benchmark XYZ = object XYZ × RT m .

[0035] In a third aspect, an embodiment of the present application provides a three-dimensional scanning device using a marker ball transfer station. A scanning module is configured to scan a target through a scanner to obtain first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marker balls are provided on the scanner;

[0036] A tracking module is configured to, when the scanner scans the target, synchronously track the scanner through a tracker to obtain second three-dimensional information of the plurality of marker balls in the tracker coordinate system;

[0037] A first conversion module is configured to, based on the spatial position relationship between the plurality of marker balls and the scanner, model the plurality of marker balls to obtain a three-dimensional model of the plurality of marker balls, and determine a first conversion matrix for converting the model coordinate system used for modeling to the scanner coordinate system;

[0038] A second conversion module is configured to, based on the three-dimensional model of the plurality of marker balls and the second three-dimensional information, determine a second conversion matrix for converting the model coordinate system to the tracker coordinate system;

[0039] A reconstruction module is configured to, based on the first conversion matrix and the second conversion matrix, convert the first three-dimensional information to the tracker coordinate system to obtain third three-dimensional information, and the third three-dimensional information is used for three-dimensional reconstruction of the target.

[0040] In a fourth aspect, an embodiment of the present application provides a computer device, including a memory and a processor. A computer program is stored in the memory, and the processor is configured to run the computer program to execute the method of the first aspect and its possible design manners, or execute the method of the second aspect and its possible design manners.

[0041] In a fifth aspect, an embodiment of the present application provides a storage medium, in which a computer program is stored. The computer program is configured to execute the method of the first aspect and its possible design manners, or execute the method of the second aspect and its possible design manners when running.

[0042] Compared with the prior art, a three-dimensional scanning method and a storage medium using a marker ball transfer station provided by an embodiment of the present application. In the method, a target is scanned by a scanner to obtain first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marker balls are provided on the scanner. When the scanner scans the target, a tracker synchronously tracks the scanner to obtain second three-dimensional information of the plurality of marker balls in the tracker coordinate system. Based on the spatial position relationship between the plurality of marker balls and the scanner, the plurality of marker balls are modeled to obtain three-dimensional models of the plurality of marker balls, and a first transformation matrix for transforming the model coordinate system used for modeling to the scanner coordinate system is determined. Based on the three-dimensional models of the plurality of marker balls and the second three-dimensional information, a second transformation matrix for transforming the model coordinate system to the tracker coordinate system is determined. Then, based on the first transformation matrix and the second transformation matrix, the first three-dimensional information is transformed to the tracker coordinate system to obtain third three-dimensional information, and the third three-dimensional information is used for three-dimensional reconstruction of the target. In this method, the marker ball replaces the target ball. The sphere of the marker ball has the characteristic of isotropy, so the shape of the marker ball scanned by the tracker from different orientations is the same. Then, the second three-dimensional information (i.e., the measured value) of the marker ball generated in the tracker coordinate system is closer to the true value of the marker ball, with a small measurement error and high accuracy. Based on the transformation from the value of the model coordinate system of the marker ball to the measured value, a transpose matrix is generated, and subsequent measurement and reconstruction work are based on this transpose matrix to ensure the benchmark unity and measurement accuracy of the spatial position of the target.

[0043] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more concise and understandable. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation of the present application. In the drawings:

[0045] Figure 1 A hardware structure block diagram of a computer device provided by an embodiment of the present application is shown;

[0046] Figure 2 A flowchart of a three-dimensional scanning method using a marker ball transfer station provided by an embodiment of the present application is shown;

[0047] Figure 3 A flowchart of a method for obtaining second three-dimensional information provided by an embodiment of the present application is shown;

[0048] Figure 4 A flowchart of a method for obtaining two-dimensional elliptical information of a marker ball provided by an embodiment of the present application is shown;

[0049] Figure 5 The flowchart of a method for obtaining an image of a marked ball provided by an embodiment of the present application is shown;

[0050] Figure 6 The flowchart of another three-dimensional scanning method using a marked ball for station transfer provided by an embodiment of the present application is shown;

[0051] Figure 7 The system framework diagram of a tracking scanner provided by an embodiment of the present application is shown;

[0052] Figure 8 The system framework diagram of a handheld scanner provided by an embodiment of the present application is shown;

[0053] Figure 9 The structural block diagram of a three-dimensional scanning device using a marked ball for station transfer provided by an embodiment of the present application is shown. Detailed implementation manners

[0054] For a clearer understanding of the purpose, technical solution, and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0055] Unless otherwise defined, the technical terms or scientific terms involved in the present application shall have the general meaning understood by those with ordinary skills in the technical field to which the present application belongs. In the present application, words such as "a", "one", "a kind of", "the", "these", etc. do not indicate a limitation in quantity, and they can be singular or plural. The terms "including", "comprising", "having" and any variants thereof involved in the present application are intended to cover non-exclusive inclusion; for example, a process, method, device, product or equipment including a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent in these processes, methods, products or equipment. The terms "connected", "coupled", etc. involved in the present application do not limit to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" involved in the present application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" may represent: A exists alone, A and B exist simultaneously, and B exists alone. Usually, the character " / " indicates that the objects associated before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in the present application only distinguish similar objects and do not represent a specific order for the objects.

[0056] A transfer station refers to the process in which, when using a scanner to scan an object, the position of the scanner continuously changes to scan the target from different orientations. The scanner usually uses a binocular camera. By acquiring two images of the same target from two orientations by the two cameras, the three-dimensional coordinates of the target can be obtained. In the prior art, marker points or target balls are used to assist in the transfer station.

[0057] In the former case, multiple marker points are pasted on the object, and the three-dimensional coordinates of each marker point collected by the scanner are extracted. Based on the model coordinates of each point, a transformation matrix for mapping the marker points from the model coordinate system to the scanner coordinate system is generated. Based on this transformation matrix, the three-dimensional coordinates of the target in each direction can be unified into the same coordinate system to achieve the three-dimensional reconstruction of the target. In this solution, since the marker points are generally made of round paper sheets and can be approximately regarded as planes, only when the scanning line of sight is perpendicular to the marker point can the front of the marker point be scanned. When the scanning line of sight is slightly offset, only the inclined marker point can be seen. Even when the scanning line of sight is parallel to the marker point, only the side of the marker point can be seen. This is because the marker points are anisotropic in space, and the accuracy will fluctuate after the marker points are tilted, so there will be stitching errors.

[0058] The target ball is actually a polyhedron formed by pasting multiple marker points. For example, if it is formed by pasting six marker points, the target ball is a hexahedron; if it is formed by pasting eight marker points, the target ball is an octahedron. Using the target ball to assist in the transfer station, only one front marker point can be seen, and the other marker points are inclined in the scanning line of sight. Once the marker points are tilted, the marking accuracy cannot be guaranteed. And in the prior art, in order to prevent relative movement between the marker points that make up the target ball, a glass target ball is usually manufactured at a relatively high cost. The glass target ball not only does not solve the problem of low modeling accuracy, but also has a high manufacturing cost.

[0059] Based on this, the embodiments of the present application provide a three-dimensional scanning method using a marker ball for transfer station, which can be applied to common handheld scanners and tracking scanners at present. The following is an explanation of the two types of scanners.

[0060] For each target scanned by the handheld scanner, the marker points need to be re-pasted on the target to be scanned. As the scanner continuously moves, the scanning of the target is completed.

[0061] Compared with the handheld scanner that needs to paste marker points on each target, the tracking scanner only needs to paste the marker points on the scanner. The scanner scans the target, and the tracker tracks the scanner. In this way, even if the target is replaced, there is no need to re-paste the marker points.

[0062] After the scanning is completed, both of them also reconstruct the three-dimensional information of the target into the reference coordinate system through the mapping relationship of the three-dimensional coordinates of the target in different coordinate systems to achieve the reconstruction of the target.

[0063] The method provided by the embodiment of the present application uses a marked ball to replace the marked point and the target ball. The marked ball is a sphere and is isotropic in space. Therefore, the marked ball looks the same at each angle, and there is no problem that the area seen from the side of the marked point is too small and the error is too large.

[0064] This method can be executed on a terminal, a computer, or a similar computing system. Taking running on a computer device as an example, Figure 1 The hardware structure block diagram of a computer device provided by the embodiment of the present application is shown. As Figure 1 shown, the computer device may include one or more ( Figure 1 only one is shown in the figure) processors 102 and a memory 104 for storing data. The processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above computer device may further include a transmission device 106 for communication functions and an input / output device 108. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above computer device. For example, the computer device may further include more or fewer components than those Figure 1 shown in the figure, or have a different configuration from that Figure 1 shown in the figure.

[0065] The memory 104 can be used to store computer programs, for example, software programs and modules of application software. The processor 102 executes various functional applications and data processing by running the computer programs stored in the memory 104, that is, the above method is implemented. The memory 104 can be used to store data. The memory 104 may include a high-speed random access memory and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some instances, the memory 104 may further include a memory remotely set relative to the processor 102, and these remote memories can be connected to the computer device through a network. Examples of the above network include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0066] The transmission device 106 is used to receive or send data via a network. Specific examples of the above network may include a wireless network provided by a communication provider of a computer device. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, abbreviated as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a Radio Frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0067] The following takes the application scenario on a tracking scanner as an example for illustration.

[0068] Figure 2 The flowchart of a three-dimensional scanning method using a marker ball transfer station provided by an embodiment of the present application is shown, as Figure 2 shown, this method includes steps S201 to S205.

[0069] Step S201: Scan the target through the scanner to obtain the first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marker balls are provided on the scanner.

[0070] The plurality of marker balls can be pasted on the outer surface of the scanner or installed on the external frame of the scanner. Since each marker ball is isotropic in space, the two-dimensional images of the marker balls obtained by photographing the marker balls from different orientations are the same, so that the three-dimensional information of the marker balls can be accurately constructed. That is to say, the method of using the two-dimensional images of multiple marker points to construct three-dimensional information has low accuracy, while using the two-dimensional images of multiple marker balls to construct three-dimensional information has high accuracy.

[0071] In this step, the scanner uses a binocular camera. The advantage of such a setting is that a three-dimensional image of the target can be obtained only through one scanner. For the method of the binocular camera collecting the three-dimensional image of the target, specific reference can be made to the description of the prior art, and no redundant introduction will be made here. In some embodiments, the function of the binocular camera can be realized by setting a plurality of scanners, and its principle is similar to that of the binocular camera, so reference can be made to the introduction of the binocular camera.

[0072] In this step, the scanner coordinate system refers to a coordinate system based on the coordinates of the scanner. Specifically, when constructing the position of the target, a reference needs to be determined, and based on the reference, the coordinate points of the X, Y, and Z axes of the target are determined, so as to obtain the three-dimensional information of the target. The coordinates of the scanner can be the coordinates of any position on the scanner. As an optional example, the center point of the scanner lens is used as the reference to construct the scanner coordinate system.

[0073] Step S202: When the scanner scans the target, the tracker synchronously tracks the scanner to obtain the second three-dimensional information of multiple marker balls in the tracker coordinate system.

[0074] In some embodiments, the number of trackers is one, and the tracker uses a binocular camera to realize the three-dimensional information of the marker ball based on two-dimensional image reconstruction.

[0075] In other embodiments, the number of trackers is multiple to realize the stereoscopic imaging function of the binocular camera.

[0076] What the tracker (or tracking camera) obtains is the planar image of the marker ball. From the planar images in multiple different orientations, the second three-dimensional information of the marker ball in the tracker coordinate system can be obtained. It can be understood that due to the isotropy of the sphere in space, the shape of the marker ball in the planar image of the marker ball does not change. In the conventional technology, the marker point is circular when viewed from the front and elliptical when viewed from the side. As the scanning line of sight continues to tilt, the ellipse becomes flatter and flatter until it becomes a line. This change makes it easy to cause deviations in calculating the three-dimensional information of the marker point. Once there is a deviation in the three-dimensional information of the marker point, it will affect the stitching of the planar image of the target, and then there will be an error between the reconstructed stereoscopic image and the actual shape of the target. In this step, by tracking the sphere with the tracker, the problem of low accuracy existing in the current marker point transfer station is well solved.

[0077] In some embodiments, step S202 includes steps S301 to S303 as Figure 3 shown.

[0078] Step S301: Through multiple cameras of the tracker, synchronously track the scanner to obtain the first image of the scanner in space collected by each camera.

[0079] Step S302: Based on the first image collected by each camera, determine the two-dimensional ellipse information of each marker ball among the multiple marker balls.

[0080] Step S303: Based on the two-dimensional ellipse information of each marker ball among the multiple marker balls, construct the second three-dimensional information of the multiple marker balls in the tracker coordinate system.

[0081] The following is an exemplary description of this embodiment.

[0082] In step S301, the number of cameras is 2. At the first moment, each of the two cameras collects a first image, obtaining two first images. Each first image contains the image information of the marker ball.

[0083] In step S302, the planar information of the marker ball is obtained from the two first images respectively.

[0084] In this step, the planar information of the marker ball is represented by two-dimensional ellipse information. This is because, affected by the camera shooting accuracy, mutations may occur during camera shooting. Therefore, the image of the marker ball captured by the camera is approximately circular, but it may be only an ellipse. Subsequently, the ellipse can be restored to a circle through an algorithm. The two-dimensional ellipse information contains key parameters for constructing the planar information of the marker ball, such as the center of the circle, the major and minor axes, etc.

[0085] In step S303, based on the planar information of multiple marker balls, the three-dimensional information of the marker balls is constructed, which can be achieved by the binocular imaging principle in related technologies. For specific details, refer to the description of step S201.

[0086] Since the planar information of the marker ball captured from various angles is circular in shape, constructing the three-dimensional information of the marker ball based on the planar information of the marker ball results in the second three-dimensional information of the marker ball in the tracker coordinate system being closer to the true three-dimensional information of the marker ball. In this way, the three-dimensional information of the target constructed using the second three-dimensional information subsequently is more accurate.

[0087] In some embodiments, step S302 further includes steps S401 to S403 as Figure 4 shown.

[0088] Step S401: Obtain the second image corresponding to each marker ball from the first image collected by each camera.

[0089] Step S402: Determine the center of the second image as the centroid of the marker ball.

[0090] Step S403: Determine the major axis and minor axis of the marker ball in the second image.

[0091] By step S401, the planar image (second image) of the marker ball is determined from the first image, and then by step S402, the centroid of the marker ball (i.e., the center of the planar circle), the major axis, and the minor axis are obtained from the planar image of the marker ball. In this embodiment, the center of the second image is determined as the centroid of the marker ball, which can quickly obtain the key parameters of the planar information of the marker ball.

[0092] In some embodiments, step S401 further includes steps S501 to S502 as Figure 5 shown.

[0093] Step S501: Obtain the edge information of each marker ball in the first image.

[0094] In this step, an image edge extraction operator can be used to obtain the edge information of the marker ball. Specifically, the first image is converted into a grayscale image, and the edge information of the marker ball is determined based on the grayscale information of each pixel point.

[0095] Step S502: Crop a second image from the first image based on the edge information of each marker ball.

[0096] In this step, the first image is cropped along the edges of the marker balls to obtain the second image. In this way, the centroid of the marker balls can be determined more quickly from the second image.

[0097] In this embodiment, by extracting the edge information of the marker balls and cropping the first image based on the edge information, the second image can be quickly obtained from the first image. The method of extracting edges is characterized by simple operation and accurate results.

[0098] Step S203: Based on the spatial position relationship between multiple marker balls and the scanner, model the multiple marker balls to obtain three-dimensional models of the multiple marker balls, and determine a first transformation matrix for transforming the model coordinate system used for modeling to the scanner coordinate system.

[0099] In this step, the three-dimensional models are obtained based on the model coordinate system (i.e., the local coordinate system of the marker balls). Since the marker balls are fixed on the scanner and remain stationary, the relative position between the marker balls and the scanner remains unchanged. Therefore, the three-dimensional models of the marker balls can be transformed to the scanner coordinate system to obtain the first transformation matrix, denoted as RT m , and the first transformation matrix is a 4×4 transposed matrix, which is used to align and transform the model coordinate system of the marker balls and the measurement coordinate system of the scanner. Subsequent measurement and reconstruction work are based on this transposed matrix to ensure the accuracy of spatial positions.

[0100] Step S204: Based on the three-dimensional models of the multiple marker balls and the second three-dimensional information, determine a second transformation matrix for transforming the model coordinate system to the tracker coordinate system.

[0101] In this step, the three-dimensional models of the marker balls and the measurement data of the marker balls in the tracker coordinate system from the tracker are used to calculate the spatial transformation relationship of the marker balls from the local coordinate system to the tracker coordinate system, that is, to obtain the second transformation matrix. The second transformation matrix represents the transformation from the model coordinate system to the tracker coordinate system, which is used to ensure that their spatial positions can match. It ensures that the measurement data from different devices can be comprehensively used in the same coordinate system to avoid data benchmark inconsistencies.

[0102] Step S205: Based on the first transformation matrix and the second transformation matrix, transform the first three-dimensional information to the tracker coordinate system to obtain third three-dimensional information, and the third three-dimensional information is used for three-dimensional reconstruction of the target.

[0103] In this step, the first transformation matrix is used to transform the three-dimensional information in the scanner coordinate system to the model coordinate system of the marker ball. Then, the second transformation matrix is used to transform the three-dimensional information in the model coordinate system to the tracker coordinate system. The tracker coordinate system serves as the reference, and all measurement data are unified to the tracker coordinate system, so that the third three-dimensional information of the target can be obtained. After this step, this embodiment further includes: performing three-dimensional reconstruction on the target based on the third three-dimensional information.

[0104] The second transformation matrix is represented by RT n The fourth three-dimensional information is represented by benchmark XYZ The first three-dimensional information is represented by object XYZ The fourth three-dimensional information is obtained using the second relational expression:

[0105] benchmark XYZ = object XYZ × RT m .

[0106] Through the above steps S201 to S205, this embodiment provides a three-dimensional scanning method using a marker ball for station transfer applied to a tracking scanner. In this method, the marker ball replaces the target ball. By utilizing the isotropic characteristics of the sphere, the shape of the marker ball scanned by the tracker from different orientations is consistent. Then, the second three-dimensional information (i.e., the measured value) of the marker ball generated in the tracker coordinate system is closer to the true value of the marker ball, with a small measurement error and high accuracy. Based on the conversion from the value in the model coordinate system of the marker ball to the measured value, a transpose matrix is generated, and subsequent measurement and reconstruction work are based on this transpose matrix to ensure the benchmark unity and measurement accuracy of the spatial position of the target.

[0107] The following takes the application scenario on a handheld scanner as an example for illustration.

[0108] Figure 6 shows a flowchart of another three-dimensional scanning method using a marker ball for station transfer provided by an embodiment of the present application. As Figure 6 shown, this method includes steps S601 to S603.

[0109] Step S601: Scan the target through the scanner to obtain the first three-dimensional information of the target in the scanner coordinate system; among them, a plurality of marker balls are provided on the target.

[0110] In this step, the plurality of marker balls can be pasted on the outer surface of the target or placed beside the target to ensure that the target captured from each orientation has a marker ball. Since each marker ball is isotropic in space, the two-dimensional images of the marker balls obtained from different orientations are the same, so that the three-dimensional information of the marker ball can be accurately constructed.

[0111] In this step, the scanner uses a binocular camera. The advantage of this setting is that a three-dimensional image of the target can be obtained by only one scanner. For the method of the binocular camera to collect the three-dimensional image of the target, please refer to the description of the prior art for details, and no redundant introduction will be made here. In some embodiments, the function of the binocular camera can be realized by setting multiple scanners. The principle is similar to that of the binocular camera, so please refer to the introduction of the binocular camera.

[0112] In this step, the scanner coordinate system refers to the coordinate system based on the coordinates of the scanner. Specifically, when constructing the position of the target, a reference needs to be determined, and based on the reference, the coordinate points of the X, Y, and Z axes of the target are determined, so as to obtain the three-dimensional information of the target. The coordinates of the scanner can be the coordinates of any position on the scanner. As an optional example, the center point of the scanner lens is used as the reference to construct the scanner coordinate system.

[0113] Step S602: Based on the spatial position relationship between multiple marker balls and the target, model the multiple marker balls to obtain the three-dimensional models of the multiple marker balls, and determine the first transformation matrix for converting the model coordinate system used for modeling to the scanner coordinate system.

[0114] In this step, the three-dimensional model is obtained based on the model coordinate system (i.e., the local coordinate system of the marker ball), while the image scanned by the scanner is data based on the scanner coordinate system. Therefore, it is necessary to convert the three-dimensional model of the marker ball to the scanner coordinate system to obtain the first transformation matrix, denoted as RT m , and the first transformation matrix is a 4×4 transposed matrix, which is used to align and transform the three-dimensional model of the marker ball and the measurement coordinate system of the scanner. Subsequent measurement and reconstruction work are based on this transposed matrix to ensure the accuracy of the spatial position.

[0115] Step S603: Based on the first transformation matrix, convert the first three-dimensional information to the scanner coordinate system to obtain the fourth three-dimensional information, and the fourth three-dimensional information is used for three-dimensional reconstruction of the target.

[0116] This step uses the three-dimensional information markerXYZ of the marker ball in the first frame of data as the reference coordinate system, and the three-dimensional information obtained in the subsequent scanner coordinate system is spliced onto the reference coordinate system to obtain the transposed matrix. Subsequently, the surface three-dimensional information (the first three-dimensional information) of the target obtained is multiplied by the first transformation matrix and transposed to the same coordinate system as the first frame of data to realize the three-dimensional reconstruction of the target surface.

[0117] The calculation formula is: benchmark XYZ =object XYZ ×RT m .

[0118] Among them, the first transformation matrix is represented by RT m and the fourth three-dimensional information is represented by benchmark XYZ and the first three-dimensional information is represented by object XYZ .

[0119] Through the above steps S601 to S603, this embodiment provides a three-dimensional scanning method using a marker ball transfer station for a handheld scanner. In this method, the marker ball replaces the marker point. By using the isotropic characteristics of the sphere, the shape of the marker ball scanned by the scanner from different orientations is the same. Then, the second three-dimensional information (i.e., the measured value) of the marker ball generated in the scanner coordinate system is closer to the true value of the marker ball, with a small measurement error and high accuracy. Based on the conversion from the value of the model coordinate system of the marker ball to the measured value, a transposed matrix is generated, and subsequent measurement and reconstruction work are based on this transposed matrix to ensure the benchmark unity and measurement accuracy of the spatial position of the target.

[0120] The following further illustrates the method provided in the embodiments of the present application with a specific example. Figure 7 shows a system framework diagram of a tracking scanner provided in the embodiments of the present application. As Figure 7 shown, the system includes a terminal device, a tracking head, a scanning head, and an object to be scanned.

[0121] The terminal device can be a computer. The tracking head is equivalent to the tracker in the above text, the scanning head is equivalent to the scanner in the above text, and the object to be scanned is equivalent to the target in the above text.

[0122] First step: Determine the coordinate system.

[0123] Use a calibration board to calibrate the two binocular cameras of the tracking head and the scanning head to obtain the internal and external parameters of the cameras, determine the origin of the world coordinate system of the tracking head, obtain the spatial position relationship of all marker balls on the scanning head to obtain the three-dimensional spatial model m of the marker balls, and obtain the fixed transposed matrix RT of m and the scanning head camera coordinates m .

[0124] Second step: Scan.

[0125] Use the tracking head to obtain the original image of the scanning head in space, use an image edge extraction operator to obtain the image edge of the marker ball, calculate the gray centroid and the major and minor axes of the marker ball on the two-dimensional images obtained by the left and right cameras, and reconstruct the three-dimensional information marker of the marker ball in space at the terminal XYZ ; at the same time, the scanning head obtains the two-dimensional image information of the object to be scanned, and reconstructs the three-dimensional point information object of the object to be scanned at the terminal XYZ .

[0126] Use the spatial model m of the marked ball and the three-dimensional information marker of the marked ball obtained by the tracking head XYZ to splice and obtain the transposed relationship RT between the world coordinate systems of m and the tracking head n . The three-dimensional points of the object to be scanned obtained by the scanning head pass through RT m and RT n 's transpose and then become the three-dimensional point information world in the world coordinate system XYZ . RT m and RT n are both 4×4 matrices. The calculation formula is world XYZ = object XYZ ×RT m ×RT n .

[0127] Repeat the above steps until the surface of the object to be scanned is scanned completely, and the scanning of the object to be scanned is completed.

[0128] After the scanning is completed, post-processing and other operations are performed on the terminal device.

[0129] Figure 8 Fig. shows the system framework diagram of a handheld scanner provided by an embodiment of the present application. As Figure 8 shown, the system includes: a terminal device, a handheld scanner, and a device to be scanned.

[0130] The handheld scanner is equivalent to the scanner in step S601, and the device to be scanned is equivalent to the target in the above text.

[0131] The first step is calibration.

[0132] Obtain the internal and external parameters of the binocular camera.

[0133] The second step is scanning.

[0134] Fix the marked ball on the surface of the object, obtain the marked ball information and the surface information of the object to be scanned through the handheld scanner, extract the two-dimensional information of the marked ball and the surface of the object to be scanned, and obtain the three-dimensional information marker of the marked ball through reconstruction XYZ and the three-dimensional information object of the object to be scanned XYZ .

[0135] Taking the three-dimensional information marker of the marked ball of the first-frame data XYZ as the reference coordinate system benchmark XYZ , the three-dimensional information marker of the marked ball obtained subsequently XYZ is spliced onto the reference coordinate system to obtain the transposed matrix RTm. Multiply the obtained three-dimensional information object of the surface of the object to be scanned XYZ by RT mIt is back-transposed to the same coordinate system as the first frame to achieve the reconstruction of the object surface. The calculation formula is:

[0136] benchmark XYZ = object XYZ × RT m .

[0137] In summary, the method provided by the embodiment of the present application uses the isotropic characteristics of the sphere in space, uses a marker sphere instead of a marker point for recognition and pose determination, collects pictures of the marker sphere through a binocular camera, uses an edge detection operator to extract the edge of the marker sphere in the picture, recognizes the two-dimensional ellipse information of the marker sphere in the image, and calculates the three-dimensional information of the marker sphere in space based on the ellipse information by using a binocular reconstruction algorithm. The stitching of the surface information of the object to be scanned is realized through the three-dimensional information. The problem of the anisotropy of the planar marker points in the three-dimensional space is solved.

[0138] Figure 9 The structural block diagram of a three-dimensional scanning device using a marker sphere for station transfer provided by the embodiment of the present application is shown. As Figure 9 shown, the device includes:

[0139] A scanning module 91, configured to scan a target through a scanner to obtain first three-dimensional information of the target in the scanner coordinate system; wherein, a plurality of marker spheres are provided on the scanner.

[0140] A tracking module 92, configured to synchronously track the scanner through a tracker when the scanner scans the target, to obtain second three-dimensional information of a plurality of marker spheres in the tracker coordinate system.

[0141] A first conversion module 93, configured to model a plurality of marker spheres based on the spatial position relationship between the plurality of marker spheres and the scanner, to obtain three-dimensional models of the plurality of marker spheres, and to determine a first conversion matrix for converting the model coordinate system used for modeling to the scanner coordinate system.

[0142] A second conversion module 94, configured to determine a second conversion matrix for converting the model coordinate system to the tracker coordinate system based on the three-dimensional models of the plurality of marker spheres and the second three-dimensional information.

[0143] A reconstruction module 95, configured to convert the first three-dimensional information to the tracker coordinate system based on the first conversion matrix and the second conversion matrix, to obtain third three-dimensional information, and the third three-dimensional information is used for three-dimensional reconstruction of the target.

[0144] In some embodiments, the tracking module 92 is further configured to synchronously track the scanner through a plurality of cameras of the tracker to obtain a first image of the scanner collected by each camera;

[0145] Based on the first images collected by each camera, determine the two-dimensional ellipse information of each of the multiple marker balls;

[0146] Based on the two-dimensional ellipse information of each of the multiple marker balls, construct the second three-dimensional information of the multiple marker balls in the tracker coordinate system.

[0147] In some embodiments, the tracking module 92 is further configured to obtain, from the first images collected by each camera, a second image corresponding to each marker ball;

[0148] Determine the centroid of the marker ball as the center of the second image;

[0149] Based on the centroid, determine the major axis and minor axis of the marker ball in the second image.

[0150] In some embodiments, the tracking module 92 is further configured to obtain the edge information of each marker ball in the first image using an image edge extraction operator;

[0151] Based on the edge information of each marker ball, crop out the second image from the first image.

[0152] It should be noted that the above-mentioned various modules can be functional modules or program modules, and can be implemented either by software or by hardware. For the modules implemented by hardware, the above-mentioned various modules can be located in the same processor; or the above-mentioned various modules can also be located in different processors in any combined form.

[0153] It should be noted that the specific examples in this embodiment can refer to the examples described in the above-mentioned embodiments and optional implementation manners, and will not be elaborated in this embodiment.

[0154] In addition, in combination with the method provided in the above-mentioned embodiments, a storage medium can also be provided in this embodiment to implement it. A computer program is stored on the storage medium; when the computer program is executed by a processor, the three-dimensional scanning method using a marker ball transfer station in any of the above-mentioned embodiments is implemented.

[0155] The embodiment of the present application also provides a computer program product, when the computer program product runs on a computer, it causes the computer to execute each function or step executed by the processor in the above-mentioned method embodiments.

[0156] It should be understood that the specific embodiments described here are only used to explain this application, rather than to limit it. According to the embodiments provided by the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0157] Obviously, the accompanying drawings are only some examples or embodiments of the present application. For those of ordinary skill in the art, the present application can also be applied to other similar situations based on these drawings without creative efforts. Additionally, it can be understood that although the work done during this development process may be complex and time-consuming, for those of ordinary skill in the art, certain design, manufacturing, or production changes based on the technical content disclosed in the present application are only conventional technical means and should not be regarded as insufficient disclosure of the present application.

[0158] The term "embodiment" in the present application means that the specific features, structures, or characteristics described in connection with the embodiments can be included in at least one embodiment of the present application. The phrase appears in various positions in the specification and does not necessarily mean the same embodiment, nor does it mean being independent or alternative to other embodiments and mutually exclusive. Those of ordinary skill in the art can clearly or implicitly understand that the embodiments described in the present application can be combined with other embodiments without conflict.

[0159] The above-described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of patent protection. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A three-dimensional scanning method using a marker ball transfer station, characterized in that: The method comprises: Scanning the target by means of a scanner to obtain first three-dimensional information of the target in a scanner coordinate system; wherein the scanner is provided with a plurality of marking balls; When the scanner scans the target, the scanner is synchronously tracked by a tracker to obtain second three-dimensional information of the multiple marker balls in a tracker coordinate system; Based on the spatial position relationship between the multiple marker balls and the scanner, the multiple marker balls are modeled to obtain a three-dimensional model of the multiple marker balls, and a first transformation matrix is ​​determined for transforming a model coordinate system used for modeling into the scanner coordinate system; Determine a second transformation matrix for transforming the model coordinate system into the tracker coordinate system based on the three-dimensional model of the plurality of marker spheres and the second three-dimensional information; Based on the first transformation matrix and the second transformation matrix, the first three-dimensional information is transformed into the tracker coordinate system to obtain third three-dimensional information, and the third three-dimensional information is used to perform three-dimensional reconstruction of the target.

2. The three-dimensional scanning method using a marker ball transfer station according to claim 1, characterized in that: The tracker includes a plurality of cameras, and the step of synchronously tracking the scanner by the tracker to obtain second three-dimensional information of the plurality of marker balls in the tracker coordinate system includes: Synchronously tracking the scanner by using multiple cameras of a tracker to obtain a first image of the scanner captured by each camera; Determine two-dimensional ellipse information of each marker sphere among the plurality of marker spheres based on the first image captured by each of the cameras; Based on the two-dimensional ellipse information of each marker sphere in the multiple marker spheres, second three-dimensional information of the multiple marker spheres in the tracker coordinate system is constructed.

3. The three-dimensional scanning method using a marker ball transfer station according to claim 2, characterized in that: The two-dimensional ellipse information includes the centroid, major axis, and minor axis of the marker sphere, and determining the two-dimensional ellipse information of each marker sphere among the plurality of marker spheres based on the first image acquired by each of the cameras includes: Acquire a second image corresponding to each marker ball from the first image captured by each camera; Determining the center of the second image as the center of mass of the marker sphere; The major axis and the minor axis of the marker sphere are determined in the second image.

4. The three-dimensional scanning method using a marker ball transfer station according to claim 3, characterized in that: The step of acquiring a second image corresponding to each marker ball from the first image acquired by each camera includes: Obtain edge information of each marked ball in the first image; The second image is cropped from the first image based on edge information of each marker sphere.

5. The three-dimensional scanning method using a marker ball transfer station according to claim 3, characterized in that: The first transformation matrix is ​​RT m Indicates that the second transformation matrix is ​​RT n Indicates that the first three-dimensional information is in the form of object XYZ Indicates that the third three-dimensional information is in the form of world XYZ express; The third three-dimensional information is obtained by using the first relationship: world XYZ =object XYZ ×RT m ×RT n 。 6. A three-dimensional scanning method using a marker ball transfer station, characterized in that: The method comprises: Scanning the target by means of a scanner to obtain first three-dimensional information of the target in a scanner coordinate system; wherein a plurality of marker balls are arranged on the target; Based on the spatial position relationship between the multiple marker balls and the target, the multiple marker balls are modeled to obtain a three-dimensional model of the multiple marker balls, and a first transformation matrix for transforming a model coordinate system used for modeling into the scanner coordinate system is determined; Based on the first transformation matrix, the first three-dimensional information is transformed into the scanner coordinate system to obtain fourth three-dimensional information, and the fourth three-dimensional information is used to perform three-dimensional reconstruction of the target.

7. The three-dimensional scanning method using a marker ball transfer station according to claim 6, characterized in that: The first transformation matrix is ​​RT m Indicates that the fourth three-dimensional information is benchmark XYZ Indicates that the first three-dimensional information is in the form of object XYZ express; The fourth three-dimensional information is obtained by using the second relationship: benchmark XYZ =object XYZ ×RT m 。 8. A three-dimensional scanning device using a marker ball transfer station, characterized in that: The device comprises: A scanning module, used for scanning a target through a scanner to obtain first three-dimensional information of the target in a scanner coordinate system; wherein a plurality of marking balls are provided on the scanner; A tracking module, configured to synchronously track the scanner through a tracker when the scanner scans the target, and obtain second three-dimensional information of the multiple marker balls in a tracker coordinate system; A first conversion module is used to model the multiple marker balls based on the spatial position relationship between the multiple marker balls and the scanner to obtain a three-dimensional model of the multiple marker balls, and determine a first conversion matrix for converting a model coordinate system used for modeling to the scanner coordinate system; A second conversion module, configured to determine a second conversion matrix for converting the model coordinate system into the tracker coordinate system based on the three-dimensional model of the plurality of marker balls and the second three-dimensional information; A reconstruction module is used to transform the first three-dimensional information into the tracker coordinate system based on the first transformation matrix and the second transformation matrix to obtain third three-dimensional information, and the third three-dimensional information is used to perform three-dimensional reconstruction of the target.

9. A computer device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to execute the three-dimensional scanning method using a marker ball transfer station as described in any one of claims 1 to 6, or to execute the three-dimensional scanning method using a marker ball transfer station as described in any one of claims 7 or 8.

10. A storage medium, characterized in that: The storage medium stores a computer program, wherein the computer program is configured to execute, when running, the three-dimensional scanning method using a marker ball transfer station as described in any one of claims 1 to 6, or the three-dimensional scanning method using a marker ball transfer station as described in any one of claims 7 or 8.

Images