Scanning method, electronic device and computer-readable storage medium

Abstract

Provided in the embodiments of the present application are a scanning method, an electronic device and a computer-readable storage medium. The method is applied to a non-rigid body scanning scene, the scanning scene being provided with a target, and the target having an identification point. The method comprises: acquiring three-dimensional data of a current frame and three-dimensional data of a previous frame of a scanning scene; on the basis of the three-dimensional data of the current frame and the three-dimensional data of the previous frame, splicing three-dimensional contour data of the current frame in the three-dimensional data of the current frame and three-dimensional contour data of the previous frame in the three-dimensional data of the previous frame; on the basis of the spliced three-dimensional contour data of the current frame and three-dimensional contour data of the previous frame, determining the matching degree between the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame; and when the matching degree satisfies a matching degree condition, acquiring a three-dimensional digital model of the scanning scene on the basis of the spliced three-dimensional contour data of the current frame and three-dimensional contour data of the previous frame. The embodiments are beneficial to improving the accuracy of splicing data and obtaining three-dimensional digital models.

Description

Scanning methods, electronic devices and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202410670945.8, filed on May 28, 2024, entitled "Scanning Method, Electronic Device and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of scanning technology, specifically to a scanning method, an electronic device, and a computer-readable storage medium. Background Technology

[0003] Scanning can produce a three-dimensional digital model of the object being scanned. For example, scanning intraoral structures can produce a three-dimensional digital model of the gingiva. The gingiva is soft tissue and easily deformed during scanning. In edentulous implant procedures and large-span implants, the intraoral scanner's window cannot always cover both scanning poles. If the gingiva deforms during scanning, using gingival features as a splicing method will lead to larger splicing errors. The final prosthesis will be based on the deformed gingival data for CAD (Computer-Aided Design), which may cause the prosthesis to compress the gingiva or create gaps when placed in the mouth. If the deformation exceeds a certain limit, it may cause discomfort to the patient or, in severe cases, lead to implant failure.

[0004] Therefore, how to improve the stitching accuracy and obtain a high-precision 3D digital model is an urgent problem to be solved. Summary of the Invention

[0005] This application provides a scanning method, an electronic device, and a computer-readable storage medium, which solves the problem that existing scanning methods are prone to deformation when scanning non-rigid bodies, leading to stitching errors and affecting the accuracy of the obtained three-dimensional digital model.

[0006] To achieve the above objectives, this application adopts the following technical solution.

[0007] In a first aspect, embodiments of this application provide a scanning method applied to a non-rigid body scanning scenario. The scanning scenario includes a target with marker points. The scanning method includes: acquiring current frame 3D data and previous frame 3D data of the scanning scenario, wherein the current frame 3D data of the scanning scenario includes current frame 3D contour data and current frame 3D marker point data, and the previous frame 3D data of the scanning scenario includes previous frame 3D contour data and previous frame 3D marker point data; stitching the current frame 3D contour data and the previous frame 3D contour data together based on the current frame 3D data and the previous frame 3D contour data; determining the matching degree between the current frame 3D contour data and the previous frame 3D contour data based on the stitched current frame 3D contour data and the previous frame 3D contour data; and acquiring a 3D digital model of the scanning scenario based on the stitched current frame 3D contour data and the previous frame 3D contour data when the matching degree meets the matching degree condition.

[0008] In the above scheme, the scanning scene includes non-rigid bodies. In order to improve the stitching accuracy, a target is set in the scanning scene. The target has marker points. Since the target is rigid, the relative positions between the marker points on the target do not change. This helps to improve the accuracy of stitching based on the 3D contour data of the marker points in the scanning scene and the 3D contour data of the scanning scene. As a result, the 3D digital model obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data is also more accurate.

[0009] In one possible implementation, obtaining the current frame 3D data of the scanned scene includes: obtaining the current frame encoded reconstructed image and the current frame marker point image of the scanned scene; reconstructing the current frame 3D contour data of the scanned scene based on the current frame encoded reconstructed image, wherein the current frame 3D contour data of the scanned scene includes the current frame 3D contour data of a non-rigid body; and reconstructing the current frame 3D marker point data based on the current frame marker point image.

[0010] In the above scheme, the determination of the three-dimensional contour data and the three-dimensional marker point data of the current frame is limited. Specifically, the three-dimensional contour data of the current frame of the scanned scene is reconstructed based on the encoded image of the current frame. The three-dimensional contour data of the current frame includes the three-dimensional contour data of the current frame that is not rigid. The three-dimensional marker point data of the current frame is reconstructed based on the marker point image of the current frame.

[0011] In one possible implementation, the current frame's 3D contour data and the previous frame's 3D contour data are stitched together based on the current frame's 3D data and the previous frame's 3D data, including:

[0012] The current frame's 3D marker point data and the previous frame's 3D marker point data are stitched together to obtain a stitching matrix; based on the stitching matrix, the current frame's 3D contour data and the previous frame's 3D contour data are stitched together.

[0013] In the above scheme, a stitching matrix is ​​obtained from the 3D data of the marker points, and then the 3D contour data of the current frame and the 3D contour data of the previous frame of the scanned scene are processed based on the stitching matrix.

[0014] In one possible implementation, the current frame's 3D contour data and the previous frame's 3D contour data are stitched together based on the current frame's 3D data and the previous frame's 3D contour data. This includes: stitching together the current frame's 3D marker point data and the previous frame's 3D marker point data; if stitching is successful, obtaining a stitching matrix, and stitching the current frame's 3D contour data and the previous frame's 3D contour data based on the stitching matrix; if stitching fails, performing feature stitching on the current frame's 3D contour data and the previous frame's 3D contour data. It should be noted that stitching failure can refer to situations where the current frame's 3D data does not include the current frame's 3D marker point data, resulting in the acquired image potentially containing images that do not include marker points.

[0015] In the above scheme, when performing 3D data stitching, the stitching matrix obtained by the marker points is used first. When stitching based on the marker points fails, feature stitching is used when stitching the 3D data contour.

[0016] In one possible implementation, the current frame's 3D marker point data and the previous frame's 3D marker point data are concatenated to obtain a concatenation matrix, including: determining the same-name marker points in the current frame's 3D marker point data and the previous frame's 3D marker point data; and concatenating the current frame's 3D marker point data and the previous frame's 3D marker point data based on the same-name marker points to obtain a concatenation matrix.

[0017] In the above scheme, the method for obtaining the splicing matrix is ​​limited. It is based on splicing the 3D data of the same-named marker points to obtain the splicing matrix from the 3D marker point data of the current frame to the 3D marker point data of the previous frame.

[0018] In one possible implementation, the matching degree between the current frame 3D contour data and the previous frame 3D contour data of the stitched scan scene is determined; when the matching degree meets the matching degree condition, a 3D digital model of the scan scene is obtained based on the current frame 3D contour data and the previous frame 3D contour data of the stitched scan scene, including: determining the overlapping area between the current frame 3D contour data and the previous frame 3D contour data of the stitched scan scene; when the matching degree of the overlapping area meets the matching degree condition, a 3D digital model of the scan scene is obtained based on the current frame 3D contour data and the previous frame 3D contour data of the stitched scan scene; when the matching degree of the overlapping area does not meet the matching degree condition, the current frame 3D contour data of the scan scene is discarded.

[0019] In one possible implementation, when the matching degree of the overlapping regions meets the matching degree condition, a three-dimensional digital model of the scanned scene is obtained based on the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame of the scanned scene after stitching. This includes: stitching the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame of the scanned scene based on the three-dimensional contour data of the overlapping regions in the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame, as well as the three-dimensional data of the same-named marker points in the three-dimensional marker point data of the current frame and the three-dimensional marker point data of the previous frame, to obtain an optimized stitching matrix; and stitching the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame of the scanned scene based on the optimized stitching matrix to obtain a three-dimensional digital model of the scanned scene.

[0020] In the above scheme, the matching degree of the overlapping regions meets the matching degree condition, that is, when the matching degree meets the matching degree threshold, the method of obtaining the optimization matrix and the method of obtaining the three-dimensional digital model of the scanning scene based on the stitched three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame are described.

[0021] In one possible implementation, the scanning method further includes: after each N frames are stitched together, verifying the most recently stitched N frames; and if the deviation value obtained from the verification is greater than a threshold, optimizing the stitching of the most recently stitched N frames.

[0022] In the above scheme, the scanning method has been further improved. After every N frames are completed, the most recently stitched N frames are checked. If the deviation is greater than the threshold, the most recently stitched N frames are optimized.

[0023] In a second aspect, this application provides an electronic device, including: a memory and one or more processors, the memory being coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method provided in the first aspect and any possible implementation thereof.

[0024] Thirdly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method provided in the first aspect and any possible implementation thereof.

[0025] Fourthly, this application provides a computer program product. When the computer program product is run on an electronic device, it causes the electronic device to perform the method provided by the first aspect and any possible implementation thereof.

[0026] Fifthly, this application provides a chip system. When the chip system is applied to an electronic device, the chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform the methods provided in the first aspect and any possible implementation thereof.

[0027] It is understood that the beneficial effects achieved by the electronic device of the second aspect, the computer-readable storage medium of the third aspect, the computer program product of the fourth aspect, and the chip system of the fifth aspect provided above can be referred to as the beneficial effects of the first aspect, and will not be repeated here. Attached Figure Description

[0028] Figure 1 is a schematic diagram of an application scenario of the scanning method provided in an embodiment of this application;

[0029] Figure 2 is a flowchart illustrating a scanning method provided in an embodiment of this application;

[0030] Figure 3 is a flowchart illustrating a scanning method provided in an embodiment of this application;

[0031] Figure 4A is a schematic flowchart of a scanning method provided in an embodiment of this application;

[0032] Figure 4B is a schematic flowchart of a scanning method provided in an embodiment of this application;

[0033] Figure 5 is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0034] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "multiple" refers to two or more than two.

[0035] Hereinafter, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first," "second," or "third" may explicitly or implicitly include one or more of that feature.

[0036] Figure 1 is a schematic diagram of an application scenario for the scanning method provided in this embodiment. In this embodiment, the scanning method can be used to scan an oral cavity equipped with a scanning rod based on an intraoral scanner to obtain a three-dimensional digital model of the gingiva. The scanning rod has marker points or coded points on its surface and is fixedly mounted on the implant, typically connected to the implant via an abutment. To obtain an accurate three-dimensional digital model of the gingiva, this embodiment employs the scanning method shown in Figure 2.

[0037] In some possible embodiments, the scanning method is applied to scanning scenarios including non-rigid bodies (such as gums), the scanning scenario having a target (as shown in Figure 1, a scanning rod placed on the gums), the target having marker points (as shown in Figure 1, white dots on the scanning rod, which can be marker points or coded points). The scanning method includes steps 201 to 205 as shown in Figure 2.

[0038] Step 201. Obtain the current frame 3D data and the previous frame 3D data of the scanned scene.

[0039] The current frame 3D data of the scanned scene includes: current frame 3D contour data and current frame 3D marker point data. The previous frame 3D data of the scanned scene includes: previous frame 3D contour data and previous frame 3D marker point data.

[0040] In some possible implementations, structured light and white light can be emitted sequentially and alternately in the scanning scene, typically projected periodically. The current frame's encoded reconstructed image of the scanning scene is obtained using structured light, while the current frame's marker point image is obtained using white light. The encoded reconstructed image and marker point image obtained based on the same period of structured light and white light can be considered as a single frame. It should be noted that, since the scanning pole cannot cover the entire scanning scene, during the scanning process, the current frame's marker point image obtained using white light may sometimes fail to include marker points.

[0041] The current frame 3D contour data of the scanned scene is reconstructed based on the current frame encoded image. The current frame 3D contour data of the scanned scene includes the current frame 3D contour data of non-rigid bodies (such as the 3D contour data of reconstructed gingiva), and the current frame 3D marker point data (such as the white dots on the scanning rod in Figure 1) is reconstructed based on the current frame marker point image.

[0042] Step 202. Based on the current frame 3D data and the previous frame 3D data of the scanned scene, stitch together the current frame 3D contour data and the previous frame 3D contour data of the scanned scene.

[0043] Step 203. Based on the stitched current frame 3D contour data and the previous frame 3D contour data, determine the matching degree between the current frame 3D contour data and the previous frame 3D contour data.

[0044] In some possible implementations, the matching degree between the current frame 3D contour data and the previous frame 3D contour data is determined based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: determining the overlapping area between the stitched current frame 3D contour data and the previous frame 3D contour data, and determining the matching degree based on the overlapping area.

[0045] Step 204. When the matching degree meets the matching degree condition, obtain the three-dimensional digital model of the scanned scene based on the stitched three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame.

[0046] In some possible implementations, the matching degree of the overlapping region is determined based on the overlapping area, and this matching degree is used as the matching degree between the stitched current frame 3D contour data and the previous frame 3D contour data. When the matching degree of the overlapping region meets the matching degree condition, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. For example, if the matching degree threshold is 80%, then when the matching degree is greater than 80%, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data.

[0047] In some possible implementations, the current frame's 3D contour data of the scanned scene is discarded when the matching degree of the overlapping region does not meet the matching degree condition. For example, if the matching degree threshold is 80%, then when the matching degree is less than 80%, it is determined that the matching degree of the overlapping region does not meet the matching degree condition, and the current frame's 3D contour data of the scanned scene is discarded.

[0048] In some possible implementations, the matching degree includes the overlap rate. The overlap rate of the overlapping region can be determined by the overlap rate of the 3D points in the overlapping region. Both the current frame's 3D contour data and the previous frame's 3D contour data include 3D points. It should also be noted that overlap can be complete overlap or overlap that meets an overlap threshold.

[0049] In some possible implementations, when the matching degree of the overlapping regions meets the matching degree condition, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: stitching the current frame 3D contour data and the previous frame 3D contour data together based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and the 3D data of the corresponding marker points in the current frame 3D marker point data and the previous frame 3D marker point data, to obtain an optimized stitching matrix. Specifically, a first stitching matrix can be determined based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and a second stitching matrix can be determined based on the 3D marker point data of the current frame 3D marker point data and the previous frame 3D marker point data. A first weight is set for the first stitching matrix, and a second weight is set for the second stitching matrix. Then, an optimized stitching matrix is ​​determined based on the first weight, the first stitching matrix, the second weight, and the second stitching matrix. It should be noted that the first weight and the second weight can be set based on experience. A 3D digital model of the scanning scene is obtained by stitching together the current frame's 3D contour data and the previous frame's 3D contour data using an optimized stitching matrix.

[0050] The technical solution in this embodiment includes a non-rigid body in the scanning scene. To improve the stitching accuracy, a target is set in the scanning scene with marker points. Since the target is rigid, the relative positions between the marker points on the target remain unchanged. The stitching of the three-dimensional data of the preceding and following frames based on the marker points in the scanning scene helps to improve the accuracy of the stitching of the three-dimensional data of the preceding and following frames in the scanning scene, thereby improving the accuracy of the stitching of the three-dimensional contour data of the non-rigid body in the scanning scene. That is, the three-dimensional digital model obtained based on the stitched current frame three-dimensional contour data and the previous frame three-dimensional contour data is also more accurate.

[0051] As shown in Figure 3, in some possible embodiments, the scanning method includes steps 301 to 306 shown in Figure 3. The scanning method is applied to non-rigid body (such as gums) scanning scenarios, where a target (a scanning rod placed on the gums as shown in Figure 1) is provided, and the target has marker points (white dots on the scanning rod as shown in Figure 1, which can be marker points or coded points).

[0052] 301. Obtain the current frame 3D data of the scanned scene, wherein the current frame 3D data of the scanned scene includes the current frame 3D contour data and the current frame 3D marker point data of the scanned scene.

[0053] In some possible implementations, structured light and white light can be emitted sequentially and alternately in the scanning scene, typically in a periodic manner. The current frame encoded and reconstructed image of the scanning scene is obtained through structured light, and the current frame marker image is obtained through white light; the encoded and reconstructed image and marker image obtained based on structured light and white light of the same period can be used as a single frame image.

[0054] The current frame 3D contour data of the scanned scene is reconstructed based on the current frame encoded image. The current frame 3D contour data of the scanned scene includes the current frame 3D contour data of non-rigid bodies (such as the 3D contour data of reconstructed gingiva), and the current frame 3D marker point data (such as the white dots on the scanning rod in Figure 1) is reconstructed based on the current frame marker point image.

[0055] 302. Obtain the preceding frame 3D data of the scanned scene, wherein the preceding frame 3D data of the scanned scene includes the preceding frame 3D contour data and the preceding frame 3D marker point data of the scanned scene.

[0056] 303. The current frame's 3D marker point data and the previous frame's 3D marker point data are spliced ​​together to obtain a splicing matrix.

[0057] 304. Based on the splicing matrix, splice the current frame's 3D contour data and the previous frame's 3D contour data.

[0058] 305. Determine the matching degree between the current frame's 3D contour data and the previous frame's 3D contour data based on the stitched current frame's 3D contour data and the previous frame's 3D contour data.

[0059] In some possible implementations, the matching degree between the current frame 3D contour data and the previous frame 3D contour data is determined based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: determining the overlapping area between the stitched current frame 3D contour data and the previous frame 3D contour data, and determining the matching degree based on the overlapping area.

[0060] 306. When the matching degree meets the matching degree condition, obtain the three-dimensional digital model of the scanned scene based on the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame after stitching.

[0061] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition. That is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. For example, if the matching degree threshold is 80%, then when the matching degree is greater than 80%, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data.

[0062] In some possible implementations, the current frame's 3D contour data of the scanned scene is discarded when the matching degree of the overlapping region does not meet the matching degree condition. For example, if the matching degree threshold is 80%, then when the matching degree is less than 80%, it is determined that the matching degree of the overlapping region does not meet the matching degree condition, and the current frame's 3D contour data of the scanned scene is discarded.

[0063] In some possible implementations, the matching degree includes the overlap rate. The overlap rate of the overlapping region can be determined by the overlap rate of the 3D points in the overlapping region. Both the current frame's 3D contour data and the previous frame's 3D contour data include 3D points. It should also be noted that overlap can be complete overlap or overlap that meets an overlap threshold.

[0064] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition, that is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: stitching the current frame 3D contour data and the previous frame 3D contour data together based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and the 3D data of the corresponding marker points in the current frame 3D marker point data and the previous frame 3D marker point data, to obtain an optimized stitching matrix. Specifically, a first stitching matrix can be determined based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and a second stitching matrix can be determined based on the 3D marker point data of the current frame 3D marker point data and the previous frame 3D marker point data. A first weight is set for the first stitching matrix, and a second weight is set for the second stitching matrix. Then, an optimized stitching matrix is ​​determined based on the first weight, the first stitching matrix, the second weight, and the second stitching matrix. It should be noted that the first weight and the second weight can be set based on experience. A 3D digital model of the scanning scene is obtained by stitching together the current frame's 3D contour data and the previous frame's 3D contour data using an optimized stitching matrix.

[0065] This embodiment obtains a stitching matrix from the 3D data of the marker points, and then stitches together the 3D contour data of the current frame and the 3D contour data of the previous frame of the scanned scene based on the stitching matrix.

[0066] As shown in Figure 4A, in some possible embodiments, the scanning method includes steps 401 to 408 shown in Figure 4A. The scanning method is applied to non-rigid body (such as gums) scanning scenarios, where a target (a scanning rod placed on the gums as shown in Figure 1) is provided, and the target has marker points (white dots on the scanning rod as shown in Figure 1).

[0067] 401. Obtain the current frame 3D data of the scanned scene, wherein the current frame 3D data of the scanned scene includes the current frame 3D contour data and the current frame 3D marker point data of the scanned scene.

[0068] In some possible implementations, structured light and white light can be emitted sequentially and alternately in the scanning scene, typically projected periodically. The current frame's coded reconstructed image of the scanning scene is obtained using structured light, while the current frame's marker image is obtained using white light. The coded reconstructed image and marker image obtained based on the same period of structured light and white light can be considered as one frame. For example, within one cycle, one frame of structured light is projected followed by one frame of white light; the resulting coded reconstructed image and marker image are then considered as one frame, and multiple frames are obtained in this cycle. Alternatively, within one cycle, two frames of structured light are projected followed by one frame of white light; the resulting two coded reconstructed images and marker image are then considered as one frame, and multiple frames are obtained in this cycle.

[0069] The current frame 3D contour data of the scanned scene is reconstructed based on the current frame encoded image. The current frame 3D contour data of the scanned scene includes the current frame 3D contour data of non-rigid bodies (such as the 3D contour data of reconstructed gingiva), and the current frame 3D marker point data (such as the white dots on the scanning rod in Figure 1) is reconstructed based on the current frame marker point image.

[0070] 402. Obtain the preceding frame 3D data of the scanned scene, wherein the preceding frame 3D data of the scanned scene includes the preceding frame 3D contour data and the preceding frame 3D marker point data of the scanned scene.

[0071] 403. The current frame's 3D marker point data and the previous frame's 3D marker point data are spliced ​​together to obtain a splicing matrix.

[0072] 404. Determine if the splicing was successful.

[0073] If the splicing is successful, proceed to step 405; if the splicing is unsuccessful, proceed to step 406.

[0074] 405. Obtain the stitching matrix, and stitch together the current frame's 3D contour data and the previous frame's 3D contour data based on the stitching matrix.

[0075] 406. Perform feature stitching on the current frame's 3D contour data and the previous frame's 3D contour data.

[0076] 407. Based on the stitched current frame 3D contour data and the previous frame 3D contour data, determine the matching degree between the current frame 3D contour data and the previous frame 3D contour data.

[0077] In some possible implementations, the matching degree between the current frame's 3D contour data and the previous frame's 3D contour data is determined based on the stitched current frame's 3D contour data and the previous frame's 3D contour data. This includes: determining the overlapping area between the current frame's 3D contour data and the previous frame's 3D contour data, and determining the matching degree based on the overlapping area.

[0078] 408. When the matching degree meets the matching degree condition, obtain the three-dimensional digital model of the scanned scene based on the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame after stitching.

[0079] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition. That is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. For example, if the matching degree threshold is 80%, then when the matching degree is greater than 80%, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data.

[0080] In some possible implementations, the current frame's 3D contour data of the scanned scene is discarded when the matching degree of the overlapping region does not meet the matching degree condition. For example, if the matching degree threshold is 80%, then when the matching degree is less than 80%, it is determined that the matching degree of the overlapping region does not meet the matching degree condition, and the current frame's 3D contour data of the scanned scene is discarded.

[0081] In some possible implementations, the matching degree includes the overlap rate. The overlap rate of the overlapping region can be determined by the overlap rate of the 3D points in the overlapping region. Both the current frame's 3D contour data and the previous frame's 3D contour data include 3D points. It should also be noted that overlap can be complete overlap or overlap that meets an overlap threshold.

[0082] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition, that is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data, including:

[0083] The first stitching matrix is ​​determined based on multi-frame 3D contour data of the scanned scene, and the second stitching matrix is ​​determined based on the 3D data of the same-named marker points in multi-frame 3D marker point data of the scanned scene.

[0084] Based on the first concatenation matrix, the second concatenation matrix, and their respective weights, obtain the optimized concatenation matrix;

[0085] A three-dimensional digital model is obtained by stitching together multiple frames of 3D contour data of a scanning scene using an optimized stitching matrix.

[0086] Based on the 3D contour data of the current frame and the previous frame of the scanned scene, which includes overlapping 3D contour data and 3D marker data of the same name from the current frame and the previous frame, the 3D contour data of the current frame and the previous frame are stitched together to obtain an optimized stitching matrix. Specifically, a first stitching matrix can be determined based on the overlapping 3D contour data of the current frame and the previous frame, and a second stitching matrix can be determined based on the 3D marker data of the current frame and the previous frame. A first weight is set for the first stitching matrix, and a second weight is set for the second stitching matrix. Then, an optimized stitching matrix is ​​determined based on the first weight, the first stitching matrix, the second weight, and the second stitching matrix. It should be noted that the first weight and the second weight can be set based on experience. A 3D digital model of the scanned scene is obtained by stitching the current frame and the previous frame 3D contour data of the scanned scene using the optimized stitching matrix.

[0087] In this embodiment, when performing 3D data stitching, the stitching matrix obtained from the marker points is used first. When stitching based on the marker points fails, feature stitching is used when stitching the 3D data contour, which helps to improve the stitching accuracy.

[0088] As shown in Figure 4B, in some possible embodiments, the scanning method includes steps 401 to 409 shown in Figure 4B. The scanning method is applied to non-rigid body (such as gums) scanning scenarios, where a target (a scanning rod placed on the gums as shown in Figure 1) is provided, and the target has marker points (white dots on the scanning rod as shown in Figure 1).

[0089] 401. Obtain the current frame 3D data of the scanned scene, wherein the current frame 3D data of the scanned scene includes the current frame 3D contour data and the current frame 3D marker point data of the scanned scene.

[0090] In some possible implementations, structured light and white light can be emitted sequentially and alternately in the scanning scene, typically in a periodic manner. The current frame encoded and reconstructed image of the scanning scene is obtained through structured light, and the current frame marker image is obtained through white light; the encoded and reconstructed image and marker image obtained based on structured light and white light of the same period can be used as a single frame image.

[0091] The current frame 3D contour data of the scanned scene is reconstructed based on the current frame encoded image. The current frame 3D contour data of the scanned scene includes the current frame 3D contour data of non-rigid bodies (such as the 3D contour data of reconstructed gingiva), and the current frame 3D marker point data (such as the white dots on the scanning rod in Figure 1) is reconstructed based on the current frame marker point image.

[0092] 402. Obtain the preceding frame 3D data of the scanned scene, wherein the preceding frame 3D data of the scanned scene includes the preceding frame 3D contour data and the preceding frame 3D marker point data of the scanned scene.

[0093] 403. The current frame's 3D marker point data and the previous frame's 3D marker point data are spliced ​​together to obtain a splicing matrix.

[0094] 404. Determine if the splicing was successful.

[0095] If the splicing is successful, proceed to step 405; if the splicing is unsuccessful, proceed to step 406.

[0096] 405. Obtain the stitching matrix, and stitch together the current frame's 3D contour data and the previous frame's 3D contour data based on the stitching matrix.

[0097] 406. Perform feature stitching on the current frame's 3D contour data and the previous frame's 3D contour data.

[0098] 407. Determine the matching degree between the current frame's 3D contour data and the previous frame's 3D contour data based on the stitched current frame's 3D contour data and the previous frame's 3D contour data.

[0099] In some possible implementations, the matching degree between the current frame 3D contour data and the previous frame 3D contour data is determined based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: determining the overlapping area between the stitched current frame 3D contour data and the previous frame 3D contour data, and determining the matching degree based on the overlapping area.

[0100] 408. When the matching degree meets the matching degree condition, obtain the three-dimensional digital model of the scanned scene based on the three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame after stitching.

[0101] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition. That is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched 3D contour data of the current frame and the 3D contour data of the previous frame. For example, if the matching degree threshold is 80%, then when the matching degree is greater than 80%, a 3D digital model of the scanned scene is obtained based on the stitched 3D contour data of the current frame and the 3D contour data of the previous frame.

[0102] In some possible implementations, the current frame's 3D contour data of the scanned scene is discarded when the matching degree of the overlapping region does not meet the matching degree condition. For example, if the matching degree threshold is 80%, then when the matching degree is less than 80%, it is determined that the matching degree of the overlapping region does not meet the matching degree condition, and the current frame's 3D contour data of the scanned scene is discarded.

[0103] In some possible implementations, the matching degree includes the overlap rate. The overlap rate of the overlapping region can be determined by the overlap rate of the 3D points in the overlapping region. Both the current frame's 3D contour data and the previous frame's 3D contour data include 3D points. It should also be noted that overlap can be complete overlap or overlap that meets an overlap threshold.

[0104] In some possible implementations, the matching degree of the overlapping regions satisfies a matching degree condition, that is, when the matching degree meets a matching degree threshold, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. This includes: stitching the current frame 3D contour data and the previous frame 3D contour data together based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and the 3D data of the corresponding marker points in the current frame 3D marker point data and the previous frame 3D marker point data, to obtain an optimized stitching matrix. Specifically, a first stitching matrix can be determined based on the 3D contour data of the overlapping regions in the current frame 3D contour data and the previous frame 3D contour data, and a second stitching matrix can be determined based on the 3D marker point data of the current frame 3D marker point data and the previous frame 3D marker point data. A first weight is set for the first stitching matrix, and a second weight is set for the second stitching matrix. Then, an optimized stitching matrix is ​​determined based on the first weight, the first stitching matrix, the second weight, and the second stitching matrix. It should be noted that the first weight and the second weight can be set based on experience. A 3D digital model of the scanning scene is obtained by stitching together the current frame's 3D contour data and the previous frame's 3D contour data using an optimized stitching matrix.

[0105] 409. After each N frames are stitched together, the most recently stitched N frames are checked. If the deviation is greater than the threshold, the most recently stitched N frames are optimized.

[0106] For example, in some possible implementations, when verifying N frames, the stitching matrix of the Nth frame relative to the first frame can be determined using the 3D data of the marker points of the first and Nth frames. If the angle relative to the stitching matrix calculated frame by frame is greater than a threshold, then the deviation is determined to be greater than the threshold, and stitching optimization is performed on these N frames. In some possible implementations, in the possible implementation of 408, if there is a case where the 3D contour data of a certain frame is discarded, and if the discarded frame was discarded during the N-frame calculation process mentioned above, then the discarded frame can be optimized during N-frame optimization. If the matching degree of the overlapping area meets the matching degree condition after optimization, then the discarded frame is added to the stitching process, and the 3D contour data corresponding to the discarded frame is stitched with the 3D contour data of the preceding frame. Generally, N is an integer greater than 2.

[0107] It is understandable that the most recently spliced ​​N frames are checked. If the deviation value obtained from the check is greater than the threshold, other methods can be used to optimize the splicing of the most recently spliced ​​N frames. This is not limited here.

[0108] In one embodiment, an electronic device is provided, which in some possible embodiments may be an intraoral scanner. Its internal structure may be as shown in Figure 5. The electronic device includes a memory and one or more processors connected via a bus, the memory being coupled to the processors; wherein the memory stores computer program code, which includes computer instructions, and when executed by the processor, causes the electronic device to perform the following methods:

[0109] Acquire the current frame 3D data and the previous frame 3D data of the scanned scene. The current frame 3D data of the scanned scene includes: the current frame 3D contour data and the current frame 3D marker point data. The previous frame 3D data of the scanned scene includes: the previous frame 3D contour data and the previous frame 3D marker point data.

[0110] Based on the current frame's 3D data and the previous frame's 3D data, stitch together the current frame's 3D contour data and the previous frame's 3D contour data;

[0111] Based on the stitched current frame 3D contour data and the previous frame 3D contour data, determine the matching degree between the current frame 3D contour data and the previous frame 3D contour data.

[0112] When the matching degree meets the matching degree condition, a three-dimensional digital model of the scanned scene is obtained based on the stitched three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame.

[0113] Those skilled in the art will understand that the structure shown in Figure 5 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific electronic devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0114] This application also provides a computer program product that, when executed by a processor, implements the interactive method of any of the method embodiments in this application.

[0115] This application also provides a computer-readable storage medium including instructions that, when executed on an electronic device, cause the electronic device to perform a scanning method as described in any of the preceding method embodiments. The specific steps of the scanning method are described in the preceding method embodiments and will not be repeated here.

[0116] This application also provides a chip coupled to a memory for reading and executing computer programs or instructions stored in the memory to perform the methods described in the above embodiments. This chip can be a general-purpose processor or a special-purpose processor.

[0117] The electronic devices, computer-readable storage media, computer program products, and chips provided in the embodiments of this application are all used to execute the methods provided in the above embodiments. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects corresponding to the methods provided above, and will not be repeated here.

[0118] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units. The replaced units may or may not be physically separate. The component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0119] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks. The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims. Industrial applicability

[0120] The solution provided in this application embodiment can be used for three-dimensional reconstruction. Specifically, it can be used to scan a scene of an oral cavity equipped with a scanning rod using an intraoral scanner to obtain a three-dimensional digital model of the gingiva. Since the scanning scene includes non-rigid bodies, a target with marker points is set in the scanning scene to improve stitching accuracy. Because the target is rigid, the relative positions of the marker points on the target remain unchanged. By acquiring the current frame's three-dimensional data and the previous frame's three-dimensional data of the scanning scene, and stitching the current frame's three-dimensional contour data and the previous frame's three-dimensional contour data based on the current frame's three-dimensional data and the previous frame's three-dimensional contour data, the accuracy of stitching the three-dimensional data of the previous and next frames of the scanning scene is improved, thereby enhancing the accuracy of stitching the three-dimensional contour data of the non-rigid bodies in the scanning scene.

[0121] Subsequently, based on the stitched current frame 3D contour data and the previous frame 3D contour data, the matching degree between the current frame 3D contour data and the previous frame 3D contour data is determined. When the matching degree meets the matching condition, a 3D digital model of the scanned scene is obtained based on the stitched current frame 3D contour data and the previous frame 3D contour data. The resulting 3D digital model is more accurate. Compared to related technologies, this solution improves the accuracy of data stitching and the obtained 3D digital model. It solves the problem that existing scanning methods easily cause deformation when scanning non-rigid bodies, leading to stitching errors and affecting the accuracy of the obtained 3D digital model, and has strong industrial applicability.

Claims

1. A scanning method applied to a non-rigid scanning scenario, the scanning scenario being provided with a target, the target having a marker point, wherein, The method includes: The current frame 3D data and the previous frame 3D data of the scanned scene are obtained. The current frame 3D data of the scanned scene includes: current frame 3D contour data and current frame 3D marker point data. The previous frame 3D data of the scanned scene includes: previous frame 3D contour data and previous frame 3D marker point data. Based on the current frame 3D data and the previous frame 3D data, the current frame 3D contour data and the previous frame 3D contour data are stitched together; Based on the stitched current frame 3D contour data and the previous frame 3D contour data, determine the matching degree between the current frame 3D contour data and the previous frame 3D contour data; When the matching degree meets the matching degree condition, a three-dimensional digital model of the scanned scene is obtained based on the stitched three-dimensional contour data of the current frame and the three-dimensional contour data of the previous frame.

2. The method according to claim 1, wherein, The acquisition of the current frame 3D data of the scanned scene includes: Obtain the current frame encoded reconstructed image and the current frame marker point image of the scanned scene; The current frame three-dimensional contour data of the scanned scene is reconstructed based on the current frame encoded image, and the current frame three-dimensional contour data of the scanned scene includes the current frame three-dimensional contour data of non-rigid bodies. The current frame's 3D marker point data is reconstructed based on the current frame's marker point image.

3. The method according to claim 1, wherein, The step of stitching together the current frame's 3D contour data and the previous frame's 3D contour data based on the current frame's 3D data and the previous frame's 3D data includes: The current frame's 3D marker point data and the previous frame's 3D marker point data are concatenated to obtain a concatenation matrix; Based on the stitching matrix, the current frame's 3D contour data and the previous frame's 3D contour data are stitched together.

4. The method according to claim 1, wherein, The step of stitching together the current frame's 3D contour data and the previous frame's 3D contour data based on the current frame's 3D data and the previous frame's 3D data includes: The current frame's 3D marker point data and the previous frame's 3D marker point data are stitched together. When the stitching is successful, the stitching matrix is ​​obtained, and the current frame's 3D contour data and the previous frame's 3D contour data are stitched together based on the stitching matrix. If stitching fails, feature stitching is performed on the current frame's 3D contour data and the previous frame's 3D contour data.

5. The method according to claim 3, wherein, The step of concatenating the current frame's 3D marker point data and the previous frame's 3D marker point data to obtain a concatenation matrix includes: Determine the corresponding marker points in the current frame's 3D marker point data and the previous frame's 3D marker point data; The current frame's 3D marker data and the previous frame's 3D marker data are spliced ​​together based on the same-named markers to obtain the splicing matrix.

6. The method according to any one of claims 1 to 5, wherein, The matching degree between the current frame 3D contour data and the previous frame 3D contour data of the stitched scanned scene is determined; when the matching degree meets the matching degree condition, a 3D digital model of the scanned scene is obtained based on the current frame 3D contour data and the previous frame 3D contour data of the stitched scanned scene, including: Determine the overlapping area between the current frame 3D contour data and the previous frame 3D contour data of the scanned scene after stitching; When the matching degree of the overlapping area meets the matching degree condition, a three-dimensional digital model of the scanning scene is obtained based on the current frame three-dimensional contour data and the previous frame three-dimensional contour data of the scanned scene after stitching. If the matching degree of the overlapping region does not meet the matching degree condition, the current frame 3D contour data of the scanned scene is discarded.

7. The method according to claim 6, wherein, When the matching degree in the overlapping region meets the matching degree condition, a three-dimensional digital model of the scanned scene is obtained based on the current frame's three-dimensional contour data and the previous frame's three-dimensional contour data of the stitched scanned scene, including: Based on the 3D contour data of the current frame and the 3D contour data of the previous frame of the scanning scene, the 3D contour data of the overlapping area, and the 3D contour data of the same-named marker points in the current frame and the 3D marker point data of the previous frame, the current frame and the 3D contour data of the scanning scene are stitched together to obtain an optimized stitching matrix. A three-dimensional digital model of the scanned scene is obtained by stitching together the current frame's three-dimensional contour data and the previous frame's three-dimensional contour data using an optimized stitching matrix.

8. The method according to claim 7, wherein, The method further includes: After each N frames are stitched together, the N most recently stitched frames are verified. If the deviation value obtained from the verification is greater than the threshold, the most recently spliced ​​N frames are spliced ​​and optimized.

9. An electronic device, wherein, include: A memory and one or more processors, the memory being coupled to the processors; wherein the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-8.

10. A computer-readable storage medium, wherein, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

Images