Scanning Data Processing Method, Apparatus, Electronic Device, and Medium

Abstract

The embodiments of the present disclosure relate to a scan data processing method, device, electronic device, and medium, the method including: acquiring a plurality of scan images including auxiliary feature points; processing the plurality of scan images to acquire three-dimensional coordinate points of the auxiliary feature points; and generating target scan data according to the three-dimensional coordinate points of the auxiliary feature points and point cloud data. According to the above technical solution, the accuracy of the positioning information of the scanning rod is improved, thereby improving the processing efficiency and accuracy of the scan data in the intraoral scanning scene.

Description

Technical Field

[0001] This disclosure claims priority to a Chinese patent application filed with the State Intellectual Property Office of China on May 2, 2022, with application number 2022104770842 and invention title "Scanning Data Processing Method, Apparatus, Electronic Device and Medium", the entire disclosure of which is incorporated herein by reference.

[0002] This disclosure relates to the field of intraoral scanning technology, and particularly to a scanning data processing method, apparatus, electronic device and medium.

Background Art

[0003] Generally, in a scanning scenario, relevant target positions are determined by scanning.

[0004] In the related art, since the scanning range is limited, when scanning data, a solution means of multi-data stitching is generally used, and cumulative errors occur, resulting in a low accuracy of the entire model finally.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The problem to be solved by this disclosure is to solve the problem that the accuracy of the entire model is not high in the conventional scanning scenario.

Means for Solving the Problems

[0006] To solve the above problems, embodiments of this disclosure include: a step of obtaining a plurality of scanning images including auxiliary feature points; a step of processing the plurality of scanning images to obtain three-dimensional coordinate points of the auxiliary feature points; a step of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points and point cloud data, and provides a scanning data processing method including the above steps. an image acquisition module used to obtain a plurality of scanning images including auxiliary feature points; An embodiment of the present disclosure provides an image processing module used to process the plurality of scanning images to obtain three-dimensional coordinate points of auxiliary feature points,

[0007] and a generation module used to generate target scanning data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data. Further provided is a scanning data processing apparatus including the above. An embodiment of the present disclosure provides an electronic device including a memory, a processor, and a computer program,

[0008] wherein the computer program is stored in the memory and configured to be executed by the processor to implement the above scanning data processing method. Further provided is an electronic device.

[0009] An embodiment of the present disclosure provides a computer-readable storage medium in which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above scanning data processing method.

Advantages of the Invention

[0010] The above technical solution according to the embodiment of the present disclosure has the following advantages compared with the prior art.

[0011] In the scanning data processing mode according to the embodiment of the present disclosure, a plurality of scanning images including auxiliary feature points are obtained, the plurality of scanning images are processed to obtain three-dimensional coordinate points of the auxiliary feature points, and target scanning data is generated based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data. According to the above technical solution, by improving the accuracy of the positioning information of the scanning rod, the processing efficiency and accuracy of the scanning data in the intraoral scanning scenario are improved.

[0012] It should be noted that the above general description and the following detailed description are merely exemplary and explanatory and cannot limit the present disclosure.

Brief Description of the Drawings

[0013] The accompanying drawings are incorporated into the specification, constitute a part of this specification, show embodiments applicable to the present disclosure, and are for explaining the principles of the present disclosure together with the specification.

[0014] Hereinafter, in order to more clearly explain the technical solutions in the embodiments of the present disclosure or in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0015] Hereinafter, in order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and comprehensively described. Of course, the described embodiments are some embodiments of the present disclosure, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present disclosure without creative labor are all included in the scope of the claims of the present disclosure.

[0016] In practical applications, for the restoration when teeth are missing, since the scanning range of oral scanning is limited, the solution of multi-data stitching is generally used when scanning oral data, and cumulative errors occur, so finally the accuracy of the entire model is reduced.

[0017] In view of the above problems, the present disclosure provides a scanning data processing method applicable to an application environment as shown in FIG. 1. FIG. 1 is a diagram showing an application scenario of scanning data processing according to an embodiment of the present disclosure. The application environment includes attaching a plurality of intraoral scanning rods to a target oral cavity. The intraoral scanning rod includes a scanning rod component 11 and an auxiliary component 12 connected to the scanning rod component 11. Auxiliary feature points are provided on the scanning rod component 11 and / or the auxiliary component 12. The scanning rod component 11 is adapted to an implant body attached to the target oral cavity. The scanning rod component 11 is attached in conformity with the implant body, whereby the intraoral scanning rod is attached to the target oral cavity.

[0018] Here, by making the auxiliary components 12 of any two of the plurality of intraoral scanning rods conform to each other, when any two of the intraoral scanning rods 10 are adjacently attached to the oral cavity, the auxiliary feature points on the two auxiliary components are continuously distributed. For example, the true value coordinate points of the auxiliary feature points can be obtained by a single-lens photography measurement system, a three-dimensional coordinate measuring instrument, a high-precision industrial three-dimensional scanner, etc. Theoretically, the three-dimensional coordinate points corresponding to the images obtained by scanning are in one-to-one correspondence with the true value coordinate points of the auxiliary feature points obtained in advance.

[0019] As an example of one scenario, a plurality of intraoral scanning rods are attached to a target oral cavity. The intraoral scanning rod includes a scanning rod component connected to the implant body and an auxiliary component Component connected to the scanning rod. Target features are provided on the intraoral scanning rod. The target features are continuously distributed on the scanning rod Component and / or the auxiliary component. The target features are not distributed only on one side of the scanning rod Component and / or the auxiliary component. It may be distributed on two or more surfaces, or it may be distributed on a non-planar surface such as the curved surface of the scanning rod component and / or the auxiliary component. No.

[0020] Specifically, an intraoral scanner scans the target oral cavity, acquires a plurality of frames of images, and transfers them to a data processing module for data processing. The data processing module Obtain images of multiple frames, and obtain initial 3D data of the target oral cavity based on the images of multiple frames. The initial 3D data includes an initial point set of the target oral cavity and 3D coordinate measurement values of target features in the same coordinate system. Obtain a preset model of the intraoral scanning rod. The preset model includes the true 3D coordinates of the target feature and the real point set of the intraoral scanning rod (true 3D coordinates of each point) in the same coordinate system. Perform stitching on the initial point set of the target oral cavity and the real point set of the intraoral scanning rod based on the correspondence between the 3D coordinate measurement values and the true values of the target features. Determine the positioning information of the intraoral scanning rod based on the stitched real point set of the intraoral scanning rod. The positioning information of the intraoral scanning rod is the positioning information of the implant body. Execute a method of designing a dental prosthesis based on the positioning information so that the designed and manufactured dental prosthesis can be fitted and attached to the implant body.

[0021] Specifically, by scanning the target oral cavity, obtain a plurality of scanning images including auxiliary feature points, process the plurality of scanning images to obtain 3D coordinate points of the auxiliary feature points, and generate target scanning data based on the 3D coordinate points of the auxiliary feature points and the point cloud data. According to the above technical solution, by improving the accuracy of the positioning information of the scanning rod, the processing efficiency and accuracy of the scanning data in the intraoral scanning scenario are improved. 。

[0022] Specifically, FIG. 2 is a flowchart of a scanning data processing method according to an embodiment of the present disclosure that can be executed by a scanning data processing apparatus. Here, the apparatus can be implemented using software and / or hardware, and can generally be integrated into an electronic device. As shown in FIG. 2, the method includes the following steps 101 to 103.

[0023] In step 101, obtain a plurality of scanning images including auxiliary feature points.

[0024] Here, the target oral cavity refers to the oral cavity that requires dental implant treatment, and it is necessary to scan the oral cavity to determine the implant position within the oral cavity. After the scanning rod is connected via the auxiliary feature body, intraoral scanning is performed.

[0025] In an embodiment of the present disclosure, the scanning rod includes an auxiliary feature body, and the shape of the auxiliary feature body may be various (for example, features such as spherical, quadrilateral, cuboid, conical, or combinations of these features).

[0026] In an embodiment of the present disclosure, the scanning rod is a feature object including auxiliary feature points. Here, an auxiliary feature point can uniquely identify one feature, that is, the scanning rod is provided with auxiliary feature points, and each auxiliary feature point can uniquely identify the position feature corresponding to the scanning rod. For example, target feature a and target feature b are respectively provided at position 1 and position 2 on the scanning rod, target feature a can uniquely identify the position feature of position 1 on the scanning rod, and target feature b can uniquely identify the position feature of position 2 on the scanning rod.

[0027] In addition, anything that can uniquely identify the position feature corresponding to the scanning rod, such as different shapes, colors, two-dimensional codes, etc. on the scanning rod, can all be used as auxiliary feature points.

[0028] For example, protruding or recessed spherical and square shapes are provided as auxiliary feature points on the scanning rod and its auxiliary feature body. Further, for example, different colors are printed on the scanning rod and its auxiliary feature body. Further, for example, different two-dimensional code patterns, circular or square shapes of different colors, etc. are printed on the scanning rod and its auxiliary feature body, and specifically, they are selected and provided based on application requirements.

[0029] In an embodiment of the present disclosure, after obtaining the scanning rod data and the auxiliary feature point data, it is necessary to align the designed scanning rod data and the real-time obtained scanning rod data by means of a stitching algorithm. Here, the scanning rod data is the data obtained by scanning the intraoral scanning rod, and the designed scanning rod data may be standard data or standard scanning rod data in the embodiments of the present application. When scanning the oral cavity, the single or multiple scanning rod data may have low accuracy because the oral cavity data has non-rigid characteristics. According to the aspects of the embodiments of the present disclosure, the overall accuracy can be improved by improving the alignment accuracy between the designed scanning rod data and the real-time scanning rod data.

[0030] Here, the target oral cavity can be scanned by a handheld oral scanner (single-lens or binocular camera), that is, a plurality of scanning images can be obtained by photographing. For example, dozens of scanning images can be collected per second and can be repeatedly collected.

[0031] Here, the auxiliary feature points refer to the points provided on the scanning rod body and / or the auxiliary feature body, and are set based on specific application scenarios.

[0032] In the embodiments of the present disclosure, there are various ways to scan the target oral cavity including the scanning rod to obtain a plurality of scanning images. In some embodiments, the camera rotates according to a certain direction and is controlled to scan the target oral cavity according to a certain frequency, so as to obtain a plurality of scanning images.

[0033] Specifically, after the scanning rod is connected to the target oral cavity, the target oral cavity including the scanning rod is scanned to obtain a plurality of scanning images.

[0034] In step 102, a plurality of scanning images are processed to obtain the three-dimensional coordinate points of the auxiliary feature points.

[0035] Here, the three-dimensional coordinate points of the auxiliary feature points refer to the three-dimensional coordinate points corresponding to the auxiliary feature points in the target oral cavity.

[0036] In the embodiments of the present disclosure, there are various ways to process a plurality of scanning images to obtain the three-dimensional coordinate points of the auxiliary feature points. In some embodiments, three-dimensional reconstruction is performed based on each scanning image to obtain an array of reconstructed diagrams, and the array of reconstructed diagrams is calculated to obtain the three-dimensional coordinate points of the auxiliary feature points and the point cloud data.

[0037] In other embodiments, the two-dimensional coordinate points of the auxiliary feature points in each scanned image are obtained, coordinate system conversion for the two-dimensional coordinate points is performed, the three-dimensional coordinate points of the auxiliary feature points in each scanned image are obtained, the three-dimensional coordinate points of the auxiliary feature points in each scanned image are stitched, and the three-dimensional coordinate points of all the auxiliary feature points and the point cloud data are obtained. The above two aspects are only examples of processing a plurality of scanned images to obtain the three-dimensional coordinate points of the auxiliary feature points, and the embodiments of the present disclosure do not limit the specific aspects of processing a plurality of scanned images to obtain the three-dimensional coordinate points of the auxiliary feature points.

[0038] In the embodiments of the present disclosure, after obtaining a plurality of scanned images, the plurality of scanned images can be processed to obtain the three-dimensional coordinate points of the auxiliary feature points.

[0039] In step 103, target scanned data is generated based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data. Here, the auxiliary feature points may be marker points.

[0040] Here, the three-dimensional coordinate points of the auxiliary feature points may be optimized three-dimensional coordinate points, which can more accurately represent the three-dimensional coordinate points of the auxiliary feature points.

[0041] In the embodiments of the present disclosure, the step of generating target scanned data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data may be understood as stitching based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data such as teeth and gums to obtain the target scanned data.

[0042] In the embodiments of the present disclosure, the step of generating target scanned data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data includes directly stitching the three-dimensional coordinate points of the auxiliary feature points and the point cloud data to obtain the target scanned data.

[0043] In the embodiments of the present disclosure, the step of generating target scanned data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data includes generating target scanned data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data.

[0044] Here, the standard data can be stitched with the three-dimensional coordinate points of the auxiliary feature points De - Data refers to scanning data obtained by pre-computer-aided design, which may be, for example, standard De - Data obtained by a single-lens reflex photography measurement system, a three-dimensional coordinate meter, a high-precision industrial three-dimensional scanner, etc. In addition, in order to improve processing efficiency, standard De - Data can be acquired in advance and stored in a database, and directly acquired when processing, or measured in real time according to the required scenario to obtain standard De - Data. In the scenario of intraoral scanning, the standard data may be standard scanning rod data.

[0045] Here, the target scanning data refers to converting the scanning rod data of computer-aided design into the same coordinate system as the three-dimensional coordinate points of the auxiliary feature points and replacing the scanning rod data scanned in real time.

[0046] In the embodiments of the present disclosure, there are various modes of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, point cloud data, and standard data. In some embodiments, the auxiliary feature points have corresponding true value coordinate points, the standard data is standard scanning rod data, and it is determined whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points. When stitching the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, a position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points is obtained, and based on the position transformation matrix, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points, and then the three-dimensional coordinate points of the auxiliary feature points and the point cloud data are stitched to obtain the target scanning data. Bu Whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points. When stitching the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, a position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points is obtained, and based on the position transformation matrix, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points, and then the three-dimensional coordinate points of the auxiliary feature points and the point cloud data are stitched to obtain the target scanning data.

[0047] In other embodiments, the standard data is standard scanning rod data. A plurality of planes are fitted based on the three-dimensional coordinate points of the auxiliary feature points, a target geometry is constructed based on the plurality of planes, at least three planes are obtained based on the standard scanning rod data, the normal vector of each plane and the intersection points of at least three planes are obtained, a position transformation matrix is obtained based on the normal vector of each plane and the intersection points of at least three planes, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then the target geometry is replaced and stitched with the point cloud data to obtain the target scanning data.

[0048] In some further embodiments, the auxiliary feature points have corresponding true value coordinate points, the standard data is standard scanning rod data, the three-dimensional coordinate points of any one auxiliary feature point and the true value coordinate points of the corresponding auxiliary feature point are stitched, the three-dimensional coordinate points of the target auxiliary feature points in the same coordinate system are obtained, a position transformation matrix between the three-dimensional coordinate points of the target auxiliary feature points and the true value coordinate points of the corresponding auxiliary feature points is obtained, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then the three-dimensional coordinate points of the auxiliary feature points and the point cloud data are stitched to obtain the target scanning data.

[0049] The above three aspects are merely examples of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data, and the embodiments of the present disclosure do not limit the specific aspects of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data.

[0050] Specifically, after obtaining the three-dimensional coordinate points of the auxiliary feature points, target scanning data can be generated based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data.

[0051] In the scanning data processing mode according to the embodiments of the present disclosure, a plurality of scanning images each including auxiliary feature points are acquired, the plurality of scanning images are processed to obtain three-dimensional coordinate points of the auxiliary feature points, and target scanning data is generated based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data. According to the above technical solution, by improving the alignment accuracy between the designed scanning rod data and the real-time scanning rod data, the processing efficiency and accuracy of the scanning data in the intraoral scanning scenario are improved.

[0052] Hereinafter, based on the description of the above embodiments, the scanning data processing methods in different scenarios will be described in detail with reference to FIGS. 3 and 4.

[0053] Specifically, FIG. 3 is a flowchart of another scanning data processing method according to the embodiments of the present disclosure. This embodiment further optimizes the above scanning data processing method based on the above embodiments. As shown in FIG. 3, the method includes the following steps 201 to 205.

[0054] In step 201, the target oral cavity is scanned to obtain a plurality of scanning images including auxiliary feature points.

[0055] Step 201 is the same as step 101. Specifically refer to the description of step 101, and the detailed description is omitted here.

[0056] In step 202, three-dimensional reconstruction is performed based on each scanning image to obtain an array of reconstructed diagrams, and the array of reconstructed diagrams is calculated to obtain three-dimensional coordinate points of the auxiliary feature points and point cloud data.

[0057] Specifically, the target oral cavity is scanned by intraoral scanning to collect scanning images (reconstructed diagrams and texture diagrams) for three-dimensional reconstruction. Using the array of reconstructed diagrams, three-dimensional data including teeth, gums, and the scanning rod is calculated, and the three-dimensional coordinate points of the auxiliary feature points are reconstructed using the array of texture diagrams. Since the reconstructed diagrams and the texture diagrams are acquired at the same time, it is considered that the three-dimensional data and the three-dimensional coordinate points of the auxiliary feature points correspond one-to-one in the same coordinate system.

[0058] In step 203, it is determined whether to stitch the three-dimensional coordinate points of the auxiliary feature points and Bu the true value coordinate points of the auxiliary feature points.

[0059] In step 204, when stitching the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, a position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points is obtained.

[0060] Specifically, auxiliary feature points are processed on the scanning rod body and its auxiliary feature body, the coordinates related to the auxiliary feature points are defined as the true value coordinate points of the auxiliary feature points, the three-dimensional coordinate points of the auxiliary feature points obtained during the scanning process are matched with the true value coordinate points of the auxiliary feature points, and when it is found that the matching is successful, the scanning rod data designed corresponding to the current scanning rod can be replaced.

[0061] In the embodiments of the present disclosure, a first auxiliary feature point distance between the three-dimensional coordinate points of the auxiliary feature points and the three-dimensional coordinate points of the auxiliary feature points within the standard distance range is obtained, the true value coordinate points of the auxiliary feature points that match the three-dimensional coordinate points of the auxiliary feature points are obtained, and the true value coordinate points of the auxiliary feature points and Standard distance a second auxiliary feature point distance between the true value coordinate points of the auxiliary feature points within the distance range is obtained, and it is determined whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points based on the first auxiliary feature point distance and the second auxiliary feature point distance. Here, Standard distance the distance range can be set based on the application scenario.

[0062] In step 205, based on the position transformation matrix, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points, and then the three-dimensional coordinate points of the auxiliary feature points and the point cloud data are stitched to obtain the target scanning data.

[0063] Specifically, the three-dimensional coordinate points of the collected auxiliary feature points and BuStitch the true coordinate points of the auxiliary feature points. When the stitching is successful, the designed scanning rod data can be transferred to the coordinate system of the auxiliary feature points by the position transformation matrix, and the scanning rod data obtained in real time can be replaced.

[0064] In the scanning data processing mode according to the embodiment of the present disclosure, the target oral cavity is scanned to obtain a plurality of scanning images each including auxiliary feature points, three-dimensional reconstruction is performed based on each scanning image to obtain an array of reconstructed diagrams, the array of reconstructed diagrams is calculated to obtain the three-dimensional coordinate points and point cloud data of the auxiliary feature points, and the three-dimensional coordinate points of the auxiliary feature points and Bu Determine whether to stitch the true coordinate points of the auxiliary feature points. When stitching the three-dimensional coordinate points of the auxiliary feature points and the true coordinate points of the auxiliary feature points, obtain the position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true coordinate points of the auxiliary feature points, transfer the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then stitch the three-dimensional coordinate points of the auxiliary feature points and the point cloud data to obtain the target scanning data. Thereby, by improving the alignment accuracy between the designed scanning rod data and the real-time scanning rod data, the processing efficiency and accuracy of the scanning data in the intraoral scanning scenario are improved.

[0065] Specifically, FIG. 4 is a flowchart of another scanning data processing method according to the embodiment of the present disclosure. This embodiment further optimizes the above scanning data processing method based on the above embodiment. As shown in FIG. 4, the method includes the following steps 301 to 305.

[0066] In step 301, scan the target oral cavity to obtain a plurality of scanning images including auxiliary feature points.

[0067] Step 301 is the same as step 101. Specifically refer to the description of step 101, and detailed description is omitted here.

[0068] In step 302, perform three-dimensional reconstruction based on each scanning image to obtain an array of reconstructed diagrams, calculate the array of reconstructed diagrams to obtain the three-dimensional coordinate points and point cloud data of the auxiliary feature points.

[0069] Specifically, the target oral cavity is scanned by intraoral scanning, scanning images (reconstruction diagrams and texture diagrams) for three-dimensional reconstruction are collected, point cloud data including teeth, gums, and the scanning rod is calculated using the array of reconstruction diagrams, and the three-dimensional coordinate points of the auxiliary feature points are reconstructed using the array of texture diagrams. In order to obtain the reconstruction diagram and the texture diagram at the same time, it is considered that the three-dimensional data and the three-dimensional coordinate points of the auxiliary feature points correspond one-to-one in the same coordinate system.

[0070] In step 303, a plurality of planes are fitted based on the three-dimensional coordinate points of the auxiliary feature points, a target geometry is constructed based on the plurality of planes, and at least three planes are obtained based on the standard scanning rod data , preferably, preset standard scanning rod data and the normal vector of each plane and the intersection points of at least three planes are obtained.

[0071] In step 304, a position transformation matrix is obtained based on the normal vector of each plane and the intersection points of at least three planes.

[0072] In step 305, after the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points based on the position transformation matrix, the target geometry is replaced and stitched with the point cloud data to obtain the target scanning data.

[0073] Here, the position transformation matrix refers to converting the standard scanning rod data into the position transformation relationship of the coordinate system of the auxiliary feature points.

[0074] Specifically, the basic geometric features of the scanning rod are fitted by a plurality of auxiliary feature points. For example, the scanning rod is a cuboid, and auxiliary feature points are distributed on all five planes of the four side faces and one upper face of the cuboid. By using the coordinates of the auxiliary feature points on each plane and fitting one cuboid respectively, the target geometry can be obtained.

[0075] Therefore, the auxiliary feature points obtained during the scanning process fit a standard target geometry based on rules, and the target geometry may be the position of the directly designed scanning rod data, or the position of the designed scanning rod data may be determined by aligning the target geometry with the designed scanning rod data.

[0076] In the scanning data processing mode according to the embodiments of the present disclosure, the target oral cavity is scanned to obtain a plurality of scanning images including auxiliary feature points, 3D reconstruction is performed based on each scanning image to obtain an array of reconstructed diagrams, the array of reconstructed diagrams is calculated to obtain the 3D coordinate points and point cloud data of the auxiliary feature points, a plurality of planes are fitted based on the 3D coordinate points of the auxiliary feature points, a target geometry is constructed based on the plurality of planes, at least three planes are obtained based on the standard scanning rod data, the normal vector of each plane and the intersection points of at least three planes are obtained, a position transformation matrix is obtained based on the normal vector of each plane and the intersection points of at least three planes, the standard scanning rod data is transferred to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then the target geometry is replaced and stitched with the point cloud data to obtain the target scanning data. Thereby, by improving the accuracy of the positioning information of the scanning rod, the processing efficiency and accuracy of the scanning data in the intraoral scanning scenario are improved.

[0077] FIG. 5 is a configuration diagram of a scanning data processing apparatus according to an embodiment of the present disclosure. The apparatus can be realized by software and / or hardware and may generally be integrated into an electronic device. As shown in FIG. 5, the apparatus includes an image acquisition module 401 used to acquire a plurality of scanning images including auxiliary feature points, an image processing module 402 used to process the plurality of scanning images to obtain the 3D coordinate points of the auxiliary feature points, and a generation module 403 used to generate target scanning data based on the 3D coordinate points and point cloud data of the auxiliary feature points.

[0078] Optionally, the generation module 403 specifically It is used to generate target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data.

[0079] Optionally, the auxiliary feature points have corresponding true value coordinate points, and specifically, the image processing module 402 Performs three-dimensional reconstruction based on each of the scanning images, obtains an array of reconstructed diagrams, It is used to calculate the array of the reconstructed diagrams to obtain the three-dimensional coordinate points of the auxiliary feature points and the point cloud data.

[0080] Optionally, the generation module 403 The three-dimensional coordinate points of the auxiliary feature points and Bu A judgment unit used to determine whether to stitch the true value coordinate points of the auxiliary feature points, and When stitching the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, an acquisition unit used to obtain a position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, and A stitching unit used to shift the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then stitch the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the true value coordinate points of the auxiliary feature points to obtain the target scanning data.

[0081] Optionally, the three-dimensional coordinate points of the auxiliary feature points and Bu Specifically, the judgment unit used to determine whether to stitch the true value coordinate points of the auxiliary feature points Obtains a first auxiliary feature point distance between the three-dimensional coordinate points of the auxiliary feature points and the three-dimensional coordinate points of the auxiliary feature points within a standard distance range, Obtains the true value coordinate points of the auxiliary feature points that match the three-dimensional coordinate points of the auxiliary feature points, and the true value coordinate points of the auxiliary feature points and the Standard Obtains a second auxiliary feature point distance between the true value coordinate points of the auxiliary feature points within the distance range, It is used to determine whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points based on the first auxiliary feature point distance and the second auxiliary feature point distance.

[0082] Optionally, the generation module 403 specifically fits a plurality of planes based on the three-dimensional coordinate points of the auxiliary feature points, constructs a target geometry based on the plurality of planes, obtains at least three planes based on the standard scanning rod data, and obtains the normal vector of each plane and the intersection point of the at least three planes, obtains a position transformation matrix based on the normal vector of each plane and the intersection point of the at least three planes, It is used to shift the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then replace the target geometry and stitch it with the point cloud data to obtain the target scanning data.

[0083] Optionally, the auxiliary feature points have corresponding true value coordinate points, and the generation module 403 specifically stitches the three-dimensional coordinate points of any one auxiliary feature point and the true value coordinate points of the corresponding auxiliary feature point to obtain the three-dimensional coordinate points of the target auxiliary feature points in the same coordinate system, obtains the position transformation matrix between the three-dimensional coordinate points of the target auxiliary feature points and the true value coordinate points of the corresponding auxiliary feature points, and based on the position transformation matrix, shifts the standard scanning rod data to the coordinate system of the auxiliary feature points, and then stitches it with the three-dimensional coordinate points of the auxiliary feature points and the point cloud data to obtain the target scanning data.

[0084] The scanning data processing device according to the embodiments of the present disclosure can execute the scanning data processing method according to any embodiment of the present disclosure, and has the functional modules and beneficial effects corresponding to the execution of the method.

[0085] Embodiments of the present disclosure further provide a computer program product including a computer program / instructions that, when executed by a processor, implement a scanning data processing method according to any embodiment of the present disclosure.

[0086] FIG. 6 is a block diagram of an electronic device according to an embodiment of the present disclosure. Hereinafter, specifically, as shown in FIG. 6, a block diagram of an electronic device 500 for implementing an embodiment of the present disclosure is shown. The electronic device 500 in the embodiments of the present disclosure may include, for example, portable terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers, but is not limited thereto. The electronic device as shown in FIG. 6 is merely an example and does not limit the functions and usage ranges of the embodiments of the present disclosure.

[0087] As shown in FIG. 6, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 501 that can execute various appropriate operations and processes based on a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic device 500 are further stored. The processing device 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0088] Generally, the I / O interface 505 may be connected to an input device 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc., an output device 507 including, for example, a liquid crystal display (LCD), a speaker, an oscillator, etc., a storage device 508 including, for example, a magnetic tape, a hard disk, etc., and a communication device 509. The communication device 509 enables the electronic device 500 to perform wireless or wired communication with other devices to exchange data. FIG. 6 shows the electronic device 500 equipped with various devices, but it should be understood that it is not required to implement or include all the shown devices. Alternatively, more or fewer devices may be implemented or included.

[0089] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a non-transitory computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network by the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, it executes the above-described functions limited to the scanning data processing method of the embodiment of the present disclosure.

[0090] Note that the computer-readable medium of the present disclosure may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above, but is not limited thereto. More specific examples of the computer-readable storage medium may include an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above, but is not limited thereto. In the present disclosure, the computer-readable storage medium may be any tangible medium that includes or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier, and the data signal carries computer-readable program code. Such a propagated data signal may use various forms, including electromagnetic signals, optical signals, or any suitable combination of the above, but is not limited thereto. The computer-readable signal medium may be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium may be used to transmit, propagate, or transmit a program used by or in combination with an instruction execution system, apparatus, or device. The program code included in the computer-readable medium may be transmitted by any suitable medium, which may include conductive wires, optical fiber cables, RF (radio frequency), etc., or any combination of the above, but is not limited thereto.

[0091] In some embodiments, the client side and the server can communicate using any currently known or future-developed network protocol, such as HTTP (Hyper Text Transfer Protocol), and can also communicate with digital data in any form or medium (e.g., a communication network) and be connected to each other. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), and include any currently known or future-developed network.

[0092] The computer-readable medium may be included in the electronic device, may not be disposed in the electronic device, and may exist individually.

[0093] One or more programs are carried on the computer-readable medium, and when the one or more programs are executed by the electronic device, the electronic device is caused to acquire a plurality of scanned images including auxiliary feature points, process the plurality of scanned images to obtain three-dimensional coordinate points of the auxiliary feature points, and generate target scanned data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data.

[0094] The computer program code for performing the operations of the present disclosure may be created in one or more programming languages or combinations thereof, including object-oriented programming languages such as the above programming languages, Java, Smalltalk, C++, and further including conventional procedural programming languages such as the "C" language or similar programming languages, but is not limited thereto. The program code may be executed entirely on the user's computer, may be partially executed on the user's computer, may be executed as a single independent software package, may be partially executed on the user's computer and partially on a remote computer, or may be executed entirely on a remote computer or server. When a remote computer is involved, the remote computer can be connected to the user's computer via any type of network including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, connecting via the Internet using an Internet service provider).

[0095] Flowcharts and block diagrams in the drawings illustrate the possible system architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram can represent a module, a program section, or a portion of code, and the module, program section, or portion of code includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions attached to the blocks may occur in a different order than the order shown in the drawings. For example, two blocks shown consecutively may actually be executed basically in parallel or in the reverse order, depending on the related functions. It should be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware system that performs a given function or operation, or may be implemented by a combination of dedicated hardware and computer instructions.

[0096] The units described in the embodiments of the present disclosure may be implemented in the form of software or in the form of hardware. Here, the name of the unit does not limit the unit itself in some cases.

[0097] The functions described above in this specification may be executed, at least in part, by one or more hardware logic devices. For example, without limitation, exemplary types of available hardware logic components include field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on chip (SOC), complex programmable logic devices (CPLD), and the like.

[0098] In the context of the present disclosure, a machine-readable medium may be a tangible medium that includes, or is capable of storing, a program for use by or in combination with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronics, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any combination of the foregoing. More specific examples of the machine-readable storage medium include electrical connections by one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0099] According to one or more embodiments of the present disclosure, the present disclosure a processor; a memory used to store executable instructions of the processor, and an electronic device, wherein the processor reads the executable instructions from the memory and executes the instructions to provide an electronic device for implementing any one of the scanning data processing methods according to the present disclosure.

[0100] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for executing any one of the scanning data processing methods according to the present disclosure. Further, embodiments of the present disclosure a processor; a memory used to store instructions executable by the processor, and an apparatus, wherein the processor acquires a plurality of scanning images including auxiliary feature points, Process the plurality of scanned images to obtain three-dimensional coordinate points of auxiliary feature points and point cloud data, Further provide an apparatus configured to generate target scan data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data.

[0101] Note that in this specification, for example, relative terms such as "first" and "second" are merely for distinguishing one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Also, technical terms such as "including" and "having" or any other variation thereof cover non-exclusive "including", so that a process, method, article or device including a series of elements includes not only those elements but also other elements not explicitly listed, or further includes elements specific to such a process, method, article or device. Without further limitation, the elements defined by the phrase "including one..." do not exclude the presence of other same elements in the process, method, article or device including the said elements.

[0102] The above content is only an embodiment of the present disclosure and is for those skilled in the art to understand or implement the present disclosure. Multiple modifications of these examples can be easily realized by those skilled in the art, and the general principles defined in this specification can be implemented in other examples without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure should not be limited to these examples shown in this specification, but should be adapted to the broadest scope consistent with the principles and novel features disclosed in this specification.

Industrial Applicability

[0103] The scanning data processing method according to the present disclosure can effectively calculate scanning data, improve the alignment accuracy between the designed scanning rod data and the real-time scanning rod data, can well consider the influence on scanning efficiency and accuracy in the scenario of intraoral scanning, optimize the scanning process, and has very high industrial applicability.

Claims

1. A scanning data processing method, comprising: obtaining a plurality of scanning images including auxiliary feature points, wherein the auxiliary feature points are provided on an intraoral scanning rod, and the auxiliary feature points have corresponding true value coordinate points; processing the plurality of scanning images to obtain three-dimensional coordinate points and point cloud data of the auxiliary feature points; generating target scanning data based on the three-dimensional coordinate points and the point cloud data of the auxiliary feature points, wherein the target scanning data is obtained by converting standard data into the same coordinate system as the three-dimensional coordinate points of the auxiliary feature points and then stitching the point cloud data to replace the scanning rod data scanned in real time.

2. The step of generating target scanning data based on the three-dimensional coordinate points and the point cloud data of the auxiliary feature points includes: generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data. The scanning data processing method according to claim 1, characterized in that.

3. The step of processing the plurality of scanning images to obtain three-dimensional coordinate points and point cloud data of the auxiliary feature points includes: performing three-dimensional reconstruction based on each of the scanning images to obtain an array of reconstructed diagrams; obtaining the three-dimensional coordinate points and the point cloud data of the auxiliary feature points by using the array of the reconstructed diagrams. The scanning data processing method according to claim 1, characterized in that.

4. The auxiliary feature points have corresponding true value coordinate points, the standard data is standard scanning rod data, and the step of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data includes: determining whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points; when stitching the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points, obtaining a position transformation matrix between the three-dimensional coordinate points of the auxiliary feature points and the true value coordinate points of the auxiliary feature points; shifting the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then stitching the three-dimensional coordinate points of the auxiliary feature points and the point cloud data to obtain the target scanning data. The scanning data processing method according to claim 2, characterized in that.

5. The standard data is standard scanning rod data, The step of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data includes: Fitting a plurality of planes based on the three-dimensional coordinate points of the auxiliary feature points, and constructing a target geometry based on the plurality of planes; Obtaining at least three planes based on the standard scanning rod data, and obtaining the normal vector of each plane and the intersection points of the at least three planes; Obtaining a position transformation matrix based on the normal vector of each plane and the intersection points of the at least three planes; Shifting the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, then replacing the target geometry and stitching it with the point cloud data to obtain the target scanning data. The scanning data processing method according to claim 2 is characterized by including the above steps.

6. The auxiliary feature points have corresponding true value coordinate points, and the standard data is standard scanning rod data. The step of generating target scanning data based on the three-dimensional coordinate points of the auxiliary feature points, the point cloud data, and the standard data includes: Stitching the three-dimensional coordinate points of any one of the auxiliary feature points with the true value coordinate points of the corresponding auxiliary feature point to obtain the three-dimensional coordinate points of the target auxiliary feature point in the same coordinate system; Obtaining a position transformation matrix between the three-dimensional coordinate points of the target auxiliary feature point and the true value coordinate points of the corresponding auxiliary feature point; Shifting the standard scanning rod data to the coordinate system of the auxiliary feature points based on the position transformation matrix, and then stitching the three-dimensional coordinate points of the auxiliary feature points with the point cloud data to obtain the target scanning data. The scanning data processing method according to claim 2 is characterized by including the above steps.

7. Obtaining images of a plurality of frames; Based on the images of the plurality of frames, obtaining initial three-dimensional data including an initial point set of the target oral cavity where the intraoral scanning rod is attached and the three-dimensional coordinate measurement values of the auxiliary feature points provided on the intraoral scanning rod in the same coordinate system; Obtaining a preset model including the true three-dimensional coordinates of the auxiliary feature points of the intraoral scanning rod and the real point set of the intraoral scanning rod in the same coordinate system. Performing stitching on the initial point set of the target oral cavity and the real point set of the intraoral scanning rod based on the correspondence between the three-dimensional coordinate measurement values of the auxiliary feature points and the true three-dimensional coordinates of the auxiliary feature points; Determining the positioning information of the intraoral scanning rod based on the stitched real point set of the intraoral scanning rod, including the steps of: The scanning data processing method according to claim 1, characterized in that it comprises:

8. The step of processing the plurality of scanning images to obtain the three-dimensional coordinate points and point cloud data of the auxiliary feature points includes: Obtaining the two-dimensional coordinate points of the auxiliary feature points in each of the plurality of scanning images; Performing coordinate system conversion on the two-dimensional coordinate points to obtain the three-dimensional coordinate points of the auxiliary feature points in each of the plurality of scanning images; Stitching the three-dimensional coordinate points of the auxiliary feature points in each of the plurality of scanning images to obtain the three-dimensional coordinate points and point cloud data of all the auxiliary feature points, including the steps of: The scanning data processing method according to claim 1, characterized in that it comprises:

9. The step of determining whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true coordinate points of the auxiliary feature points includes: Obtaining a first auxiliary feature point distance between the three-dimensional coordinate points of the auxiliary feature points and the three-dimensional coordinate points of the auxiliary feature points within a standard distance range; Obtaining the true coordinate points of the auxiliary feature points that match the three-dimensional coordinate points of the auxiliary feature points; Obtaining a second auxiliary feature point distance between the true coordinate points of the auxiliary feature points and the true coordinate points of the auxiliary feature points within the standard distance range; Determining whether to stitch the three-dimensional coordinate points of the auxiliary feature points and the true coordinate points of the auxiliary feature points based on the first auxiliary feature point distance and the second auxiliary feature point distance, including the steps of: The scanning data processing method according to claim 4, characterized in that it comprises:

10. A scanning data processing device, comprising: An image acquisition module used to acquire a plurality of scanning images including auxiliary feature points, wherein the auxiliary feature points are provided on an intraoral scanning rod, and the auxiliary feature points have corresponding true coordinate points; An image processing module used to process the plurality of scanning images to obtain the three-dimensional coordinate points of the auxiliary feature points; A generation module used to generate target scan data based on the three-dimensional coordinate points of the auxiliary feature points and the point cloud data, wherein the target scan data is obtained by converting standard data into the same coordinate system as the three-dimensional coordinate points of the auxiliary feature points, stitching the point cloud data, and replacing the scanned rod data scanned in real time. The scanning data processing apparatus is characterized by including the generation module.

11. Specifically, the image processing module Performs three-dimensional reconstruction based on each of the scanned images to obtain an array of reconstructed diagrams, The scanning data processing apparatus according to claim 10, wherein the array of the reconstructed diagrams is used to calculate the three-dimensional coordinate points of the auxiliary feature points.

12. An electronic device, A processor, A memory used to store executable instructions of the processor, and The processor reads the executable instructions from the memory and executes the executable instructions to implement the scanning data processing method according to any one of claims 1 to 9. The electronic device is characterized by this.

13. A computer program used to execute the scanning data processing method according to any one of claims 1 to 9.

Images