Scan data processing method, device, equipment and medium

Abstract

An embodiment of the present disclosure relates to a scanning data processing method, device, equipment and medium, in which the method includes: acquiring a plurality of frames of processing target images including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values, performing processing according to the plurality of frames of processing target images, combining with a current camera pose to acquire all three-dimensional coordinate points in the same coordinate system, performing measurement processing for all three-dimensional coordinate points, acquiring target three-dimensional coordinate points, and determining a transformation relationship of a target position according to the target three-dimensional coordinate points and the distribution true values. By using the above technical solution, the transformation relationship of the target position is determined according to the preset distribution true values ​​of the auxiliary feature points and the three-dimensional coordinate points calculated by scanning, thereby enabling accurate positioning of the relative position between the scanning rods, and improving the efficiency and accuracy of data processing in the case of intraoral scanning.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This disclosure claims priority to a Chinese patent application filed with the China National Intellectual Property Office on May 2, 2022, bearing application number 202210494057.6 and titled "Scanned Data Processing Method, Apparatus, Instrument and Medium," the entire contents of which are incorporated herein by reference.

[0002] FIELD OF THE DISCLOSURE The present disclosure relates to the field of intraoral scanning technology, and in particular to scan data processing methods, devices, equipment and media. [Background technology]

[0003] Generally, for restoration of missing teeth, the implant position is determined by scanning with a scanning rod.

[0004] In related art, due to the limited scanning range of intraoral scanning, the solution of stitching multi-data is commonly used when scanning intraoral data, which has accumulated errors and ultimately leads to low overall accuracy of the model. Summary of the Invention [Problem to be solved by the invention]

[0005] The technical problem to be solved by the present disclosure is to solve the problem that in the conventional solution of stitching multi-data when scanning intraoral data, there is an accumulated error, which ultimately leads to a low overall accuracy of the model. [Means for solving the problem]

[0006] To solve the above technical problems, embodiments of the present disclosure provide a scan data processing method, apparatus, device, and medium.

[0007] In a first aspect, there is provided a method of processing scan data, said method comprising: acquiring a plurality of frames of a processing target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values; performing processing based on the plurality of frames of processing target images to obtain all three-dimensional coordinate points in the same coordinate system; performing measurement processing on all of the three-dimensional coordinate points to obtain a target three-dimensional coordinate point; determining a transformation relationship of a target position based on the target three-dimensional coordinate point and the distribution true value.

[0008] In a second aspect, there is further provided an apparatus for processing scan data, said apparatus comprising: an image acquisition module for acquiring a plurality of frames of a processing target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values; an image processing module for performing processing based on the plurality of frames of processing target images to obtain all three-dimensional coordinate points in the same coordinate system; a measurement processing module for performing measurement processing on all of the three-dimensional coordinate points to obtain a target three-dimensional coordinate point; and a determination module for determining a transformation relationship of a target position according to the target three-dimensional coordinate point and the distribution true value.

[0009] In a third aspect, there is further provided an electronic device including a processor and a memory for storing executable instructions for the processor, the processor being configured to read the executable instructions from the memory and execute the instructions to realize a scan data processing method according to an embodiment of the present disclosure.

[0010] In a fourth aspect, there is further provided a computer storage medium, wherein the storage medium has stored thereon a computer program, for example for performing the scan data processing method according to the first aspect of the present disclosure. Effect of the Invention

[0011] The above technical solutions according to the embodiments of the present disclosure have the following advantages over the prior art:

[0012] The scanning data processing method according to the embodiment of the present disclosure includes obtaining a plurality of frames of processing target images including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values, performing processing according to the plurality of frames of processing target images, obtaining all three-dimensional coordinate points in the same coordinate system, performing measurement processing for all three-dimensional coordinate points, obtaining target three-dimensional coordinate points, and determining a transformation relationship of the target position according to the target three-dimensional coordinate points and the distribution true values. The adoption of the above technical solution enables accurate positioning of the relative position between the scanning rods by determining the transformation relationship of the target position according to the preset distribution true values ​​of the auxiliary feature points and the three-dimensional coordinate points calculated by scanning, thereby improving the efficiency and accuracy of data processing in the case of intraoral scanning.

[0013] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. [Brief description of the drawings]

[0014] The drawings are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description serve to explain the principles of the present disclosure.

[0015] In order to more clearly describe the embodiments of the present disclosure or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art are briefly described below, and it goes without saying that those skilled in the art may further obtain other drawings based on these drawings without requiring creative work. [Figure 1] FIG. 2 illustrates an application scenario of scan data processing according to an embodiment of the present disclosure. [Diagram 2] 1 is a flow chart of a method for processing scan data according to an embodiment of the present disclosure. [Diagram 3] 10 is a flow chart of another method for processing scan data according to an embodiment of the present disclosure. [Figure 4] 10 is a flow chart of another method for processing scan data according to an embodiment of the present disclosure. [Diagram 5] FIG. 1 is a block diagram of a scan data processing apparatus according to an embodiment of the present disclosure. [Figure 6] FIG. 1 is a configuration diagram of an electronic device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] In order to make the objectives, technical solutions and advantages of the embodiments of the present disclosure more clear, the following clearly and comprehensively describes the technical solutions in the embodiments of the present disclosure, and it is to be understood that the described embodiments are only some of the embodiments of the present disclosure, and not all of the embodiments, and any other embodiments obtained by a person skilled in the art based on the embodiments of the present disclosure without creative work are all within the scope of the claims of the present disclosure.

[0017] In practical applications, for restoration of missing teeth, the scanning range of intraoral scanning is limited, so the solution of stitching multi-data is commonly used when scanning intraoral data, which has accumulated errors and ultimately leads to low overall accuracy of the model.

[0018] In response to the above problems, the present disclosure provides a scan data processing method, which may be applied to an application environment as shown in Fig. 1. Fig. 1 is a diagram showing an application scenario of scan data processing according to an embodiment of the present disclosure, in which the application environment includes attaching a plurality of intraoral scanning rods to a target oral cavity, the intraoral scanning rods include scanning rod parts 11 and auxiliary parts 12 connected to the scanning rod parts 11, auxiliary features are provided on the scanning rod parts 11 and / or auxiliary parts 12, where the shape feature of the auxiliary parts 12 itself is the auxiliary feature, the scanning rod parts 11 are matched with an implant body attached to the target oral cavity, and the scanning rod parts 11 are fitted to the implant body, so that the intraoral scanning rod is attached to the target oral cavity.

[0019] Here, by fitting the auxiliary parts 12 of any two of the multiple intraoral scanning rods to each other, when any two intraoral scanning rods 10 are adjacently attached to the oral cavity, the auxiliary feature points of the two auxiliary parts are continuously distributed, and the true coordinate points of the auxiliary feature points can be obtained in advance, for example, by a single-lens photographing measuring system, a three-dimensional coordinate meter, etc., and theoretically, the three-dimensional coordinate points corresponding to the image obtained by scanning correspond one-to-one to the true coordinate points of the auxiliary feature points obtained in advance.

[0020] As an example of one scenario, multiple intraoral scanning rods are attached to a target oral cavity, the intraoral scanning rods include scanning rod parts connected to the implant body and auxiliary parts connected to the scanning rod parts, the intraoral scanning rods are provided with target features, the target features are continuously distributed on the scanning rods and / or the auxiliary parts, and the target features are not distributed on one side of the scanning rods and / or the auxiliary parts.

[0021] Specifically, the intraoral scanner scans the target intraoral cavity, acquires multiple frames of images, and transfers them to the data processing module for data processing, which performs the following method.

[0022] Acquire a plurality of frames of images, and acquire initial three-dimensional data of the target oral cavity based on the plurality of frames of images, the initial three-dimensional data including an initial set of points of the target oral cavity and three-dimensional coordinate measurements of the target features in the same coordinate system; Obtain a preset model of the intraoral scanning rod, the preset model including a 3D coordinate true value of the target feature in the same coordinate system and a real point set (the 3D coordinate true value of each point) of the intraoral scanning rod; stitching the initial point set of the target oral cavity and the real point set of the intraoral scanning rod according to the correspondence relationship between the three-dimensional coordinate measurement value of the target feature and the true value; The positioning information of the intraoral scanning rod is determined based on the real point set of the stitched intraoral scanning rod, and the positioning information of the intraoral scanning rod is the positioning information of the implant body, and the tooth body can be designed based on the positioning information, and the designed and manufactured tooth body can be fitted to the implant body.

[0023] Specifically, a plurality of frames of processing target images including auxiliary feature points are obtained, where the auxiliary feature points have corresponding true distribution values, and processing is performed based on the plurality of frames of processing target images to obtain all three-dimensional coordinate points in the same coordinate system, perform measurement processing on all three-dimensional coordinate points to obtain target three-dimensional coordinate points, and determine a transformation relationship of the target position based on the target three-dimensional coordinate points and the true distribution values, thereby achieving the determination of the transformation relationship of the target position based on the preset true distribution values ​​of the auxiliary feature points and the three-dimensional coordinate points calculated by scanning, enabling accurate positioning of the relative positions between the scanning rods, and improving the efficiency and accuracy of data processing in the case of intraoral scanning.

[0024] Specifically, Figure 2 is a flow chart of a scan data processing method according to an embodiment of the present disclosure, which may be performed by a scan data processing device, where the device may be implemented using software and / or hardware, and may be generally integrated into electronic equipment. As shown in Figure 2, the method includes the following steps: In step 101, a multi-frame image to be processed containing auxiliary feature points is obtained, where the auxiliary feature points have corresponding distribution true values.

[0025] Here, the scanning rod includes a scanning rod body and an auxiliary feature body, and an auxiliary feature point is set on the scanning rod body, or the scanning rod body has no auxiliary feature point, and an auxiliary feature point is set on the auxiliary feature body between the scanning rod bodies, or the shape of the auxiliary feature body itself is the auxiliary feature point.

[0026] Here, the target oral cavity refers to the oral cavity where dental implant treatment needs to be performed, and the intraoral cavity needs to be scanned to determine the implant position within the oral cavity. The auxiliary parts of any two of the multiple intraoral scanning rods are previously attached adjacent to each other in the target oral cavity, and the auxiliary feature points of the two auxiliary parts are continuously distributed, so that intraoral scanning of the target oral cavity can be performed and multiple frames of images to be processed can be obtained.

[0027] Here, the target oral cavity can be scanned using a handheld oral cavity scanner (single-lens or binocular camera), i.e., multiple frames of images to be processed are obtained by photographing, for example, tens of frames of images to be processed can be collected per second, and the collection can be repeated.

[0028] In an embodiment of the present disclosure, the scanning rod is a feature object including auxiliary feature points, where a feature can be uniquely identified by an auxiliary feature point; that is, when auxiliary feature points are set on the scanning rod, each auxiliary feature point can uniquely identify a position feature corresponding to the scanning rod; for example, target feature a and target feature b are set at positions 1 and 2 on the scanning rod, respectively, target feature a can uniquely identify the position feature at position 1 on the scanning rod, and target feature b can uniquely identify the position feature at position 2 on the scanning rod.

[0029] It should be understood that any uniquely identifying position features corresponding to the scanning rod, such as different shapes, colors, two-dimensional codes on the scanning rod, etc., can be the auxiliary features.

[0030] Specifically, the true distribution values ​​of auxiliary feature points or the true distribution values ​​of computer-aided design auxiliary feature points, i.e., the coordinate values ​​of the auxiliary feature points, can be obtained using higher precision equipment, such as a single-lens camera measuring system or a three-dimensional coordinate machine.

[0031] In the embodiments of the present disclosure, each processing target image acquired by each scan includes at least a predetermined number of auxiliary feature points, and exhibits distribution continuity of the auxiliary feature points to ensure the accuracy of subsequent calculations, where the predetermined number can be set based on the application scenario.

[0032] For example, the specified number is 3, which indicates that when the processing target image obtained by scanning contains two auxiliary feature points, the distribution of the auxiliary feature points is not continuous, and when the processing target image obtained by scanning contains three auxiliary feature points, the distribution of the auxiliary feature points is continuous.

[0033] In the embodiments of the present disclosure, there are various methods for acquiring multiple frames of images to be processed, and in some embodiments, multiple frames of images to be processed are acquired by controlling a monocular camera to rotate in a certain direction and scan the target oral cavity at a certain frequency.

[0034] In another embodiment, the binocular camera is controlled to repeatedly scan the target oral cavity to obtain multiple frames of processing target images. The above two methods are merely examples of obtaining multiple frames of processing target images, and the embodiments of the present disclosure do not limit the specific manner of obtaining multiple frames of processing target images.

[0035] Specifically, after a scanning rod is connected to the target oral cavity, the target oral cavity including the scanning rod is scanned to obtain a plurality of frames of images to be processed.

[0036] In step 102, processing is performed based on a plurality of frames of the processing target image to obtain all three-dimensional coordinate points in the same coordinate system.

[0037] Here, all the three-dimensional coordinate points refer to three-dimensional coordinate points corresponding to all the auxiliary feature points in the target oral cavity.

[0038] In the embodiments of the present disclosure, there are various methods for performing processing based on multiple frames of target images and obtaining all three-dimensional coordinate points in the same coordinate system. In some embodiments, two-dimensional coordinate points of auxiliary feature points in the target image of each frame are obtained, three-dimensional reconstruction is performed based on a predetermined internal parameter matrix and the two-dimensional coordinate points to obtain three-dimensional coordinate points corresponding to the auxiliary feature points of the target image of each frame, and the three-dimensional coordinate points corresponding to the auxiliary feature points of the target images of the multiple frames are stitched together to obtain three-dimensional coordinate points in the same coordinate system of the auxiliary feature points of the target images of the multiple frames.

[0039] In another embodiment, two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame are obtained, three-dimensional reconstruction is performed based on the relative positions between the two cameras and the two-dimensional coordinate points, three-dimensional coordinate points corresponding to the auxiliary feature points of the processing target image of each frame are obtained, and three-dimensional coordinate points corresponding to the auxiliary feature points of the processing target images of multiple frames are stitched together to obtain three-dimensional coordinate points in the same coordinate system of the auxiliary feature points of the processing target images of multiple frames. The above two methods are merely examples of performing processing based on processing target images of multiple frames and obtaining three-dimensional coordinate points in the same coordinate system, and the embodiments of the present disclosure do not limit the specific aspects of performing processing based on processing target images of multiple frames and obtaining three-dimensional coordinate points in the same coordinate system.

[0040] In the embodiment of the present disclosure, after acquiring a plurality of frames of the processing target image, processing is performed based on the plurality of frames of the processing target image, and three-dimensional coordinate points in the same coordinate system can be acquired.

[0041] In step 103, a measurement process is performed on the three-dimensional coordinate points to obtain target three-dimensional coordinate points.

[0042] Here, the target 3D coordinate point refers to a 3D coordinate point obtained by measuring and processing a 3D coordinate point, and can more accurately indicate the 3D coordinate point of the auxiliary feature point.

[0043] In the embodiments of the present disclosure, there are various methods for performing measurement processing on all three-dimensional coordinate points and obtaining a target three-dimensional coordinate point. In some embodiments, each three-dimensional coordinate point is projected onto an image coordinate system to obtain a two-dimensional pixel coordinate point. When the Euclidean distance between the two-dimensional pixel coordinate point and the two-dimensional coordinate point corresponding to the three-dimensional coordinate point is the smallest, the three-dimensional coordinate point is determined as the target three-dimensional coordinate point.

[0044] In another embodiment, an image of an Nth frame in which a three-dimensional coordinate point appears is obtained, a two-dimensional pixel coordinate point of the image of the Nth frame is obtained, and a target three-dimensional coordinate point is determined based on a distance between the two-dimensional coordinate point projected by the three-dimensional coordinate point and the two-dimensional pixel coordinate point, where N is a positive integer. The above two methods are merely examples of performing measurement processing on all three-dimensional coordinate points and obtaining a target three-dimensional coordinate point, and the embodiments of the present disclosure do not limit the specific manner of performing measurement processing on all three-dimensional coordinate points and obtaining a target three-dimensional coordinate point.

[0045] Specifically, after all three-dimensional coordinate points are obtained, a measurement process is performed on all three-dimensional coordinate points to obtain the target three-dimensional coordinate points.

[0046] In step 104, a transformation relationship of the target position is determined based on the target three-dimensional coordinate points and the distribution true value.

[0047] Here, the transformation relationship of the target position refers to transforming the target three-dimensional coordinate points acquired by scanning into a transformation matrix of the correspondingly designed distribution true value.

[0048] In the embodiments of the present disclosure, there are various methods for determining the transformation relationship of the target position based on the target 3D coordinate points and the distribution true values. In some embodiments, calculations are performed based on a predetermined scale factor, the target 3D coordinate points, and the distribution true values ​​to obtain the transformation relationship of the initial position, optimization calculations are performed on the transformation relationship of the scale factor, the target 3D coordinate points, the distribution true values, and the initial position based on a predetermined optimization formula to obtain an optimized value, and the transformation relationship between the scale factor and the initial position is adjusted. When the optimization value is smaller than a predetermined threshold, the transformation relationship of the corresponding initial position is obtained as the transformation relationship of the target position.

[0049] In another embodiment, the position conversion relationship from each target 3D coordinate point to the distribution true value is calculated, and the conversion relationship of the target position is obtained based on the conversion relationships of the multiple positions. The above two methods are merely examples of determining the conversion relationship of the target position based on the target 3D coordinate point and the distribution true value, and the embodiment of the present disclosure does not limit the specific manner of determining the conversion relationship of the target position based on the target 3D coordinate point and the distribution true value.

[0050] In the scanning data processing method according to the embodiment of the present disclosure, a plurality of frames of processing target images are obtained, where each frame of the processing target image includes auxiliary feature points, and the auxiliary feature points have corresponding distribution true values, and processing is performed according to the plurality of frames of processing target images to obtain all three-dimensional coordinate points, and measurement processing is performed for all three-dimensional coordinate points to obtain target three-dimensional coordinate points, and a transformation relationship of the target position is determined according to the target three-dimensional coordinate points and the distribution true values. By adopting the above technical solution, the transformation relationship of the target position is determined according to the preset distribution true values ​​of the auxiliary feature points and the three-dimensional coordinate points calculated by scanning, thereby enabling accurate positioning of the relative position between the scanning rods, and improving the efficiency and accuracy of data processing in the case of intraoral scanning.

[0051] Based on the description of the above embodiment, scanning can be performed by a monocular camera and a binocular camera. The monocular camera and the binocular camera will be described in detail below in conjunction with FIG. 3 and FIG. 4 respectively.

[0052] Specifically, Figure 3 is a flow chart of another scan data processing method according to an embodiment of the present disclosure, which further optimizes the scan data processing method on the basis of the above embodiment. As shown in Figure 3, the method includes: In step 201, a scanning device is controlled to rotate according to a predetermined direction and scan a target oral cavity including a scanning rod according to a predetermined frequency, and obtains a plurality of frames of processing target images, in which the auxiliary feature points have corresponding distribution true values.

[0053] Specifically, a monocular camera is controlled to rotate in a fixed direction and scan the target oral cavity at a fixed frequency to obtain multiple frames of images to be processed, or a binocular camera is controlled to repeatedly scan the target oral cavity to obtain multiple frames of images to be processed.

[0054] In step 202, the two-dimensional coordinate points of the auxiliary feature points in the processing target image of each frame are obtained, and calculations are performed based on a predetermined internal parameter matrix and the two-dimensional coordinate points to obtain the three-dimensional coordinate points of the processing target image of each frame, and the three-dimensional coordinate points of the processing target image of each frame are stitched to obtain all the three-dimensional coordinate points in the same coordinate system.

[0055] Specifically, a plurality of frames of target images are obtained by scanning (for example, tens of frames of target images are collected per second by using a hardware triggered shooting method), and the two-dimensional coordinate points of the auxiliary feature points in the target images of each frame are obtained based on the target images (that is, the pixel coordinates of the auxiliary feature points in the target images of each frame are extracted by using an image identification processing method), and the auxiliary feature points of the current frame are reconstructed based on the internal parameters of the camera (for example, focal length, principal point, tilt factor and lens distortion) and the attitude of texture tracking (for example, the geometry and texture information of the scanning rod surface is obtained to stitch adjacent frames), and the reconstructed three-dimensional coordinate points are based on the camera coordinate system of the first frame during scanning (the camera coordinate system of the first frame is determined by the image of the first frame, and all subsequent frames are stitched with the first frame, so the coordinate systems are all the camera coordinate system of the first frame, and the two-dimensional coordinate points are image-identified, which need to be converted into the camera coordinate system to obtain the three-dimensional coordinate points).

[0056] Here, the internal parameter matrix refers to a matrix composed of the internal parameters of the camera, and the two-dimensional coordinate points are back-projected into the camera's coordinate system based on the internal parameter matrix to obtain three-dimensional coordinate points.

[0057] Specifically, stitching is performed based on the 3D coordinate points of the processing target image of each frame to obtain all the 3D coordinate points, and the different coordinate systems of the two frames are aligned through their common parts based on a predetermined stitching algorithm to stitch and obtain all the 3D coordinate points. That is, the 3D coordinate points reconstructed in all one frame are merged into the whole frame points, i.e., all the 3D coordinate points are based on the coordinate system (world coordinate system) of the whole tooth model.

[0058] In step 203, each 3D coordinate point is projected into the image coordinate system to obtain a 2D pixel coordinate point. When the Euclidean distance between the 2D pixel coordinate point and the 2D coordinate point corresponding to the 3D coordinate point is the smallest, the 3D coordinate point is set as the target 3D coordinate point.

[0059] JPEG2025502852000002.jpg50170JPEG2025502852000003.jpg14170Here, m indicates the number of three-dimensional coordinate points, and n indicates the number of images to be processed.

[0060] In step 204, calculations are performed based on a predetermined scale factor, a target 3D coordinate point, and a distribution true value to obtain a transformation relationship of an initial position, and optimization calculations are performed on the transformation relationship of the scale factor, the target 3D coordinate point, the distribution true value, and the initial position based on a predetermined optimization formula to obtain an optimized value.

[0061] In step 205, the scale factor and the transformation relationship of the initial position are adjusted, and when the optimized value is smaller than a predetermined threshold, the transformation relationship of the corresponding initial position is obtained as the transformation relationship of the target position.

[0062] JPEG2025502852000004.jpg18170 Every scanning rod can be understood as one small area) Stitching (stitching is a rigid body and does not change scale, when the scale changes, S i By performing the calculation using the scaling factor, the transformation relationship between the target positions of different scanning rods is determined. minf() refers to the nonlinear least squares algorithm. The specific formula is as shown in (2): JPEG2025502852000005.jpg7170JPEG2025502852000006.jpg46170

[0063] This allows accurate positioning of the relative positions between the scanning rods by scanning the entire oral cavity.

[0064] In the scanning data processing method according to the embodiment of the present disclosure, multiple frames of target images including auxiliary feature points are obtained, where the auxiliary feature points have corresponding distribution true values; two-dimensional coordinate points of the auxiliary feature points in the target image of each frame are obtained, and calculated based on a predetermined internal parameter matrix and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the target image of each frame; the three-dimensional coordinate points of the target image of each frame are stitched together to obtain all three-dimensional coordinate points in a same coordinate system; each three-dimensional coordinate point is projected into an image coordinate system to obtain a two-dimensional pixel coordinate point; when the Euclidean distance between the two-dimensional pixel coordinate point and the two-dimensional coordinate point corresponding to the three-dimensional coordinate point is smallest, the three-dimensional coordinate point is taken as a target three-dimensional coordinate point; calculation is performed based on a predetermined scale factor, the target three-dimensional coordinate point, and the distribution true value to obtain a transformation relationship of an initial position; optimization calculation is performed for the transformation relationship between the scale factor, the target three-dimensional coordinate point, the distribution true value, and the initial position according to a predetermined optimization formula to obtain an optimized value; the transformation relationship between the scale factor and the initial position is adjusted; when the optimization value is smaller than a predetermined threshold, the transformation relationship of the corresponding initial position is obtained as the transformation relationship of the target position. This enables accurate positioning of the relative positions between the scanning rods by determining the transformation relationship of the target position based on the preset distribution true values ​​of the auxiliary feature points and the 3D coordinate points calculated by scanning, thereby improving data processing efficiency and accuracy in the case of intraoral scanning.

[0065] Specifically, Figure 4 is a flow chart of another scan data processing method according to an embodiment of the present disclosure, which further optimizes the scan data processing method on the basis of the above embodiment. As shown in Figure 4, the method includes: In step 301, a scanning device is controlled to rotate according to a predetermined direction and scan a target oral cavity including a scanning rod according to a predetermined frequency, and obtains multiple frames of processing target images, in which the auxiliary feature points have corresponding distribution true values.

[0066] Specifically, a monocular camera is controlled to rotate in a fixed direction and scan the target oral cavity at a fixed frequency to obtain multiple frames of images to be processed, or a binocular camera is controlled to repeatedly scan the target oral cavity to obtain multiple frames of images to be processed.

[0067] Step 302: obtain the two-dimensional coordinate points of the auxiliary feature points in the processing target image of each frame, calculate based on the relative positions between the two cameras and the two-dimensional coordinate points, obtain the three-dimensional coordinate points of the processing target image of each frame, stitch the three-dimensional coordinate points of the processing target image of each frame, and obtain all the three-dimensional coordinate points in the same coordinate system.

[0068] Specifically, a processing target image is obtained by scanning, and two-dimensional coordinate points of auxiliary feature points in the processing target image for each frame are obtained based on the processing target image, and three-dimensional coordinate points of the auxiliary feature points in one frame are reconstructed based on the relative positions between the two cameras, and the reconstructed three-dimensional coordinate points are based on the coordinate system of the left camera of the current frame (generally, the reconstructed depth information is based on the coordinate system of the left camera).

[0069] Specifically, based on the 3D coordinate points of the reconstructed auxiliary feature points of each frame and the distance distribution information of the auxiliary feature points based on the adjacent frames (the distance between the coordinate points of every two auxiliary feature points can be calculated), the same auxiliary feature points captured in the adjacent frames are searched for, and the auxiliary feature points are stitched to obtain all the 3D coordinate points. That is, the 3D coordinate points reconstructed in every single frame are merged into the whole frame points, i.e., all the 3D coordinate points, according to the distance constraint condition, and all the 3D coordinate points are based on the coordinate system (world coordinate system) of the whole tooth model.

[0070] In step 303, each 3D coordinate point is projected into the image coordinate system to obtain a 2D pixel coordinate point. When the Euclidean distance between the 2D pixel coordinate point and the 2D coordinate point corresponding to the 3D coordinate point is the smallest, the 3D coordinate point is set as the target 3D coordinate point.

[0071] JPEG2025502852000007.jpg47170

[0072] JPEG2025502852000008.jpg13170 Here, m indicates the number of three-dimensional coordinate points, and n indicates the number of images to be processed.

[0073] In step 304, calculations are performed based on a predetermined scale factor, a target 3D coordinate point, and a distribution true value to obtain a transformation relationship of an initial position, and optimization calculations are performed on the transformation relationship of the scale factor, the target 3D coordinate point, the distribution true value, and the initial position based on a predetermined optimization formula to obtain an optimized value.

[0074] In step 305, the scale factor and the transformation relationship of the initial position are adjusted, and when the optimized value is smaller than a predetermined threshold, the transformation relationship of the corresponding initial position is taken as the transformation relationship of the target position.

[0075] JPEG2025502852000009.jpg18170 Every scanning rod can be understood as one small area) Stitching (stitching is a rigid body and does not change scale, when the scale changes, S i By performing the calculation using the scaling factor, the transformation relationship between the target positions of different scanning rods is determined. minf() refers to the nonlinear least squares algorithm. The specific formula is as shown in (2): JPEG2025502852000010.jpg7170JPEG2025502852000011.jpg46170

[0076] This allows accurate positioning of the relative positions between the scanning rods by scanning the entire oral cavity.

[0077] The scanning data processing method according to the embodiment of the present disclosure includes obtaining a plurality of frames of a target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values, obtaining two-dimensional coordinate points of the auxiliary feature points in the target image of each frame, calculating according to the relative positions between the two cameras and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the target image of each frame, stitching the three-dimensional coordinate points of the target image of each frame to obtain all three-dimensional coordinate points in the same coordinate system, projecting each three-dimensional coordinate point into an image coordinate system to obtain two-dimensional pixel coordinate points, and calculating the two-dimensional pixel coordinate points. When the Euclidean distance between the reference point and the corresponding two-dimensional coordinate point of the three-dimensional coordinate point is the smallest, the three-dimensional coordinate point is taken as the target three-dimensional coordinate point, and calculation is performed based on a predetermined scale factor, the target three-dimensional coordinate point and the distribution true value to obtain the conversion relationship of the initial position, and optimization calculation is performed for the conversion relationship between the scale factor, the target three-dimensional coordinate point, the distribution true value and the initial position based on a predetermined optimization formula to obtain an optimized value, and the conversion relationship between the scale factor and the initial position is adjusted, and when the optimization value is smaller than a predetermined threshold, the conversion relationship of the corresponding initial position is taken as the conversion relationship of the target position. Thus, the conversion relationship of the target position is determined based on the preset distribution true value of the auxiliary feature point and the three-dimensional coordinate point calculated by scanning, thereby enabling accurate positioning of the relative position between the scanning rods, and improving the efficiency and accuracy of data processing in the case of intraoral scanning.

[0078] 5 is a block diagram of a scan data processing apparatus according to an embodiment of the present disclosure, which can be realized by software and / or hardware and can be generally integrated into electronic equipment. As shown in FIG. 5, the apparatus includes: an image acquisition module 401 for acquiring a plurality of frames of a processing target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values; an image processing module 402 for performing processing based on the plurality of frames of processing target images to obtain all three-dimensional coordinate points in the same coordinate system; a measurement processing module 403 for performing measurement processing on all the three-dimensional coordinate points to obtain a target three-dimensional coordinate point; and a determination module 404 for determining a transformation relationship of the target position based on the target three-dimensional coordinate point and the distribution true value.

[0079] Preferably, the image processing module 402 specifically: acquiring two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame; calculating based on a predetermined inner parameter matrix and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the processing target image of each frame; and stitching the 3D coordinate points of the processing target image of each frame to obtain all the 3D coordinate points in the same coordinate system.

[0080] Preferably, the image processing module 402 specifically: acquiring two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame; calculating based on the relative positions between the two cameras and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the processing target image for each frame; and stitching the 3D coordinate points of the processing target image of each frame to obtain all the 3D coordinate points in the same coordinate system.

[0081] Preferably, the measurement processing module 403 specifically comprises: projecting each of the three-dimensional coordinate points into an image coordinate system to obtain two-dimensional pixel coordinate points; When the Euclidean distance between the two-dimensional pixel coordinate point and a two-dimensional coordinate point corresponding to the three-dimensional coordinate point is smallest, the three-dimensional coordinate point is set as the target three-dimensional coordinate point.

[0082] Preferably, the determination module 404 specifically: Calculating based on a predetermined scale factor, the target three-dimensional coordinate point, and the distribution true value to obtain a transformation relationship of an initial position; performing an optimization calculation for a transformation relationship between the scale factor, the target three-dimensional coordinate point, the distribution true value, and the initial position based on a predetermined optimization formula, and obtaining an optimization value; The transformation relationship between the scale factor and the initial position is adjusted, and when the optimization value is smaller than a predetermined threshold, the corresponding transformation relationship of the initial position is acquired as the transformation relationship of the target position.

[0083] Preferably, the image acquisition module 401 specifically comprises: The scanning device is controlled to rotate in a predetermined direction and scan the target oral cavity including the scanning rod in accordance with a predetermined frequency, and is used to obtain the multiple frames of images to be processed.

[0084] The scan data processing apparatus according to the embodiments of the present disclosure can execute the scan data processing method according to any embodiment of the present disclosure, and has beneficial effects with corresponding functional modules for executing the method.

[0085] An embodiment of the present disclosure further provides a computer program product, comprising computer programs / instructions which, when executed by a processor, implement the scan data processing method according to any embodiment of the present disclosure.

[0086] FIG. 6 is a configuration diagram of an electronic device according to an embodiment of the present disclosure. As shown in FIG. 6 specifically, a configuration diagram of an electronic device 500 for implementing an embodiment of the present disclosure is shown below. The electronic device 500 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (bullet computers), PMPs (portable multimedia players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG. 6 is merely an example and does not limit the functions and scope of use of the embodiment of the present disclosure.

[0087] 6, the electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 501 that can perform various suitable operations and processes based on programs stored in a read-only memory (ROM) 502 or programs loaded from a storage device 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data necessary for the operation of the electronic device 500 are further stored. The processing unit 501, the ROM 502, and the RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0088] In general, the I / O interface 505 may be connected to input devices 506 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc., output devices 507 including, for example, a liquid crystal display (LCD), a speaker, an oscillator, etc., storage devices 508 including, for example, a magnetic tape, a hard disk, etc., and a communication device 509. The communication device 509 allows the electronic device 500 to communicate wirelessly or via wires with other devices to exchange data. Although FIG. 6 shows the electronic device 500 with various devices, it should be understood that it is not required to implement or include all the devices shown. Alternatively, more or less devices may be implemented or included.

[0089] In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts may be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product including a computer program carried on a non-transitory computer readable medium, the computer program including program code for performing the method illustrated in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, it performs the functions described above that are specific to the scan data processing method of the embodiment of the present disclosure.

[0090] It should be explained that the computer readable medium of the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the above. The computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer readable storage medium may include, but are not limited to, an electrical connection having one or more conductors, a portable computer magnetic disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact magnetic disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, the computer readable storage medium may be any tangible medium that contains or stores a program, which may be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer readable signal medium may include a data signal propagated in baseband or as part of a carrier, the data signal carrying a computer readable program code. Such a propagated data signal may take a variety of forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination of the above. A computer readable signal medium may be any computer readable medium other than a computer readable storage medium, the computer readable signal medium may be used to transmit, propagate, or transmit a program used by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer readable medium may be transmitted by any suitable medium, including, but not limited to, conductive wire, fiber optic cable, RF (radio frequency), or the like, or any combination of the above.

[0091] In some embodiments, the clients and servers may communicate using any now known or later developed network protocol, such as, for example, HyperText Transfer Protocol (HTTP), and may be connected to each other (e.g., a communications network) in communication with digital data in any form or medium. Examples of communications networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), and any now known or later developed networks.

[0092] The computer-readable medium may be included in the electronic device, or may be separate from and not located in the electronic device.

[0093] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device receives user information and displays a trigger operation during the playback of a video, obtains at least two target information related to the video, and displays a first target information of the at least two target information in an information display area of ​​the playback page of the video, where a size of the information display area is smaller than a size of the playback page, and receives a first switching trigger operation from a user and switches the first target information displayed in the information display area to a second target information of the at least two target information.

[0094] Computer program code for carrying out the operations of the present disclosure may be written in one or more programming languages ​​or combinations thereof, including, but not limited to, the above programming languages, object-oriented programming languages ​​such as Java, Smalltalk, C++, and even conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a single independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. When a remote computer is involved, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0095] The flowcharts and block diagrams in the drawings illustrate possible system architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, program section or part of code, which includes one or more executable instructions for implementing a specified logical function. It should be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than the order noted in the drawings. For example, two successively shown blocks may in fact essentially be executed in parallel or may be executed in the reverse order, depending on the functionality involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented in a dedicated system of hardware to perform a given function or operation, or may be implemented in a combination of dedicated hardware and computer instructions.

[0096] The units described in the embodiments of the present disclosure may be implemented in the form of software or hardware, and the names of the units do not necessarily limit the units themselves in some cases.

[0097] The functionality described herein above may be performed, at least in part, by one or more hardware logic devices. For example, without limitation, exemplary types of hardware logic components that may be utilized include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), etc.

[0098] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of machine-readable storage media include an electrical connection by one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0099] According to one or more embodiments of the present disclosure, the present disclosure provides an electronic device, comprising: A processor; a memory for storing executable instructions for said processor; The processor is for reading the executable instructions from the memory and executing the instructions to implement any one of the scan data processing methods according to the present disclosure.

[0100] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium having a computer program stored therein, the computer program being adapted to perform any one of the scan data processing methods according to the present disclosure.

[0101] It should be explained that in this specification, the use of related terms such as "first" and "second" is merely intended to distinguish one entity or operation from another, and does not necessarily require or imply that any such actual relationship or sequence exists between these entities or operations. Furthermore, the technical terms "comprise", "include", "includes" or any other variations thereof cover the non-exclusive "comprise", thereby indicating that a process, method, article or device that includes a set of elements not only includes those elements, but also further includes other elements not expressly listed, or further includes elements that are specific to such process, method, article or device. Unless further limited, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device that includes the element.

[0102] The above contents are merely embodiments of the present disclosure, and are intended for those skilled in the art to understand or realize the present disclosure. A number of modifications of these embodiments can be easily realized by those skilled in the art, and the general principles defined herein can be realized in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to these embodiments shown herein, but should be accorded the widest scope consistent with the principles and novel features disclosed herein. [Industrial Applicability]

[0103] The scanning data processing method disclosed in the present disclosure determines the transformation relationship of the target position based on the preset distribution true value of the auxiliary feature points and the 3D coordinate points calculated by scanning, thereby enabling accurate positioning of the relative position between the scanning rods, improving the data processing efficiency and accuracy in the case of intraoral scanning, and having extremely high industrial applicability.

Claims

1. 1. A method for processing scan data, comprising: obtaining a plurality of frames of a processing target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values; performing processing based on the plurality of frames of processing target images to obtain three-dimensional coordinate points of auxiliary feature points in the same coordinate system; performing a measurement process on the three-dimensional coordinate points of the auxiliary feature points to obtain target three-dimensional coordinate points; determining a transformation relationship of a target position based on the target three-dimensional coordinate points and the distribution true value.

2. Performing processing based on the processing target image of the plurality of frames and acquiring all three-dimensional coordinate points in the same coordinate system acquiring two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame; calculating based on a predetermined internal parameter matrix and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the processing target image of each frame; and stitching the three-dimensional coordinate points of the image to be processed of each frame to obtain all the three-dimensional coordinate points in the same coordinate system.

3. Performing processing based on the processing target image of the plurality of frames and acquiring all three-dimensional coordinate points in the same coordinate system acquiring two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame; calculating based on the relative positions between the two cameras and the two-dimensional coordinate points to obtain three-dimensional coordinate points of the processing target image for each frame; and stitching the three-dimensional coordinate points of the image to be processed of each frame to obtain all the three-dimensional coordinate points in the same coordinate system.

4. To perform measurement processing for all three-dimensional coordinate points and obtain a target three-dimensional coordinate point, projecting each of the three-dimensional coordinate points into an image coordinate system to obtain two-dimensional pixel coordinate points; 2. The method of claim 1, further comprising: determining the three-dimensional coordinate point as the target three-dimensional coordinate point when a Euclidean distance between the two-dimensional pixel coordinate point and a two-dimensional coordinate point corresponding to the three-dimensional coordinate point is smallest.

5. Determining a transformation relationship of a target position based on the target three-dimensional coordinate point and the distribution true value, Calculating based on a predetermined scale factor, the target three-dimensional coordinate point, and the distribution true value to obtain a transformation relationship of an initial position; performing an optimization calculation for a transformation relationship between the scale factor, the target three-dimensional coordinate point, the distribution true value, and the initial position based on a predetermined optimization formula, and obtaining an optimization value; 2. The scan data processing method of claim 1, further comprising: adjusting the transformation relationship between the scale factor and the initial position; and acquiring the corresponding transformation relationship of the initial position as the transformation relationship of the target position when the optimization value is smaller than a predetermined threshold value.

6. Acquiring a plurality of frames of a processing target image including auxiliary feature points includes:

2. The method of claim 1, further comprising controlling a scanning device to rotate in a predetermined direction and scan a target oral cavity including a scanning rod at a predetermined frequency, thereby acquiring the plurality of frames of images to be processed.

7. 1. A scanning data processing apparatus comprising: an image acquisition module for acquiring a plurality of frames of a processing target image including auxiliary feature points, where the auxiliary feature points have corresponding distribution true values; an image processing module for performing processing based on the plurality of frames of processing target images to obtain all three-dimensional coordinate points in the same coordinate system; a measurement processing module for performing measurement processing on all the three-dimensional coordinate points to obtain a target three-dimensional coordinate point; a determination module for determining a transformation relationship of a target position based on the target three-dimensional coordinate points and a distribution true value.

8. Specifically, the image processing module includes: acquiring two-dimensional coordinate points of auxiliary feature points in the processing target image of each frame; calculating, based on a predetermined parameter matrix, a camera attitude of a current frame, and the two-dimensional coordinate points, to obtain three-dimensional coordinate points of the processing target image of each frame; and stitching three-dimensional coordinate points of the processing target image of each frame to obtain all of the three-dimensional coordinate points.

9. An electronic device, A processor; a memory for storing executable instructions for said processor; The electronic device, characterized in that the processor is for reading the executable instructions from the memory and executing the instructions to realize the scan data processing method according to any one of claims 1 to 6.

10. A computer-readable storage medium having a computer program stored therein, the computer program being for executing the scan data processing method according to any one of claims 1 to 6.

Images