Artificial intelligence-based pneumatic gingival retraction control method and intraoral three-dimensional scanner using same

Abstract

Disclosed in the present invention are an artificial intelligence-based pneumatic gingival retraction control method and an intraoral three-dimensional scanner using the same, applied to the technical field of oral medicine. The method comprises: acquiring an intraoral scanning image and a corresponding air outlet tube angle and an air blowing intensity, and labeling whether a gingiva is effectively blown open to obtain a training set; constructing an improved convolutional neural network structure, and training the training set to obtain a pneumatic gingival retraction control output model, wherein each convolutional layer comprises: a convolutional layer, an attention module, and a spatial pyramid pooling layer which are connected in sequence, and an output of the first convolutional layer and an output of the last convolutional layer are connected by means of a residual connection; and using a Laplace operator to calculate a second derivative of an image to be tested, determining a definition of said image, inputting said image into the pneumatic gingival retraction control output model to obtain an air outlet tube angle and an air blowing intensity, and performing pneumatic gingival retraction control. The present invention achieves the autonomous control of the optimal air blowing angle and airflow intensity during the pneumatic gingival retraction process.

Description

An air-pumping gingival retraction control method based on artificial intelligence and an intraoral three-dimensional scanner using the control method TECHNICAL FIELD

[0001] The present application relates to the technical field of stomatology, in particular to an air-pumping gingival retraction control method based on artificial intelligence and an intraoral three-dimensional scanner using the control method. BACKGROUND

[0002] Currently, the method of obtaining intraoral three-dimensional scanning data of the tooth preparation edge by using air-pumping gingival retraction has begun to be tested in the clinic. This method can effectively blow away the loose gingival tissue and obtain a clear tooth preparation edge shape.

[0003] As a device for three-dimensional scanning and measurement in the field of dentistry using digital imaging technology, the intraoral three-dimensional scanner mainly records the data of the tooth and gum surface to make a digital impression, and replicates the intraoral situation to the outside for restoration fabrication, implant site design, orthodontic treatment planning, etc. Currently, the air-pumping gingival retraction technology relies on the intraoral three-dimensional scanner. During the use of the air-pumping gingival retraction intraoral three-dimensional scanner, the blowing direction, air flow intensity, and blowing hole distance have a great influence on the distance of the gingiva blown away. A short distance, high flow rate, and appropriate angle can maximize the blowing of the gingival tissue and fully expose the tooth preparation edge shape, so that the intraoral three-dimensional scanner can more easily capture the preparation edge information and obtain higher quality edge shape data. However, the blowing hole of the air-pumping gingival retraction three-dimensional intraoral scanner is fixed, and the operator needs to try multiple times to find the appropriate blowing angle based on experience, which can fully expose the preparation edge. This brings inconvenience to clinical operation. Moreover, the air flow can cause the mirror to be splashed with saliva and blood, affecting the scanning quality, and the operator needs to clean it in time, greatly increasing the clinical operation time.

[0004] Therefore, how to provide an air-pumping gingival retraction control method based on artificial intelligence that can automatically control the optimal blowing angle and air flow intensity during the air-pumping gingival retraction process, reduce the inconvenience of clinical operation, and effectively improve the precision of intraoral scanning images is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides an air-pumping gingival retraction control method based on artificial intelligence and an intraoral three-dimensional scanner using the control method.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] An air-pumping gingival retraction control method based on artificial intelligence, comprising:

[0008] Step 1: Obtain intraoral scan images and corresponding air tube angles and blowing intensities of the intraoral scan images, and perform labeling of whether the gums are effectively blown open, to obtain a training set;

[0009] Step 2: Construct an improved convolutional neural network structure and train the training set to obtain an aerodynamic gingival retraction control output model; wherein each convolutional layer of the improved convolutional neural network structure comprises: a convolutional layer, an attention module and a spatial pyramid pooling layer connected in turn, and the output of the first convolutional layer is connected with the output of the last convolutional layer through a residual connection;

[0010] Step 3: Calculate the second derivative of the to-be-tested image using the Laplacian operator to judge the definition of the to-be-tested image, input the clear to-be-tested image into the aerodynamic gingival retraction control output model to obtain the air tube angle and blowing intensity that make the gums effectively blown open, and perform aerodynamic gingival retraction control.

[0011] Optionally, in step 1, before constructing the training set based on the labeled intraoral scan images, it further includes: performing image gray scale transformation and histogram correction on the labeled intraoral scan images, and converting the color picture into a gray scale image using binaryzation.

[0012] Optionally, in step 2, the convolutional layer is as follows:

[0013] Wherein, y is the output of the convolution calculation; i and j are the positions of the output; k is the size of the convolution kernel; m and n are the positions of the convolution kernel; x is the input of the convolution calculation; w is the convolution kernel.

[0014] Optionally, in step 2, the attention module is as follows:

[0015] Attention(x)=σ(MLP(AvgPool(x))+MLP(MaxPool(x)));

[0016] Wherein, Attention is the output of the attention; sigma is the sigmoid function; MLP is the fully connected layer; AvgPool and MaxPool are the average pooling and maximum pooling operations respectively; x is the input of the attention.

[0017] Optionally, in step 3, the second derivative of the to-be-tested image is calculated using the Laplacian operator to judge the definition of the to-be-tested image, which is specifically:

[0018] The input color image is converted into a gray image, and the gray image is processed using the Laplacian operator to calculate the second derivative, as follows:

[0019] The variance operation is performed on the Laplace transform result, and it is judged whether the variance operation result is greater than a preset threshold value, if greater, it represents that the image is clear, and if less, it represents that the image is blurred.

[0020] Optionally, in step 3, after the air-driven gingival retraction control based on the air outlet pipe angle and the blowing strength output by the air-driven gingival retraction control output model, the method further comprises: detecting whether the shoulder platform preparation body clearly exposes the shoulder platform edge line, the jaw gingival depth of the gingiva blown open at the shoulder platform part, and performing adaptive adjustment and supplementary scanning of the air outlet pipe angle and the blowing strength based on the air outlet pipe angle and the blowing strength output by the air-driven gingival retraction control output model until the complete subgingival edge is obtained.

[0021] Optionally, the method further comprises: updating the air-driven gingival retraction control output model in real time based on the training set and the supplementary scanning data.

[0022] The application also provides an intraoral three-dimensional scanner using an air-driven gingival retraction control method based on artificial intelligence, comprising: a self-adjustable blowing pipe, a mirror lens, a blowing pipe direction control device, a blowing pipe flow control device, a scanner main lens, an integrated control module, an external independent air pump, and an automatic windshield wiper.

[0023] The integrated control module controls the scanner main lens to obtain the intraoral scanning image reflected by the mirror lens, and calculates the second derivative of the intraoral scanning image using a Laplace operator to judge the definition of the intraoral scanning image; when the intraoral scanning image is blurred, the integrated control module controls the direction of the self-adjustable blowing pipe to be aligned with the mirror lens through the blowing pipe direction control device, and controls the blowing flow of the self-adjustable blowing pipe through the air outlet pipe flow control device, and cooperates with the automatic windshield wiper to clean the mirror lens.

[0024] The integrated control module processes the clear intraoral scanning image using a pre-trained air-driven gingival retraction control output model to obtain the air outlet pipe angle and the blowing strength that effectively blow open the gingiva, and controls the air-driven gingival retraction of the self-adjustable blowing pipe through the blowing pipe direction control device and the air outlet pipe flow control device to obtain the subgingival edge information of the tooth preparation body.

[0025] The external independent air pump is connected with the blowing pipe flow control device to provide a compressed air source.

[0026] Optionally, the integrated control module is further configured to perform the pneumatic gingival retraction control based on the air outlet pipe angle and the air blowing strength output by the pneumatic gingival retraction control output model, and after obtaining the subgingival margin information, detect whether the shoulder preparation clearly exposes the shoulder margin line and the depth of the gingival opening in the gingival direction of the gingival blowing part, and perform adaptive adjustment and supplementary scanning of the air outlet pipe angle and the air blowing strength based on the air outlet pipe angle and the air blowing strength output by the pneumatic gingival retraction control output model until the complete subgingival margin is obtained.

[0027] Optionally, the method further comprises: updating the pneumatic gingival retraction control output model in real time based on the training set and the supplementary scanning data.

[0028] Compared with the prior art, the application provides a pneumatic gingival retraction control method based on artificial intelligence and an intraoral three-dimensional scanner using the control method. By training the pneumatic gingival retraction control output model based on artificial intelligence, the optimal air blowing angle and air flow strength in different states during the pneumatic gingival retraction process are output, the autonomous control of the optimal air blowing angle and air flow strength during the pneumatic gingival retraction process is realized, the inconvenience of clinical operation is reduced, and the precision of the intraoral scanning image is effectively improved. By calculating the second derivative of the to-be-detected image based on the Laplace operator to detect the definition of the intraoral scanning image, real-time detection and processing of whether the reflection lens is affected by splashed saliva and blood during the pneumatic gingival retraction process is realized, and the acquisition precision of the intraoral scanning image is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0030] Fig. 1 is a flowchart of the method of the present application.

[0031] Fig. 2 is a schematic diagram of the improved convolutional neural network structure of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0033] Embodiment 1:

[0034] Embodiment 1 of the present application discloses a pneumatic gingival retraction control method based on artificial intelligence, as shown in FIG. 1, which comprises:

[0035] Step 1: Obtain intraoral scan images and the corresponding air outlet pipe angles and air blowing intensities (including various situations of gingival state, such as being blown open and not being blown open, gingival thickness, whether there is gingival crevicular fluid blood, etc.), and label whether the gingiva is effectively blown open, to obtain a training set.

[0036] Before constructing the training set based on the labeled intraoral scan images, it also includes: performing image gray scale transformation and histogram correction on the labeled intraoral scan images to reduce the influence of noise, and using binarization to convert color pictures into gray scale pictures to reduce the amount of calculation and facilitate subsequent algorithm extraction of gingival state related key features such as edges, colors, textures and shapes, etc.

[0037] Step 2: Construct an improved convolutional neural network structure as shown in FIG. 2, and train the training set to obtain a pneumatic gingival retraction control output model; wherein each convolutional layer of the improved convolutional neural network structure comprises: a convolutional layer, an attention module (in the convolution process, the jaw gingiva in the specific area (shoulder platform part) needs to be focused on, so the attention module (Attention) is added, which gives more weight to important features in the process of image convolution feature extraction, thereby improving the network prediction effect) and a spatial pyramid pooling layer (considering that observing the gingival image at different scales may have different effects, the spatial pyramid pooling (SPP) is introduced to aggregate the context information of different scales, so that the model can better understand and recognize the features in the image), and the output of the first convolutional layer is connected with the output of the last convolutional layer through a residual connection (which helps the model to learn more complex features).

[0038] It also includes: testing the model performance using test set data, and adjusting the model structure or parameters as needed. Then performance evaluation is carried out, and the accuracy and generalization ability of the model are evaluated using the validation data set that does not participate in the training. The precision (Precision) index is used to evaluate the performance of the model. According to the results of performance evaluation, the learning rate, optimizer and other parameters are adjusted.

[0039] Convolutional layer, as follows:

[0040] Wherein y is the output of the convolution calculation; i and j are the positions of the output; k is the size of the convolution kernel; m and n are the positions of the convolution kernel; x is the input of the convolution calculation; w is the convolution kernel.

[0041] Attention module, as follows: Attention(x)=σ(MLP(AvgPool(x))+MLP(MaxPool(x)));

[0042] wherein Attention is the output of attention; σ is the sigmoid function; MLP is the fully connected layer; AvgPool and MaxPool are the average pooling and max pooling operation respectively; x is the input of attention. Through the pooling operation, the global maximum and average value in the image are extracted, and then they are fused through the fully connected layer to obtain an attention coefficient. The coefficient will weight the input feature map, emphasizing the most useful information, thereby improving the performance of the model.

[0043] Step 3: Calculate the second derivative of the image to be tested using the Laplacian operator to judge the sharpness of the image to be tested, input the clear image to be tested into the pneumatic gingival retraction control output model, and obtain the air outlet pipe angle and air blowing strength that can effectively blow open the gingiva, and control the pneumatic gingival retraction.

[0044] The second derivative of the image to be tested is calculated using the Laplacian operator to judge the sharpness of the image to be tested, specifically:

[0045] The input color image is converted into a gray image, and the gray image is processed using the Laplacian operator to calculate the second derivative, as follows:

[0046] The variance operation is performed on the Laplace transform result, and the greater the variance, the more high-frequency components there are in the image, i.e. the clearer the image; the smaller the variance, the more blurred the image, and it is determined whether the variance operation result is greater than a preset threshold value, if greater, it represents that the image is clear, and if less, it represents that the image is blurred.

[0047] By using the Laplacian operator to perform real-time processing on the sharpness of the image, it can be detected in real time whether the reflection lens is affected by spitting saliva and blood during the actual pneumatic gingival retraction process, so as to be cleaned in time and improve the accuracy of intraoral scanning image acquisition.

[0048] After the pneumatic gingival retraction control is performed based on the air outlet pipe angle and air blowing strength output by the pneumatic gingival retraction control output model, it further includes: detecting whether the shoulder platform preparation body clearly exposes the shoulder platform edge line and the depth of the gingiva in the jaw is blown open in the shoulder platform part based on the intraoral scanning image after the pneumatic gingival retraction control, and performing adaptive adjustment and supplementary scanning of the air outlet pipe angle and air blowing strength based on the air outlet pipe angle and air blowing strength output by the pneumatic gingival retraction control output model, until the complete subgingival edge is obtained.

[0049] It further includes: updating the pneumatic gingival retraction control output model in real time based on the training set and the supplementary scanning data.

[0050] Example 2:

[0051] The embodiment 2 of the present application discloses an intraoral three-dimensional scanner using the artificial intelligence-based pneumatic gingival retraction control method, comprising: an autonomously controllable air blowing pipe, a mirror lens, an air blowing pipe direction control device, an air blowing pipe flow control device, a scanner main lens, an integrated control module, an external independent air pump, and an automatic windshield wiper.

[0052] The integrated control module controls the scanner main lens to obtain the intraoral scanning image reflected by the mirror lens, calculates the second derivative of the intraoral scanning image by using the Laplacian operator, and judges the definition of the intraoral scanning image; when the intraoral scanning image is blurred, the integrated control module controls the direction of the autonomously controllable air blowing pipe to be aligned with the mirror lens through the air blowing pipe direction control device, and controls the air blowing flow of the autonomously controllable air blowing pipe through the air blowing pipe flow control device, and cooperates with the automatic windshield wiper to clean the mirror lens.

[0053] The integrated control module processes the clear intraoral scanning image by using the pre-trained pneumatic gingival retraction control output model, obtains the air blowing pipe angle and the air blowing strength that effectively blow open the gingiva, and controls the pneumatic gingival retraction of the autonomously controllable air blowing pipe through the air blowing pipe direction control device and the air blowing pipe flow control device to obtain the subgingival margin information of the tooth preparation body.

[0054] The external independent air pump is connected with the air blowing pipe flow control device, and is used to provide a compressed air source.

[0055] The integrated control module is also used to, after the pneumatic gingival retraction control based on the air blowing pipe angle and the air blowing strength output by the pneumatic gingival retraction control output model and the subgingival margin information of the tooth preparation body are obtained, detect whether the shoulder preparation body clearly exposes the shoulder edge line and the depth of the gingival blowing of the gingiva in the jaw, and based on the air blowing pipe angle and the air blowing strength output by the pneumatic gingival retraction control output model, perform adaptive adjustment and supplementary scanning of the air blowing pipe angle and the air blowing strength until the complete subgingival margin is obtained.

[0056] It also includes: updating the pneumatic gingival retraction control output model in real time based on the training set and the supplementary scanning data.

[0057] It also includes: providing a user interface to allow the doctor to manually adjust the air flow strength when needed. Manual and automatic mode switching: allowing the doctor to switch between automatic and manual modes to cope with different clinical situations.

[0058] The embodiment of the application discloses a kind of based on artificial intelligence's pneumatic gingival control method and utilize the intraoral three-dimensional scanner of this control method. By based on artificial intelligence, training obtains pneumatic gingival control output model, outputs the best blowing angle and airflow intensity in different state conditions in pneumatic gingival process, realizes the best blowing angle and airflow intensity in pneumatic gingival process The autonomous control of intensity, reduce the inconvenience of clinical operation, and effectively improve the intraoral scanning image precision.Meanwhile, by calculating the second derivative of the image to be measured based on Laplace operator, the intraoral scanning image definition is detected, the real-time detection and processing of whether the mirror lens is affected by spitting saliva and blood in the pneumatic gingival process is realized, and the acquisition accuracy of the intraoral scanning image is further improved.

[0059] Each embodiment in the specification is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between each embodiment can be referred to each other. For the device disclosed by the embodiment, since it corresponds to the method disclosed by the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0060] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An artificial intelligence-based pneumatic gingival retraction control method, characterized by, The method comprises the following steps: Step 1: obtaining an intraoral scanning image and corresponding air tube angle and air blowing strength of the intraoral scanning image, and labeling whether the gingiva is effectively blown open to obtain a training set; Step 2: constructing an improved convolutional neural network structure and training the training set to obtain an air-driven gingival retraction control output model; wherein each convolutional layer of the improved convolutional neural network structure comprises a convolutional layer, an attention module and a spatial pyramid pooling layer connected in sequence, and the output of the first convolutional layer is connected with the output of the last convolutional layer through a residual connection; Step 3: calculating the second derivative of the to-be-tested image using Laplace operator to judge the definition of the to-be-tested image, inputting the clear to-be-tested image into the air-driven gingival retraction control output model to obtain the air tube angle and air blowing strength that make the gingiva effectively blown open, and performing air-driven gingival retraction control.

2. The artificial intelligence-based pneumatic gingival retraction control method according to claim 1, characterized by, In step 1, before constructing the training set based on the labeled intraoral scanning image, it further comprises: performing image gray scale transformation and histogram correction on the labeled intraoral scanning image, and converting the color picture into a gray scale picture using binaryzation.

3. The artificial intelligence-based pneumatic gingival retraction control method according to claim 1, characterized by, In step 2, the convolutional layer is as follows: Wherein, y is the output of convolution calculation; i and j are the positions of the output; k is the size of the convolution kernel; m and n are the positions of the convolution kernel; x is the input of convolution calculation; w is the convolution kernel.

4. The artificial intelligence-based pneumatic gingival retraction control method according to claim 1, characterized by, In step 2, the attention module is as follows: Attention(x)=σ(MLP(AvgPool(x))+MLP(MaxPool(x))); Wherein, Attention is the output of attention; sigma is the sigmoid function; MLP is the full connection layer; AvgPool and MaxPool are the average pooling and maximum pooling operations respectively; x is the input of attention.

5. The artificial intelligence-based pneumatic gingival retraction control method according to claim 1, characterized by, In step 3, the second derivative of the to-be-tested image is calculated using Laplace operator to judge the definition of the to-be-tested image, which is specifically: The input color image is converted into a gray image, and the gray image is processed using a Laplacian operator to calculate a second derivative, as follows: Performing variance operation on the Laplace transform result, and judging whether the variance operation result is greater than a preset threshold, if greater, it represents that the image is clear, if less, it represents that the image is blurred.

6. The artificial intelligence-based pneumatic gingival retraction control method according to claim 1, wherein, In step 3, after performing air-driven gingival retraction control based on the air tube angle and air blowing strength output by the air-driven gingival retraction control output model, it further comprises: detecting whether the shoulder platform preparation body clearly exposes the shoulder platform edge line, the depth of the gingiva blown open in the shoulder platform part, and performing adaptive adjustment and scanning of the air tube angle and air blowing strength based on the air tube angle and air blowing strength output by the air-driven gingival retraction control output model until the complete subgingival edge is obtained.

7. The artificial intelligence-based pneumatic gingival retraction control method according to claim 6, characterized by, Further comprising: Real-time updating the air-driven gingival retraction control output model based on the training set and the scanning data.

8. An intraoral three-dimensional scanner using the artificial intelligence-based pneumatic gingival retraction control method according to any one of claims 1 to 7, characterized in that, Comprise: Self-controlling air blowing pipe, reflector lens, air blowing pipe direction control device, air blowing pipe flow control device, scanner main lens, integrated control module, external independent air pump, automatic wiper; The integrated control module controls the scanner main lens to obtain the intraoral scanning image reflected by the reflector lens, and calculates the second derivative of the intraoral scanning image by using a Laplacian operator to determine the sharpness of the intraoral scanning image; when the intraoral scanning image is blurred, the integrated control module controls the direction of the self-regulating blow pipe by the blow pipe direction control device to align the direction of the self-regulating blow pipe with the reflector lens, and controls the blow flow of the self-regulating blow pipe by the blow pipe flow control device, and cooperates with the automatic windshield wiper to clean the reflector lens; The integrated control module processes the clear intraoral scanning image by using a pre-trained pneumatic gingival retraction control output model to obtain the blow pipe angle and blow strength that can effectively blow open the gingiva, and controls the self-regulating blow pipe by the blow pipe direction control device and the blow pipe flow control device to obtain the subgingival margin information of the tooth preparation body; The external independent air pump is connected with the blow pipe flow control device to provide a compressed air source.

9. An intraoral three-dimensional scanner according to claim 8, wherein, The integrated control module is also used to perform the detection of whether the shoulder preparation body clearly exposes the shoulder edge line and the depth of the gingival direction of the gingiva blown open at the non-shoulder position after the pneumatic gingival retraction control based on the blow pipe angle and the blow strength output by the pneumatic gingival retraction control output model and the acquisition of the subgingival margin information of the tooth preparation body, and to perform the adaptive adjustment and supplementary scanning of the blow pipe angle and the blow strength based on the blow pipe angle and the blow strength output by the pneumatic gingival retraction control output model until the complete subgingival margin is obtained.

10. An intraoral three-dimensional scanner according to claim 9, wherein, Also comprising: Based on the training set and the supplementary scanning data, the pneumatic gingival retraction control output model is updated in real time.

Images