Angle's classification-based automatic recognition method, electronic device, and medium

Abstract

An Angle's classification-based automatic recognition method, an electronic device, and a medium. The automatic recognition method comprises the following steps that are automatically executed by the electronic device: obtaining tooth image data to be recognized, wherein the tooth image data to be recognized comprises a two-dimensional photo and / or a three-dimensional dental model; and using a trained neural network model for use in Angle's classification to recognize the tooth image data to be recognized, and determining an Angle's classification result of a patient, wherein first-type tooth image data used by the trained neural network model for use in Angle's classification at least comprises a first maxillary molar and a first mandibular molar of the patient, and Angle's classification can be automatically performed on the jaw of the patient, so that the Angle's classification is more accurate and efficient, thereby further providing a more accurate and credible classification result for doctors or professionals.

Description

Angle classification automatic identification method, electronic device and medium CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application is based on the Chinese patent application with application number "202410338132.9" and application date of March 22, 2024, and claims the priority of the above-mentioned Chinese patent application. The entire content of the above-mentioned Chinese patent application is hereby incorporated into this application by introduction. Technical Field

[0002] The embodiments of the present application relate to the technical field of digital design of medical devices, and in particular to an automatic recognition method for Angle's classification, an electronic device, and a medium. Background Art

[0003] Malocclusion refers to the developmental deformities of the teeth, jaws, and face during a child's growth and development, caused by various factors such as genetics and the environment. (An ideal normal jaw refers to the presence of all upper and lower teeth, neatly arranged, with good cusp-to-fossa contact, normal jaw development, and good dental arch morphology), which affects the beauty and health of the face. When orthodontists determine the corrective device for patients with malocclusion, they generally first determine the type of malocclusion and then use the corresponding corrective device. Therefore, the accuracy of the classification can significantly improve the accuracy of subsequent corrective device selection.

[0004] The Angle classification is a widely used malocclusion classification system proposed by Dr. Angle in 1899. It primarily uses the maxillary first permanent molar as a reference point and is based on the anterior-posterior relationship between the upper and lower dental arches. Currently, this system relies primarily on the physician's judgment, which is limited by experience and can lead to a certain degree of error. This incorrect judgment can lead to incorrect selection of orthodontic devices, compromising patient treatment outcomes.

[0005] It can be seen that we need a more accurate and efficient method for identifying malocclusion, so as to more efficiently and accurately identify the type of malocclusion in patients and provide doctors with a more reliable reference basis. Summary of the Invention

[0006] The purpose of the embodiments of the present application is to provide an automatic Angle classification recognition method, electronic equipment and medium, which can automatically perform Angle classification on the patient's jaw, making Angle classification more accurate and efficient, and further providing doctors or professionals with more accurate and reliable classification results.

[0007] To solve the above technical problems, an embodiment of the present application provides an automatic Angle classification recognition method, which includes the following steps automatically performed by an electronic device: obtaining tooth image data to be recognized, the tooth image data to be recognized including: two-dimensional photos and / or three-dimensional dental models; using a trained neural network model for Angle classification to recognize the tooth image data to be recognized, and determining the patient's Angle classification result; wherein, before using the trained neural network model for Angle classification to recognize the tooth image data to be recognized, the method includes: collecting a first type of tooth image data as a first type of sample image, saving the first type of sample image and the corresponding Angle classification label to form a first sample set; constructing a neural network model; using the first sample set to train the constructed neural network model to obtain the trained neural network model for Angle classification; wherein, the first type of tooth image data includes at least the patient's maxillary first molars and mandibular first molars.

[0008] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned Angle's classification automatic identification method.

[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned Angle's classification automatic recognition method when executed by a processor.

[0010] The automatic Angle's classification recognition method in the embodiments of the present application uses an electronic device to automatically determine the Angle's classification corresponding to a patient's teeth based on the tooth image data to be identified. The electronic device automatically recognizes the tooth image data using a trained neural network model, which is simple, convenient, and fast, with low operational difficulty. Furthermore, the recognition results obtained by analyzing the features of the maxillary first molars and mandibular first molars in the tooth image data using a machine learning algorithm are incorporated into the automatic Angle's classification, making the recognition results more accurate, providing doctors or professionals with a more accurate and reliable basis for judgment, and improving the accuracy and credibility of the orthodontic devices selected based on the classification results. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0012] FIG1 is a flow chart of an automatic identification method of Angle's classification according to an embodiment of the present application;

[0013] FIG2 is a schematic diagram of a side intraoral photograph used in an automatic identification method for Angle's classification according to an embodiment of the present application;

[0014] FIG3 is a schematic diagram of the structure of a neural network model used in an automatic recognition method of Andrzej Kroeberman classification according to an embodiment of the present application;

[0015] FIG4 is a flow chart of an automatic identification method of Angle's classification according to another embodiment of the present application;

[0016] FIG5 is a schematic diagram of the structure of a neural network model constructed in an automatic recognition method for Andreas classification according to another embodiment of the present application;

[0017] FIG6 is a schematic diagram of a tooth image after tooth separation in an automatic recognition method of Angle's classification according to another embodiment of the present application;

[0018] FIG7 is a schematic diagram of a tooth image after pixel changes in an automatic recognition method for Angle's classification according to another embodiment of the present application;

[0019] FIG8 is a schematic diagram of a tooth image after cropping a region block and changing pixels in an automatic recognition method for Angle's classification according to another embodiment of the present application;

[0020] FIG9 is a schematic diagram of a three-dimensional dental model used in an automatic identification method of Angle classification according to another embodiment of the present application;

[0021] FIG10 is a schematic diagram of the maxillary first molar and the mandibular first molar segmented from the three-dimensional dental model shown in FIG9 ;

[0022] FIG11 is a schematic diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0024] The "anterior region" and "posterior region" mentioned in various embodiments of this application are defined according to the classification of teeth in the second edition of "Introduction to Stomatology" published by Peking University Medical Press, pages 36-38. The posterior region includes premolars and molars, which are represented as teeth 4-8 using the FDI (Fédération Dentaire Internationale, World Dental Federation) notation. The anterior region is represented as teeth 1-3 using the FDI notation, and the teeth in the anterior region include central incisors, lateral incisors, and canines.

[0025] One embodiment of the present application relates to an automatic identification method for Andr's classification. The specific process of the automatic identification method for Andr's classification of this embodiment may be shown in FIG1 , including:

[0026] Step 101: collect first-category dental image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels, and form a first sample set. The first-category dental image data includes at least the patient's maxillary first molars and mandibular first molars.

[0027] Step 102: construct a neural network model, and use the first sample set to train the constructed neural network model to obtain a trained neural network model for Andrzej Kroeberman classification.

[0028] Step 103: Acquire the tooth image data to be identified, wherein the tooth image data to be identified includes: a two-dimensional photo and / or a three-dimensional dental model.

[0029] Step 104: Use the trained neural network model to identify the tooth image data to be identified, and determine the patient's Angle's classification result.

[0030] Through the above steps 101 to 104, the recognition results obtained by analyzing the features of the maxillary first molars and the mandibular first molars in the dental image data using a machine learning algorithm, and introducing the machine learning algorithm into the automatic Angle classification make the recognition results more accurate, provide doctors or professionals with a more accurate and reliable judgment basis, and improve the accuracy and credibility of the orthodontic devices selected based on the classification results.

[0031] The following is a detailed description of the implementation details of the automatic identification method of the Andersen classification of this embodiment. The following content is only provided for ease of understanding and is not necessary for implementing this solution.

[0032] It should be noted that the automatic Angle's classification recognition method in this embodiment can be implemented through hardware or a combination of computer software and hardware. Regarding hardware implementation, the automatic Angle's classification recognition method can be implemented through one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the automatic Angle's classification recognition function, or a combination of these devices.

[0033] In step 101, the first type of dental image data is collected as the first type of sample image. The first type of dental image data can be obtained from the patient's historical case records or from a dedicated database, which will not be listed here one by one. Among them, the first type of dental image data includes at least the patient's maxillary first molar and mandibular first molar. In this embodiment, the first type of dental image data is a two-dimensional photo as an example for explanation, wherein the two-dimensional photo can be a side intraoral photo (as shown in Figure 2). Since the side intraoral photo is the data routinely collected in the early stage of orthodontic treatment, the buccal information of all posterior teeth will be collected, which can effectively perform Angle's classification recognition. Therefore, in some cases, the side intraoral photo can be limited to Angle's classification recognition, which is conducive to reducing the amount of additional data collection, simplifying the operation on the doctor's side, and facilitating the promotion of the Angle's classification recognition method in this application. In actual applications, the two-dimensional photo can also be other two-dimensional photos, such as two-dimensional photos specially collected for Angle's classification, which will not be listed here one by one.

[0034] In some embodiments, if the collected first-category sample images already carry Andersen classification labels, they can be directly saved and used. If the collected first-category sample images do not carry Andersen classification labels, the sample images can be annotated, the corresponding Andersen classification labels added, and then saved. After collecting a certain number of first-category sample images, a first sample set is formed. The specific annotation process can be determined by professionals, or a partial set of labels can be created first, a neural network model trained, and then the trained model automatically annotated, gradually expanding the number of samples in the first sample set.

[0035] In some examples, the first-category sample images can be further processed, such as by segmenting the maxillary first molars and mandibular first molars from the first-category sample images and adding background to them to form updated first-category sample images. Because Angle's classification is primarily determined by the positional relationship between the upper and lower first molars, in some examples, this can be further limited to segmenting the sample images to extract portions with higher feature correlation as independent images and adding background. For example, by setting all pixels in areas other than the upper and lower first molars to 0, this increases the difference between the background and the upper and lower first molars, effectively improving the feature focus of the data, which is beneficial for increasing the training speed of the neural network model and improving the accuracy of the training results.

[0036] In some examples, adding background to the above-mentioned maxillary first molar and mandibular first molar includes: adding multiple different backgrounds to the maxillary first molar and mandibular first molar to form multiple different first-category sample images. When the sample data is small, the sample size can be increased by adding different backgrounds to improve the accuracy of the training results. Since the sample data that can be collected in the field of orthodontics is generally small, the above-mentioned method of changing the background can maximize the number of samples while ensuring the accuracy of the sample data. Moreover, the reference features of Angle's classification are concentrated on the upper and lower first molars on the same side, so the upper and lower first molars on the same side can be segmented separately to effectively concentrate the feature areas of the sample image, which is simple to implement and has better results. It should be noted that in addition to the above-mentioned method of changing the background, the traditional sample processing methods such as rotation, translation, adding Gaussian noise, etc. can also be used to increase the sample size, which will not be listed here one by one.

[0037] In step 102, a neural network model is constructed. The constructed neural network model can be a deep learning model. In some examples, the structure of the neural network model may include: an input layer, a feature extractor, an attention network, a fully connected (i.e., fully connected, here and in the following, fully connected has the same meaning as fully connected) module, and an output layer. In some examples, as shown in FIG3 , the neural network model can extract features of the input original image (described as a “feature map” in FIG3 ) through a feature extractor of a CNN (Convolutional Neural Networks) structure (described as a “CNN feature extractor” in FIG3 ), and input the output feature map into an attention network. For example, the attention network can be an attention candidate network (APN, Attention Proposal Network), etc. FIG3 provides an example of an “attention network APN”. The APN determines the feature-related area (described as an “output attention area” in FIG3 ), and the extracted feature map is input into a classifier of a fully connected structure (described as a “fully connected module” in FIG3 ) to predict the image classification. Afterwards, according to the area given by APN, the area is cropped from the original image (i.e., the "cropping" action described in Figure 3) as the network input for the next step, and the cycle continues.

[0038] It should be noted that the above description describes the case where the constructed neural network model has one scale layer. In some examples, the neural network model constructed in this application can also have two or more scale layers. Taking Figure 3 as an example, the constructed neural network model is provided with two scale layers (the original scale layer and the cropped scale layer). The structure of each scale layer is roughly the same. The previous scale layer will crop out a more focused attention area. The next scale layer continues to operate according to the attention area output by the previous scale layer to further obtain the Angle classification result. By setting two scale layers, the feature area is cropped out by the attention network, so that the prediction of the image is concentrated on the cropped feature area, so as to speed up the training speed and improve the accuracy of the recognition and classification results. It is known to those skilled in the art that although this example is described with two scale layers as an example, in actual applications, more scale layers can be set, such as three or more layers, to further crop the cropped image and focus more on the feature-related area so that the final classification result is more accurate. Each scale layer will give a probability distribution corresponding to the three classification results in the Angle classification. Each probability output position corresponds to an Angle category, and the category corresponding to the position with the largest probability value is the Angle classification result of the module.

[0039] In some examples, an encoder (such as an encoder with a Transformer structure, which is described as a "Transformer encoder" in Figure 3) can be added to each scaling layer. For example, the original image is input into the encoder in parallel at the original scaling layer in Figure 3. In this example, the encoder is also used to extract features in parallel with the CNN feature extractor. The output of the encoder is then concatenated with the output of the CNN feature extractor and then input into the classifier (i.e., the "fully connected module" shown in Figure 3) to obtain the Andersen classification result at the current level. During subsequent model training, the Transformer structure contains a multi-head self-attention mechanism, which helps the CNN feature extractor better extract key features, allowing the APN to more efficiently find the key areas that determine the classification results of the input image.

[0040] In some examples, the above neural network model can use the following loss function Ltot during training, which consists of two parts: one is the classification loss Lcls, and the other is Lrank:

[0041]

[0042]

[0043]

[0044] Among them, Y (s) and Y * are the model prediction category and label category, It represents the class probability predicted by the model when the scale is s, t represents the Angle class label of the image, s represents the image scale, and n represents the number of scales. For example, in some examples, two scales are used, so n = 2; margin is a value set by humans to ensure than It is large enough to enable the neural network to be trained more effectively, and its specific value can be set by technicians according to actual needs.

[0045] It should be noted that in scenarios where malocclusion classification is achieved through tooth regions, although tooth images have individual characteristics, their overall morphological features are very similar. For example, the tooth region generally consists of a crown and a gingival region, and the tooth region has a curved contour. Therefore, the characteristic portion accounts for a very small proportion relative to the entire image. In addition, the image is affected by different objective factors such as the acquisition scene, the morphology and color of saliva in the mouth, and there are many interfering features. Therefore, traditional neural networks require a lot of preliminary data processing when automatically identifying teeth, which is also a major factor affecting the accuracy of automatic tooth classification. In this regard, after studying the characteristics of teeth, the inventors of this application realized that it is necessary to first identify the characteristic regions related to the classification features. The visual Tansformer itself divides the image into multiple regions during calculation, which is more suitable for solving the task of solving the problem of finding characteristic tooth regions. Moreover, since Angle's classification is mainly determined by the relative position of the upper and lower first molars, the model can focus on the region containing these two molars, which will improve the subsequent training speed and the accuracy of the trained model.

[0046] To continue the explanation, after the neural network model is constructed, the first sample set obtained in step 101 is used to train the neural network model to obtain a trained neural network model for Andrzej Krohn's classification.

[0047] It is worth mentioning that the above steps 101 and 102 are model building and training steps for obtaining a trained neural network model. Therefore, the model can be trained in advance and retrieved when recognition is required, or trained after recognition is required. In other words, the above steps 101-102 can also be set after step 103. Those skilled in the art will understand that changing the order of the steps will not change the inventive concept of this application.

[0048] In step 103, the tooth image data to be identified is obtained. The tooth image data to be identified is the tooth image data that needs to be classified by Angle's classification. It can be a two-dimensional image or a three-dimensional dental model collected by the doctor, or it can be collected separately for Angle's classification. This is not limited here.

[0049] In step 104, the trained neural network model is used to identify the dental image data to determine the patient's Angle's classification result. In some examples, the dental image data acquired in step 103 may be input into the neural network model trained in step 102 to obtain the final output Angle's classification result, which serves as the patient's Angle's classification result. Angle's classification is generally divided into three categories: Angle's Class I, Angle's Class II, and Angle's Class III.

[0050] It should be noted that, taking the above two scaling layers as an example, the Andrzej Klein classification result of the output of the cropped scaling layer can be used as the final classification result, or the output results of the two fully linked modules in the two-layer model can be merged, which can be normalized first and then merged to obtain a new set of classification probability data, which is then operated by the softmax module (i.e., the activation function layer formed based on the softmax function) to obtain the final Andrzej Klein classification result. In other words, the one with the largest probability is determined from the classification probability data as the classification result.

[0051] It is worth mentioning that in addition to the neural network model constructed above, existing neural network models can also be used, such as RA-CNN (Recurrent attention convolutional neural network), CMAL-Net (Cross-layer Mutual Attention Learning Network), ViT (Vision Transformer), SR-GNN (Spatial Relation-aware Graph Neural Network), ResNet (Residual Neural Network), DesNet (Dense Convolutional Network), Yolo (You Only Look Once) or EfficientNet network. Further limiting the available neural network model categories in the method of identifying Andrology classification, this application has more optional networks when implemented, which is convenient for selecting different neural network models for different application scenarios, so as to expand the application scope of this application and facilitate the promotion of this application.

[0052] It can be seen that the Angle classification automatic recognition method in this embodiment automatically determines the Angle classification corresponding to the patient's teeth based on the tooth image data to be identified by the electronic device. The electronic device automatically recognizes the tooth image data through the trained neural network model, which is simple, convenient, fast, and has low operation difficulty. In addition, the recognition results obtained by analyzing the features of the maxillary first molars and mandibular first molars in the tooth image data through the machine learning algorithm are introduced into the automatic Angle classification, making the recognition results more accurate, providing doctors or professionals with a more accurate and reliable judgment basis, and improving the accuracy and credibility of the orthodontic devices selected by the classification results. In addition, the sample image is segmented into a part with higher feature correlation as an independent image, and the background is added / changed, which effectively improves the feature focus of the data and is conducive to improving the training speed of the neural network model.

[0053] Another embodiment of the present application also relates to an automatic recognition method for Angle's classification. This embodiment is roughly the same as the above embodiment, with the main improvement being: adding preliminary processing of the tooth image data to be recognized, determining the parts that are more relevant to the Angle's classification features, and then using the trained neural network for recognition, reducing information interference, and making the recognition results more accurate after the neural network model analyzes the image.

[0054] The automatic Angle's classification recognition method in this embodiment is shown in FIG4 . In this embodiment, the tooth image data to be recognized is still described using a two-dimensional photo as an example. The method may include the following steps:

[0055] Step 401 : collect first-category tooth image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels, and form a first sample set.

[0056] Step 402: construct a neural network model, and use the first sample set to train the constructed neural network model to obtain a trained neural network model for Andrzej Kroeberman classification.

[0057] In some examples, the neural network model constructed in this step may include a convolutional layer, a pooling layer, and a fully connected layer (as shown in Figure 5, where the fully connected layer in Figure 5 is described as a "fully connected module" and the convolutional layer is composed of one or more convolutional modules). In some cases, the convolutional layer may include three convolutional modules, each with a different convolution kernel size (taking Figure 5 as an example, the three convolutional modules are convolutional module 1, convolutional module 2, and convolutional module 3. The convolution kernel size in convolutional module 1 is 7×7 and the number of convolution kernels is 64; the convolution kernel size in convolutional module 2 is 5×5 and the number of convolution kernels is 64; the convolution kernel size in convolutional module 3 is 3×3 and the number of convolution kernels is 128). Shallow convolutional modules extract relatively basic features, such as lines and textures in the image. As the convolutional layer deepens, it extracts more advanced and complex features. In shallow convolution, each position on the feature map corresponds to a smaller local position in the original image. As the number of convolution layers increases, the position on the feature map corresponds to a larger area in the original image. In other words, as the number of convolution layers increases, the feature extraction refers to more information in the original image.

[0058] It is understood that although the above-mentioned neural network model is illustrated using the model structure of FIG5 as an example, in actual applications, neural network models with other structures can also be used, such as networks based on CNN structures such as Resnet, DesNet, Inception, etc., networks based on Transformer structures such as ViT, etc., and models based on graph neural networks such as GAT (Graph Attention Networks, graph attention model) and SR-GNN, RA-CNN, etc., which will not be described in detail here. Further limiting the categories of available neural network models in the method of identifying the Angle classification after identifying the upper and lower first molars, this application has a large number of optional networks when implemented, which facilitates the selection of different neural network models for different application scenarios, so as to expand the scope of application of this application.

[0059] Step 403 : obtaining second-category tooth image data as second-category sample images, wherein each tooth in the second-category sample images corresponds to a tooth classification label, thereby forming a second sample set.

[0060] In some examples, the second type of dental image data can also be obtained from the patient's historical medical records or from a dedicated database. The acquisition method is roughly the same as that of the first type of dental image data and will not be repeated here.

[0061] Step 404: construct a convolutional network, and use the second sample set to train the constructed convolutional network to obtain a trained convolutional network for identifying the maxillary first molar and the mandibular first molar.

[0062] The above steps 403 to 404 are used to construct and train a network model for identifying single teeth (especially the maxillary first molars and mandibular first molars). In some examples, when making the second sample set in step 403, the first type of sample images in the first sample set obtained in step 101 can be directly used. In some examples, the sample images obtained in themselves carry tooth classification labels, that is, pixel-level tooth classification labels corresponding to single teeth. In this case, the tooth classification labels can be directly saved and used to train the convolutional network. In some examples, the sample images obtained do not directly carry tooth classification labels. In this case, the first sample image can be processed for tooth classification, and the tooth classification labels corresponding to the single teeth in the first sample image can be added. In other words, in actual applications, steps 401 and 403 can be combined to make only one sample set, which covers both the Angle classification labels in the first sample set and the tooth classification labels in the second sample set. It can be seen that the sample images in this embodiment can be used to make multiple types of labels, and the same batch of images can be used to make multiple types of labels so that they can be used in the neural network model training at different steps, reducing data usage.

[0063] That is, in some examples, obtaining the second category of dental image data as the second category of sample images can be achieved by obtaining each first category of dental image data from the first sample set as each second category of dental image data in the second sample set. In other words, the sample images used during training have two types of labels: Angle's classification labels and dental classification labels. By creating multiple types of labels for the sample images, that is, using the same batch of images to create multiple types of labels, these labels can be used in different training steps. Furthermore, because the constructed convolutional network can adopt a simple model structure and concentrated data features, training speed is fast and the requirement for the number of sample sets is relatively low.

[0064] It is worth mentioning that although the above description uses convolutional networks for instance segmentation as an example, in actual applications, instance segmentation can be performed through other methods, such as other neural network models, which are not listed here one by one.

[0065] In some embodiments, using a trained neural network model for Angle's classification to identify to-be-identified dental image data and determine the Angle's classification result can be achieved by: identifying the to-be-identified dental image data to obtain the maxillary first molars and mandibular first molars in the to-be-identified dental image data; and using the trained neural network model for Angle's classification to identify the maxillary first molars and mandibular first molars to determine the Angle's classification result. In this case, the maxillary and mandibular first molars are first identified from the dental image data, and then the patient's Angle's classification result is determined based on the first molars. Since the primary reference for Angle's classification is the maxillary and mandibular first molars, the model's recognition information is focused on the maxillary and mandibular first molars to reduce information interference, thereby making the neural network model's image analysis more accurate.

[0066] In some examples, identifying the tooth image data to be identified and obtaining the maxillary first molars and mandibular first molars from the tooth image data to be identified can be achieved by using instance segmentation to identify the tooth image data to obtain the maxillary first molars and mandibular first molars. In this case, identifying the maxillary first molars and mandibular first molars through instance segmentation makes identification more accurate. In some examples, this is achieved through steps 405 and 406.

[0067] Step 405: Acquire the tooth image data to be identified.

[0068] The specific process of this step is similar to step 103 of the aforementioned embodiment, and will not be described again here to avoid repetition.

[0069] Step 406 : Identify the tooth image data to be identified, and obtain the maxillary first molar and the mandibular first molar.

[0070] In this embodiment, the convolutional network trained in the above step 404 for identifying the maxillary first molar and the mandibular first molar is used for identification. Of course, this embodiment is only an example. For example, in some embodiments, after obtaining the tooth image data to be identified, the teeth on the tooth image data to be identified can be segmented into single teeth using the existing tooth segmentation method, and the maxillary first molar and the mandibular first molar can be determined. For example, a pre-trained first dentition segmentation model is used to segment the dentition in the two-dimensional photo (i.e., the tooth image data to be identified) to obtain each single tooth, and the maxillary first molar and the mandibular first molar can be determined. In some examples, the first dentition segmentation model can be obtained by training based on a CNN structure network such as FCIS (Fully Convolutional Instance-aware Semantic Segmentation), Mask R-CNN (Mask Region-based Convolutional Neural Network), U-Net (Convolutional Networks for Biomedical Image Segmentation) series or a Transformer structure network such as ISTR (End-to-End Instance Segmentation with Transformers), Mask2Former, etc. It can be understood that the specific first dentition segmentation model in actual application can adopt an existing model or be trained separately, which will not be repeated here. Those skilled in the art will understand that in addition to using a trained dentition segmentation model for tooth segmentation, other methods of segmentation can also be used. For example, in some embodiments, the trained neural network model for Angle's classification is used to identify the tooth image data to be identified, which can be achieved in the following way: identifying the tooth image data to be identified, obtaining the maxillary first molar and the mandibular first molar in the tooth image data to be identified; using the trained neural network model for Angle's classification to identify the maxillary first molar and the mandibular first molar, and determining the Angle's classification result. In this case, the upper and lower first molars are first identified from the dental image data, and then the patient's Angle's classification result is determined based on the first molars. Since the primary reference for Angle's classification is the upper and lower first molars, the model recognition information is concentrated on the upper and lower first molars to reduce information interference, which can make the neural network model's recognition results after analyzing the image more accurate. For example, in some embodiments, identifying the dental image data to be identified and obtaining the upper and lower first molars in the dental image data to be identified can be achieved as follows: identifying the dental image data to be identified using instance segmentation to obtain the upper and lower first molars. In this case, identifying the upper and lower first molars using instance segmentation makes the identification more accurate.Other possible implementations will not be listed here one by one. The segmentation results can be indicated by labels, and in some examples, they can be indicated by visual labels, such as marking different teeth with different colors. The specific colors can be set as needed and are not limited here.

[0071] In some examples, the maxillary first molar and mandibular first molar obtained in this step can be marked by adding annotations, or the tooth image data to be identified can be further processed: in some examples, the pixels of the tooth image data to be identified can be changed, and the pixels of the data other than the maxillary first molar and mandibular first molar can be set to 0. As shown in Figures 6 and 7, Figure 6 shows that different teeth are marked with different colors (Figure 6 is a grayscale image, in this case, different colors refer to different grayscale values) after the teeth are separated, and Figure 7 shows that the pixels of the data other than the maxillary first molar and mandibular first molar are all set to 0. By changing the pixels, the feature-related area is focused. Since setting the pixels to 0 is simple and fast, and can produce a large difference from the area where the pixels are not changed, the data of the maxillary first molar and mandibular first molar are highlighted without the need for additional data production. In some examples, it is also possible to only crop out the area block including the maxillary first molar and mandibular first molar, and change the pixels of the data other than the maxillary first molar and mandibular first molar in the cropped area block, as shown in Figure 8. During recognition, excessive non-feature-related data not only affects the recognition speed but may also reduce the accuracy of the recognition results. Therefore, in some examples, to avoid excessive background interference, the area blocks including the maxillary first molar and the mandibular first molar are cropped out, and the cropped area blocks are subsequently processed by changing the pixels. This reduces the amount of data processing while ensuring the integrity of the feature information, effectively improving the accuracy of the recognition results.

[0072] Step 407 : Using the trained neural network model for Angle's classification, the maxillary first molar and the mandibular first molar are identified to determine the Angle's classification result.

[0073] In some examples, this step can use the trained neural network model for Angle's classification obtained in step 402 to identify the maxillary first molar and the mandibular first molar obtained in step 406, thereby obtaining the corresponding Angle's classification results.

[0074] As can be seen, this embodiment first identifies the ipsilateral first molars of the upper and lower jaws from the tooth image data to be identified, and then determines the patient's Angle's classification result based on the first molars. Since the primary reference for Angle's classification is the ipsilateral first molars of the upper and lower jaws, the model recognition information is concentrated on the ipsilateral first molars of the upper and lower jaws, reducing information interference. This allows the trained neural network model for Angle's classification to analyze the image and produce more accurate recognition results. In addition, instance segmentation is further limited to using a convolutional network trained to identify maxillary and mandibular first molars. Due to the rich variety of convolutional networks, the specific implementation is more feasible and easier to promote. Moreover, in the neural network recognition stage of this embodiment, due to the centralized data features, the training speed is fast, and the requirement for the number of sample sets is relatively low, so the neural network model structure that can be adopted can be simpler.

[0075] Another embodiment of the present application also relates to a method for automatic Angle's classification recognition. This embodiment is substantially the same as the above embodiment, with the main difference being that the above embodiment uses a two-dimensional image as an example, while this embodiment uses a three-dimensional dental model as an example of the tooth image data to be recognized. A three-dimensional dental model is also a common data that needs to be collected in the early stages of orthodontics, and the information contained in a three-dimensional dental model is richer than that of a two-dimensional photograph. Therefore, in this embodiment, the use of a three-dimensional dental model as the tooth data to be recognized can, on the one hand, make the results of the automatic Angle's classification recognition more accurate, and on the other hand, it can expand the application scenarios of this application and adapt to different needs.

[0076] In some examples, the three-dimensional dental model can be obtained by oral scanning, such as using a scanning device to perform intraoral or extraoral scanning of the patient's upper and lower teeth, thereby obtaining a digital three-dimensional tooth model, which can have high-precision crown information. It should also be noted that the dental model obtained by oral scanning in this example has been occlusally matched to maintain the upper and lower jaw models of the patient's true occlusal relationship. In some examples, the three-dimensional dental model can also be obtained by scanning a physical model, such as taking a mold of the patient's dentition in advance, making a plaster model, and then scanning the plaster model to obtain the above-mentioned three-dimensional dental model. It can be seen that there is more than one way to obtain a three-dimensional dental model, and it can be obtained according to actual needs, which is not limited here.

[0077] The process of the automatic identification method for Angle's classification in this embodiment is similar to the process of the automatic identification method in the aforementioned embodiment. In this embodiment, the steps of identifying the tooth image data to be identified and obtaining the maxillary first molars and mandibular first molars therein include: extracting the geometric features of the three-dimensional dental model, segmenting the three-dimensional dental model based on the geometric features, and determining the maxillary first molars and mandibular first molars. In practical applications, tooth classification can also be performed by combining geometric features with color features. Specific algorithms can adopt machine learning methods, spectral clustering methods, etc., which are not listed here one by one.

[0078] In some examples, independent image data can be generated for the obtained maxillary first molar and mandibular first molar to obtain first image data; identifying the maxillary first molar and mandibular first molar using the trained neural network model for Angle's classification may include: identifying the first image data using the trained neural network model for Angle's classification. In some examples, independent image data can also be generated so that the trained neural network model for Angle's classification can focus more on feature data during recognition, thereby improving recognition speed and accuracy. For example, after identifying the maxillary first molars and the mandibular first molars, the maxillary first molars and the mandibular first molars (shown in Figure 10) are segmented from the three-dimensional dental model (shown in Figure 9), and these two teeth are made into first image data, that is, new model data. It can be understood that even if these two first molars are segmented from the overall dental model, they still retain their original spatial position relationship. Therefore, in the subsequent recognition process, the trained neural network model for Angle's classification can automatically identify the corresponding Angle's classification results only by identifying the maxillary first molars and mandibular first molars as shown in Figure 10. In this way, during recognition, the features of the tooth image data to be identified are concentrated on these two teeth, so the recognition speed is not only faster, but the results will also be more accurate.

[0079] In some examples, the neural network model trained for Andrzej Kroeberman classification can use classic models such as PointNet and PointNet++ to process the image data to be identified to generate relevant point cloud data, and then classify the generated point cloud data.

[0080] That is, if the tooth image data to be identified is a two-dimensional photograph, then identifying the tooth image data to be identified and obtaining the maxillary first molars and mandibular first molars in the tooth image data to be identified can be achieved as follows: using a pre-trained first dentition segmentation model to segment the dentition in the two-dimensional photograph to obtain each individual tooth, thereby determining the maxillary first molars and mandibular first molars. If the tooth image data to be identified is a three-dimensional jaw model, then identifying the tooth image data to be identified and obtaining the maxillary first molars and mandibular first molars in the tooth image data to be identified can be achieved as follows: extracting geometric features of the three-dimensional jaw model, segmenting the three-dimensional jaw model based on the geometric features, and determining the maxillary first molars and mandibular first molars. Thus, by respectively limiting the tooth classification methods corresponding to different types of tooth data, the tooth data in the two-dimensional photograph and the three-dimensional jaw model can be more appropriately processed, and the maxillary first molars and mandibular first molars can be more accurately located.

[0081] It can be understood from the above embodiments that this application not only covers the application scenarios of two-dimensional image recognition, but also covers the application scenarios of three-dimensional dental model recognition. It has wide applications and high accuracy, and contributes to improving the data accuracy in the orthodontic digital design stage.

[0082] It should be noted that the above examples in the embodiments of the present application are illustrative for ease of understanding and do not constitute a limitation on the technical solutions of the present application.

[0083] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0084] Another embodiment of the present application relates to an electronic device, as shown in Figure 11, comprising: at least one processor 801; and a memory 802 communicatively connected to the at least one processor 801; wherein the memory 802 stores instructions that can be executed by the at least one processor 801, and the instructions are executed by the at least one processor 801 to enable the at least one processor 801 to execute the automatic recognition method of the Angle's classification in the above-mentioned embodiments.

[0085] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0086] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0087] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0088] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0089] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for automatically identifying Andr's classification, the method comprising the following steps automatically performed by an electronic device: Acquire tooth image data to be identified, the tooth image data to be identified includes: 2D photographs and / or 3D dental models; Using the trained neural network model for Angle's classification to identify the tooth image data to be identified, and determining the patient's Angle's classification result; Wherein, before using the trained neural network model for Angle's classification to identify the tooth image data to be identified, the method includes: Collect first-category dental image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels to form a first sample set; construct a neural network model; use the first sample set to train the constructed neural network model to obtain the trained neural network model for Angle's classification; wherein the first-category dental image data includes at least the patient's maxillary first molars and mandibular first molars.

2. The automatic identification method of Angle's classification according to claim 1, wherein: The method of using the trained neural network model for Angle's classification to identify the tooth image data to be identified includes: Identify the to-be-identified tooth image data, and obtain the maxillary first molar and the mandibular first molar in the to-be-identified tooth image data; The trained neural network model for Angle's classification is used to identify the maxillary first molar and the mandibular first molar to determine the Angle's classification result.

3. The automatic identification method of Angle's classification according to claim 2, wherein: The step of identifying the to-be-identified tooth image data and obtaining the maxillary first molar and the mandibular first molar in the to-be-identified tooth image data includes: The tooth image data to be identified is identified by instance segmentation to obtain the maxillary first molar and the mandibular first molar.

4. The automatic identification method of Angle's classification according to claim 3, wherein: The identifying the tooth image data to be identified by instance segmentation includes: Performing instance segmentation on the tooth image data to be identified using a convolutional network trained to identify the maxillary first molar and the mandibular first molar; Wherein, before identifying the tooth image data to be identified, the method includes: Acquire second-category tooth image data as second-category sample images, wherein each tooth in the second-category sample images corresponds to a tooth classification label, forming a second sample set; construct a convolutional network, and use the second sample set to train the constructed convolutional network to obtain the trained convolutional network for identifying the maxillary first molars and the mandibular first molars.

5. The automatic identification method of Angle's classification according to claim 4, wherein: The acquiring of the second type of tooth image data as the second type of sample images includes: acquiring each piece of the first type of tooth image data from the first sample set as each piece of the second type of tooth image data in the second sample set.

6. The automatic identification method of Angle's classification according to claim 2, wherein: If the tooth image data to be identified is a two-dimensional photograph, identifying the tooth image data to be identified and obtaining the maxillary first molar and the mandibular first molar in the tooth image data to be identified includes: using a pre-trained first dentition segmentation model to segment the dentition in the two-dimensional photograph to obtain each single tooth, and determining the maxillary first molar and the mandibular first molar; If the tooth image data to be identified is a three-dimensional dental model, then identifying the tooth image data to be identified and obtaining the maxillary first molar and mandibular first molar in the tooth image data to be identified includes: extracting the geometric features of the three-dimensional dental model, segmenting the three-dimensional dental model according to the geometric features, and determining the maxillary first molar and mandibular first molar.

7. The automatic identification method of Angle's classification according to any one of claims 2 to 5, wherein: If the tooth image data to be identified is a two-dimensional photo, identifying the tooth image data to be identified and obtaining the maxillary first molar and the mandibular first molar in the tooth image data to be identified includes: The pixels of the tooth image data to be identified are changed, and the pixels of the data other than the maxillary first molar and the mandibular first molar are set to 0.

8. The automatic identification method of Angle's classification according to claim 7, wherein: Before changing the pixels of the to-be-identified tooth image data, the method further includes: cropping a region block including the maxillary first molar and the mandibular first molar; The changing of the pixels of the to-be-identified tooth image data includes: changing the pixels of data other than the maxillary first molar and the mandibular first molar in the cropped area block.

9. The automatic identification method of Angle's classification according to any one of claims 2 to 5, wherein: Before using the trained neural network model for Angle's classification to identify the maxillary first molar and the mandibular first molar, the method includes: constructing independent image data for the obtained maxillary first molar and the mandibular first molar to obtain first image data; The identifying the maxillary first molar and the mandibular first molar by using the trained neural network model for Angle's classification includes: identifying the first image data by using the trained neural network model for Angle's classification.

10. The automatic identification method of Angle's classification according to any one of claims 2 to 5, wherein: The trained neural network model for Andrzej Krohn's classification is one of the following: RA-CNN, Resnet, DesNet, Inception, ViT, GAT or SR-GNN network.

11. The automatic identification method of Angle's classification according to claim 1, wherein: If the tooth image data to be identified is a two-dimensional photo, the first type of tooth image data is collected as a first type of sample image, and the first type of sample image and the corresponding Angle's classification label are saved to form a first sample set, including: Processing the first type of sample image; wherein the processing of the first type of sample image includes: segmenting the maxillary first molar and the mandibular first molar in the first type of sample image, adding a background to the maxillary first molar and the mandibular first molar, and forming an updated first type of sample image.

12. The automatic identification method of Angle's classification according to claim 11, wherein: Adding backgrounds to the maxillary first molar and the mandibular first molar includes adding multiple different backgrounds to the maxillary first molar and the mandibular first molar to form multiple different updated first-category sample images.

13. The automatic identification method of Angle's classification according to any one of claims 1, 11 or 12, wherein: The trained neural network model for Andrzej Krohn's classification is one of the following: RA-CNN, CMAL-Net, ViT, SR-GNN, ResNet, DesNet, Yolo or EfficientNet network.

14. The automatic identification method of Angle's classification according to claim 1, wherein: The two-dimensional photo included in the tooth image data to be identified is a side intraoral photo.

15. An electronic device comprising: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic identification method of the Angle's classification according to any one of claims 1 to 14.

16. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the method for automatically identifying the Angle's classification system according to any one of claims 1 to 14 is implemented.

Images