Ansheng classification automatic identification method, electronic equipment and medium

Abstract

The embodiment of the invention relates to the technical field of digital design of medical instruments, and discloses an Angler classification automatic identification method, electronic equipment and a medium, the identification method comprises the following steps automatically executed by the electronic equipment: obtaining to-be-identified tooth image data, the tooth image data comprising a two-dimensional photo and / or a three-dimensional dental model; identifying the obtained tooth image data by using the trained neural network model, and determining an Andru classification result of the patient; wherein the tooth image data at least comprises first molars of the upper jaw and first molars of the lower jaw of the patient. The invention further discloses an electronic device and a medium, Ansheng classification can be automatically carried out on the tooth jaw of the patient, Ansheng classification is more accurate and efficient, and more accurate and credible classification results are further provided for doctors or professionals.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of digital design of medical devices, and in particular to an automatic recognition method for Angle's classification, an electronic device, and a medium. Background Art

[0002] Malocclusion refers to the developmental deformities of the teeth, jaws, and face that may occur during a child's growth and development due to various factors, including genetics and the environment. (An ideal normal jaw refers to the presence of all upper and lower teeth, neatly arranged, with good cusp-to-fossa contact, normal jaw development, and good dental arch morphology), which can affect the beauty and health of the face. When orthodontists determine the corrective device for patients with malocclusion, they generally first determine the type of malocclusion and then use the corresponding corrective device. Therefore, the accuracy of the classification can significantly improve the accuracy of subsequent corrective device selection.

[0003] The Angle classification is a widely used malocclusion classification system proposed by Dr. Angle in 1899. It primarily uses the maxillary first permanent molar as a reference point and is based on the anterior-posterior relationship between the upper and lower dental arches. Currently, this system relies primarily on the physician's judgment, which is limited by experience and can lead to a certain degree of error. This incorrect judgment can lead to incorrect selection of orthodontic devices, compromising patient treatment outcomes.

[0004] It can be seen that we need a more accurate and efficient method for identifying malocclusion, so as to more efficiently and accurately identify the type of malocclusion in patients and provide doctors with a more reliable reference basis. Summary of the Invention

[0005] The purpose of the embodiments of the present application is to provide an automatic Angle classification recognition method, electronic equipment and medium, which can automatically perform Angle classification on the patient's jaw, making Angle classification more accurate and efficient, and further providing doctors or professionals with more accurate and reliable classification results.

[0006] To solve the above technical problems, an embodiment of the present application provides an automatic Angle's classification recognition method, which includes an electronic device automatically performing the following steps: obtaining tooth image data to be recognized, the tooth image data including: two-dimensional photos and / or three-dimensional dental models; using a trained neural network model to recognize the obtained tooth image data to determine the patient's Angle's classification result; wherein the tooth image data includes at least the patient's maxillary first molars and mandibular first molars; wherein, before using the trained neural network model to recognize the obtained tooth image data, the method includes: collecting a first type of tooth image data as a first type of sample image, saving the first type of sample image and the corresponding Angle's classification label to form a first sample set; constructing a neural network model; using the first sample set to train the neural network model to obtain a trained neural network model for Angle's classification.

[0007] An embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned Angle's classification automatic identification method.

[0008] An embodiment of the present application further provides a computer-readable storage medium storing a computer program, which implements the above-mentioned Angle's classification automatic recognition method when executed by a processor.

[0009] The automatic Angle's classification recognition method in the embodiments of the present application specifically utilizes an electronic device to automatically determine the Angle's classification corresponding to a patient's teeth based on the tooth image data to be recognized. The electronic device automatically recognizes the tooth image data using a trained neural network model, which is simple, convenient, and fast, with minimal operational difficulty. Furthermore, the recognition results obtained by analyzing the features of the maxillary first molars and mandibular first molars in the tooth image data using a machine learning algorithm are incorporated into the automatic Angle's classification, making the recognition results more accurate, providing doctors or professionals with a more accurate and reliable basis for judgment, and improving the accuracy and reliability of the orthodontic devices selected based on the classification results.

[0010] Furthermore, the method of using the trained neural network model to recognize the acquired dental image data includes: recognizing the acquired dental image data to obtain the maxillary first molars and the mandibular first molars therein; and using the trained neural network model to recognize the maxillary first molars and the mandibular first molars to determine the Angle's classification result. Furthermore, the method is limited to first identifying the upper and lower first molars from the dental image data, and then determining the patient's Angle's classification result based on the first molars. Since the primary reference for Angle's classification is the upper and lower first molars, the model recognition information is focused on the upper and lower first molars to reduce information interference, thereby making the recognition result after the neural network model analyzes the image more accurate.

[0011] Additionally, identifying the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes identifying the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein by instance segmentation. Furthermore, the instance segmentation method is limited to identifying the maxillary first molar and the mandibular first molar, thereby increasing recognition accuracy.

[0012] In addition, the instance segmentation method includes: performing instance segmentation using a trained convolutional network; wherein, before identifying the acquired dental image data, the method includes: acquiring a second type of dental image data as a second type of sample image, wherein each tooth in the second type of sample image has a corresponding dental classification label, forming a second sample set; constructing a convolutional network for instance segmentation, and training the neural network model using the second sample set to obtain a trained convolutional network for identifying the maxillary first molars and mandibular first molars. Furthermore, instance segmentation is performed using a trained convolutional network. Due to the rich variety of convolutional networks, the method is more feasible and easier to generalize in practice.

[0013] Furthermore, obtaining the second category of dental image data as the second category of sample images includes: obtaining each first category of dental images from the first sample set as each second category of dental images in the second sample set. Furthermore, the sample images used during training are limited to two types of labels, namely, Angle's classification labels and dental classification labels. In this embodiment, multiple types of labels can be generated for the sample images, and the same batch of images can be used to generate multiple types of labels so that they can be used in different steps of neural network model training. In this approach, the neural network model used in the second stage has a simple structure, concentrated data features, fast training speed, and a low requirement for the number of sample sets.

[0014] In addition, if the dental image data is a two-dimensional photograph, then the identification of the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes: using a pre-trained first dental arch segmentation model to segment the dental arch in the two-dimensional photograph to obtain each individual tooth, and determining the maxillary first molar and the mandibular first molar; if the dental image data is a three-dimensional dental jaw model, then the identification of the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes: extracting geometric features of the three-dimensional dental jaw model, segmenting the three-dimensional dental jaw model according to the geometric features, and determining the maxillary first molar and the mandibular first molar. In this embodiment, the tooth classification methods corresponding to different types of dental data are respectively limited so that the dental data in the two-dimensional photograph and the three-dimensional dental jaw model can be more appropriately processed and the maxillary first molar and the mandibular first molar can be more accurately located.

[0015] Alternatively, if the dental image data is a two-dimensional photograph, identifying the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes the sub-step of altering pixels in the dental image data to set pixels other than those of the maxillary first molar and the mandibular first molar to 0. Furthermore, altering pixels is limited to focusing on feature-related areas. Because altering pixels is simple and rapid and can produce significant differences from areas where pixels remain unchanged, the data of the maxillary and mandibular first molars can be highlighted without the need for additional data generation.

[0016] In addition, before changing the pixels of the dental image data, the method further includes: cropping out a region block including the maxillary first molar and the mandibular first molar; and changing the pixels of the dental image data includes: changing the pixels of the data other than the maxillary first molar and the mandibular first molar in the cropped region block. Because excessive non-feature-related data during recognition not only affects recognition speed but may also reduce the accuracy of the recognition results, in this embodiment, to avoid excessive background interference, the region block including the maxillary first molar and the mandibular first molar is cropped out, and the cropped region block is subsequently subjected to pixel modification processing. This ensures the integrity of the feature information while reducing the amount of data processing, effectively improving the accuracy of the recognition results.

[0017] Furthermore, before using the trained neural network model to identify the maxillary first molar and the mandibular first molar, the method includes: constructing independent image data for the obtained maxillary first molar and the mandibular first molar to obtain first image data; and using the trained neural network model to identify the maxillary first molar and the mandibular first molar includes: using the trained neural network model to identify the first image data. Furthermore, by creating independent image data, the neural network model can focus more on feature data during recognition, thereby improving recognition speed and accuracy.

[0018] In addition, the neural network model is one of the following: RA-CNN, Resnet, DesNet, Inception, ViT, GAT, or SR-GNN network. Further limiting the types of neural network models available in the method of identifying the upper and lower first molars and then identifying the Angle classification, this application provides a wide range of optional networks during implementation, facilitating the selection of different neural network models for different application scenarios, thereby expanding the scope of application of this application.

[0019] In addition, if the dental image data is a two-dimensional photograph, the method of collecting the first type of dental image data as a first type of sample image, saving the first type of sample image and the corresponding Angle's classification label to form a first sample set includes: processing the first type of sample image; including: segmenting the maxillary first molar and the mandibular first molar in the first type of sample image, adding background to the maxillary first molar and the mandibular first molar, and forming an updated first type of sample image. This embodiment is further limited to segmenting the sample to a portion with higher feature relevance as an independent image, and adding background, which effectively improves the feature focus of the data and facilitates improving the training speed of the neural network model.

[0020] Furthermore, adding backgrounds to the maxillary first molar and the mandibular first molar includes adding multiple different backgrounds to the maxillary first molar and the mandibular first molar to form multiple different first-category sample images. When sample data is limited, adding different backgrounds can increase the sample size and thus improve the accuracy of the training results.

[0021] In addition, the neural network model is one of the following: RA-CNN, CMAL-Net, ViT, SR-GNN, ResNet, DesNet, Yolo, or EfficientNet. Further limiting the types of neural network models available in the method of identifying the upper and lower first molars and then identifying the Angle classification, this application provides a wide range of optional networks during implementation, facilitating the selection of different neural network models for different application scenarios, thereby expanding the scope of application of this application and facilitating its promotion.

[0022] In addition, the two-dimensional photograph is a side intraoral photograph. Since side intraoral photographs are routinely collected in the early stages of orthodontic treatment, this embodiment limits the use of side intraoral photographs for Angle's classification recognition, which helps reduce the amount of additional data collection, simplifies the operation on the doctor's side, and facilitates the promotion of the Angle's classification recognition method in this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0024] Figure 1 This is a flow chart of an automatic identification method of Angle's classification provided according to one embodiment of the present application;

[0025] Figure 2 is a schematic diagram of a side intraoral photograph used in an automatic identification method of Angle's classification according to one embodiment of the present application;

[0026] Figure 3 1 is a schematic structural diagram of a neural network model used in an automatic identification method of Andrzej Kroeberman classification according to one embodiment of the present application;

[0027] Figure 4 is a flow chart of an automatic identification method of Angle's classification provided according to another embodiment of the present application;

[0028] Figure 5 This is a schematic structural diagram of a neural network model constructed in an automatic recognition method for Andrzej Kroeberman classification according to another embodiment of the present application;

[0029] Figure 6 This is a schematic diagram of a tooth image after tooth separation in an automatic recognition method of Angle's classification according to another embodiment of the present application;

[0030] Figure 7 This is a schematic diagram of a tooth image after pixel change in an automatic recognition method of Angle's classification according to another embodiment of the present application;

[0031] Figure 8 This is a schematic diagram of a tooth image after cropping a region block and changing pixels in an automatic identification method of Angle's classification according to another embodiment of the present application;

[0032] Figure 9 is a schematic diagram of a three-dimensional dental model used in an automatic identification method of Angle's classification according to another embodiment of the present application;

[0033] Figure 10 It is from Figure 9 Schematic diagram of the maxillary first molar and mandibular first molar segmented on the three-dimensional dental model shown;

[0034] Figure 11 is a schematic diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0035] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, each embodiment of the present application will be described in detail below with reference to the accompanying drawings. However, it will be understood by those skilled in the art that in each embodiment of the present application, many technical details are proposed to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can be implemented. The division of the following embodiments is for convenience of description and should not constitute any limitation on the specific implementation of the present application. The various embodiments can be combined and referenced with each other under the premise of no contradiction.

[0036] The "anterior region" and "posterior region" mentioned in various embodiments of this application are defined according to the classification of teeth in the second edition of "Introduction to Stomatology" published by Peking University Medical Press, pages 36-38. The posterior region includes premolars and molars, which are represented as teeth 4-8 using the FDI notation. The anterior region is represented as teeth 1-3 using the FDI notation, and the teeth in the anterior region include central incisors, lateral incisors, and canines.

[0037] One embodiment of the present application relates to an automatic identification method for Andr's classification. The specific process of the automatic identification method for Andr's classification in this embodiment can be as follows: Figure 1 Shown, including:

[0038] Step 101: collect first-category dental image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels, and form a first sample set, wherein the dental image data at least includes the patient's maxillary first molars and mandibular first molars.

[0039] Step 102: construct a neural network model, and use the first sample set to train the neural network model to obtain a trained neural network model for Andrzej Kroeberman classification.

[0040] Step 103: Acquire the tooth image data to be identified. Specifically, the tooth image data includes: a two-dimensional photo and / or a three-dimensional dental model.

[0041] Step 104: Use the trained neural network model to identify the acquired dental image data to determine the patient's Angle's classification result.

[0042] Through the above steps 101 to 104, the recognition results obtained by analyzing the features of the maxillary first molars and the mandibular first molars in the dental image data using a machine learning algorithm, and introducing the machine learning algorithm into the automatic Angle classification make the recognition results more accurate, provide doctors or professionals with a more accurate and reliable judgment basis, and improve the accuracy and credibility of the orthodontic devices selected based on the classification results.

[0043] The following is a detailed description of the implementation details of the automatic identification method of the Andersen classification of this embodiment. The following content is only provided for ease of understanding and is not necessary for implementing this solution.

[0044] It should be noted that the automatic Angle's classification recognition method in this embodiment can be implemented by hardware or a combination of computer software and hardware. Regarding hardware implementation, the automatic Angle's classification recognition method can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the automatic Angle's classification recognition function, or a combination of these devices.

[0045] In step 101, first-class dental image data is collected as first-class sample images. The first-class dental images can be obtained from the patient's historical medical records or from a dedicated database, which will not be listed here. Specifically, the dental image data includes at least the patient's maxillary first molar and mandibular first molar. In this embodiment, the dental image data is a two-dimensional photo, which can be a side intraoral photo (such as Figure 2 As shown in the figure, since the lateral intraoral photograph is routinely collected in the early stages of orthodontic treatment, it will collect buccal information of all posterior teeth and can effectively perform Angle's classification identification. Therefore, this embodiment limits the use of lateral intraoral photographs for Angle's classification identification. This helps reduce the amount of additional data collection, simplifies the operation on the doctor's side, and facilitates the promotion of the Angle's classification identification method in this application. In actual applications, the two-dimensional photograph can also be other two-dimensional photographs, such as two-dimensional photographs collected specifically for Angle's classification, which will not be listed here one by one.

[0046] In some embodiments, if the collected first-category sample images already carry Andersen classification labels, they can be directly saved and used. If the collected first-category sample images do not carry Andersen classification labels, the sample images can be annotated, the corresponding Andersen classification labels added, and then saved. After collecting a certain number of first-category sample images, a first sample set is formed. The specific annotation process can be determined by professionals, or a partial set of labels can be created first, a neural network model trained, and then the trained model automatically annotated, gradually expanding the number of samples in the first sample set.

[0047] In some embodiments, the first type of sample image can be further processed, specifically including: segmenting the maxillary first molar and the mandibular first molar in the first type of sample image, adding background to the maxillary first molar and the mandibular first molar, and forming an updated first type of sample image. Because Angle's classification is primarily determined by the positional relationship between the upper and lower first molars, this embodiment is further limited to segmenting the sample image to obtain a portion with higher feature correlation as an independent image, and adding a background, such as setting all pixels in the area other than the upper and lower first molars to 0, thereby increasing the difference between the background and the upper and lower first molars, effectively improving the feature focus of the data, and facilitating the training speed of the neural network model and improving the accuracy of the training results.

[0048] In addition, in some embodiments, adding background to the above-mentioned maxillary first molar and mandibular first molar includes: adding multiple different backgrounds to the maxillary first molar and mandibular first molar to form multiple different first-category sample images. When the sample data is small, the sample size can be increased by adding different backgrounds to improve the accuracy of the training results. Since the sample data that can be collected in the field of orthodontics is generally small, the above-mentioned method of changing the background can increase the number of samples as much as possible while ensuring the accuracy of the sample data, and the reference features of Angle's classification are concentrated on the upper and lower first molars on the same side, so the upper and lower first molars on the same side can be separated separately to effectively concentrate the feature areas of the sample image, which is simple to implement and has better results. It should also be noted that in addition to the above-mentioned method of changing the background, the method of increasing the sample size can also adopt traditional sample processing methods, such as rotation, translation, adding Gaussian noise, etc., which are not listed here one by one.

[0049] In step 102, a neural network model is constructed. The constructed neural network model may be a deep learning model, such as Figure 3 As shown in the figure, it includes an input layer, a feature extractor, an attention network, a fully connected module, and an output layer. The network extracts features from the input image using a CNN-based feature extractor. The output features are then fed into an attention candidate network (APN). The APN identifies regions with relevant features, and the extracted features are then fed into a fully connected classifier to predict the image classification. The region identified by the attention candidate network (APN) is then cropped from the original image as input to the next network, and this cycle continues.

[0050] Further Figure 3As shown, the neural network model constructed in this application can be set with two scale layers (original scale layer and cropped scale layer). The structure of each scale layer is roughly the same. The previous scale layer will crop out a more focused attention area. The next scale layer continues to operate based on the attention area output by the previous scale layer to further obtain the Angle classification result. By setting two scale layers, the feature area is cropped by the attention network, and the prediction of the image is concentrated in the cropped feature area, so as to speed up the training speed and improve the accuracy of the recognition and classification results. It is known to those skilled in the art that although this embodiment is described with two scale layers as an example, in actual applications, more scale layers can be set, such as three or more layers, to further crop the cropped image and focus more on the feature-related areas so that the final classification result is more accurate. Each scale layer will give a probability distribution corresponding to the three classification results in the Angle classification. Each probability output position corresponds to an Angle category, and the category corresponding to the position with the largest probability value is the Angle classification result of the module.

[0051] In some embodiments, an encoder (such as an encoder with a Transformer structure) can be added to each scale layer, and the original image is input into the encoder in parallel. In this embodiment, the encoder is used to extract features in parallel with the CNN feature extractor. The output of the encoder is then spliced ​​with the output of the CNN feature extractor and then input into the classifier (i.e., the fully connected module). In the subsequent model training process, since the transformer structure contains a multi-head self-attention mechanism, it helps the CNN feature extractor to better extract key features, thereby allowing the attention candidate network (APN) to more efficiently find the key areas that determine the classification results of the input image.

[0052] More specifically, the above neural network model can use the following loss function L during training: tot , consists of two parts, one is the classification loss L cls , the other part is L rank :

[0053]

[0054]

[0055]

[0056] Among them, Y (s) and Y * are the model prediction category and label category, P t (s)It represents the class probability predicted by the scale s module, t represents the Angle class label of the image, s represents the image scale, and n represents the number of scales. For example, in this embodiment, two scales are used, so n = 2; margin is a value set manually to ensure that P t (s+1) Than p t (s) It is large enough to enable the neural network to be trained more effectively, and its specific value can be set by technicians according to actual needs.

[0057] It should be noted that in scenarios where malocclusion classification is achieved through tooth regions, although tooth images have individual characteristics, their overall morphological features are very similar. For example, the tooth region generally consists of a crown and a gum region, and the tooth region has a curved contour. Therefore, the characteristic portion accounts for a very small proportion relative to the entire image. In addition, the image is affected by different objective factors such as the acquisition scene, the form and color of saliva in the mouth, and there are many interfering features. Therefore, traditional neural networks require a lot of preliminary data processing when automatically identifying teeth, which is also a major factor affecting the accuracy of automatic tooth classification. In this regard, after studying the characteristics of teeth, the inventors of this application realized that it is necessary to first identify the characteristic regions related to the classification features. The visual transformer itself divides the image into multiple regions during calculation, which is more suitable for solving the task of finding characteristic tooth regions. Moreover, since Angle's classification is mainly determined by the relative position of the upper and lower first molars, the model can focus on the region containing these two molars, which will improve the subsequent training speed and the accuracy of the trained model.

[0058] To continue the explanation, after the neural network model is constructed, the first sample set obtained in step 101 is used to train the neural network model to obtain a trained neural network model for Andrzej Krohn's classification.

[0059] It is worth mentioning that the above steps 101 and 102 are model building and training steps for obtaining a trained neural network model. Therefore, the model can be trained in advance and retrieved when recognition is required, or trained after recognition is required. In other words, the above steps 101-102 can also be set after step 103. Those skilled in the art will understand that changing the order of the steps will not change the inventive concept of this application.

[0060] In step 103, the tooth image data to be identified is obtained. Specifically, the tooth image data is the tooth image data required for Angle's classification. It can be a two-dimensional image or a three-dimensional dental model collected by the doctor, or it can be collected separately for Angle's classification, which is not limited here.

[0061] In step 104, the trained neural network model is used to identify the acquired dental image data to determine the patient's Angle's classification result. Specifically, the dental image data acquired in step 103 is input into the trained neural network model described above in step 102, resulting in a final output Angle's classification result, which serves as the patient's Angle's classification result. Angle's classification is generally divided into three categories: Angle's Class I, Angle's Class II, and Angle's Class III.

[0062] It should be noted that, taking the above two scaling layers as an example, the Andrzej Klein classification result output by the cropped scaling layer can be used as the final classification result, or the output results of the two fully connected modules in the two-layer model can be merged, which can be normalized first and then merged to obtain a new set of classification probability data, which is then operated by the softmax module to obtain the final Andrzej Klein classification result. In other words, the one with the largest probability is determined from the classification probability data as the classification result.

[0063] It is worth mentioning that in addition to the neural network model constructed above, existing neural network models such as CMAL-Net, ViT, SR-GNN, ResNet, DesNet, Yolo, or EfficientNet networks can also be used. Further limiting the types of neural network models available in the method of identifying the upper and lower first molars on the same side and then identifying the Angle classification, this application has a large number of optional networks during implementation, making it easier to select different neural network models for different application scenarios, thereby expanding the scope of application of this application and facilitating its promotion.

[0064] It can be seen that the Angle classification automatic recognition method in this embodiment specifically determines the Angle classification corresponding to the patient's teeth based on the tooth image data to be identified by the electronic device. The electronic device automatically recognizes the tooth image data through the trained neural network model, which is simple, convenient, fast, and has low operation difficulty. In addition, the recognition results obtained by analyzing the features of the maxillary first molars and mandibular first molars in the tooth image data through the machine learning algorithm are introduced into the automatic Angle classification, making the recognition results more accurate, providing doctors or professionals with a more accurate and reliable judgment basis, and improving the accuracy and credibility of the orthodontic devices selected by the classification results. In addition, the sample image is segmented into a part with higher feature correlation as an independent image, and the background is added / changed, which effectively improves the feature focus of the data and is conducive to improving the training speed of the neural network model.

[0065] Another embodiment of the present application also relates to an automatic recognition method for Angle's classification. This embodiment is roughly the same as the above embodiment, with the main improvement being: adding preliminary processing of the tooth image data to be classified, determining the parts that are more relevant to the Angle's classification features, and then using the trained neural network for recognition, reducing information interference, and making the recognition results more accurate after the neural network model analyzes the image.

[0066] The automatic identification method of Angle's classification in this embodiment is as follows Figure 4 As shown, the tooth image data in this embodiment is still described using a two-dimensional photo as an example, as follows:

[0067] Step 401 : collect first-category tooth image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels, and form a first sample set.

[0068] Step 402: construct a neural network model, and use the first sample set to train the neural network model to obtain a trained neural network model for Andrzej Kroeberman classification.

[0069] Specifically, the neural network model constructed in this step may include a convolutional layer-pooling layer and a fully connected layer (such as Figure 5 Specifically, a convolutional layer can include three convolutional modules, each with a different kernel size. Shallow convolutional modules extract relatively basic features, such as lines and textures in the image. As the convolutional layer deepens, it extracts more advanced and complex features. Furthermore, in shallow convolutions, each position on the feature map corresponds to a smaller local location in the original image. As more convolutional layers are added, the positions on the feature map correspond to larger areas in the original image. In other words, as more convolutional layers are added, feature extraction references more information in the original image.

[0070] It can be understood that although the above neural network model is based on Figure 5 The model structure is used as an example to illustrate. In actual applications, neural network models with other structures can also be used, such as networks based on CNN structure such as Resnet, DesNet, Inception, etc., networks based on transformer structure such as ViT, etc., and models based on graph neural networks such as GAT and SR-GNN, etc., which will not be listed here one by one. Further limiting the categories of available neural network models in the method of identifying the Angle classification after identifying the upper and lower first molars, this application has more optional networks when implementing, which is convenient for selecting different neural network models for different application scenarios, so as to expand the scope of application of this application.

[0071] Step 403 : Acquire second-category tooth image data as second-category sample images, wherein each tooth in the second-category sample images corresponds to a tooth classification label, thereby forming a second sample set.

[0072] In some embodiments, the second type of dental image data can also be obtained from the patient's historical medical records or from a dedicated database. The acquisition method is roughly the same as that of the first type of dental image data and will not be repeated here.

[0073] Step 404: construct a convolutional network for instance segmentation, and use the second sample set to train the neural network model to obtain a trained convolutional network for identifying the maxillary first molar and the mandibular first molar.

[0074] Specifically, the above steps 403 to 404 are used to construct and train a network model for identifying a single tooth. In some embodiments, when making the second sample set in step 403, the first type of sample images in the first sample set obtained in step 101 can be directly used. In some embodiments, the sample images obtained themselves carry tooth classification labels, that is, pixel-level tooth classification labels corresponding to single teeth. In this case, the tooth classification labels can be directly saved and used to train the convolutional network. In some embodiments, the sample images obtained do not directly carry tooth classification labels. In this case, the first sample images can be processed for tooth classification, and tooth classification labels corresponding to single teeth can be added to the teeth in the first sample images. In other words, in actual applications, steps 401 and 403 can be combined to make only one sample set, which simultaneously covers the Angle classification labels in the first sample set and the tooth classification labels in the second sample set. It can be seen that the sample images in this embodiment can be used to make multiple types of labels, and the same batch of images can be used to make multiple types of labels so that they can be used in the neural network model training at different steps, reducing data usage.

[0075] It is worth mentioning that although the above description uses convolutional networks for instance segmentation as an example, in actual applications, instance segmentation can be performed through other methods, such as other neural network models, which are not listed here one by one.

[0076] Step 405: Acquire the tooth image data to be identified.

[0077] Specifically, the specific process of this step is similar to step 103 of the aforementioned embodiment, and will not be described again here to avoid repetition.

[0078] Step 406 : Identify the acquired tooth image data to obtain the maxillary first molar and the mandibular first molar.

[0079] Specifically, the convolutional network trained in the above step 404 can be used for recognition. In some embodiments, in this step, the existing tooth segmentation method can also be used to segment the teeth in the image into single teeth, so that the maxillary first molar and the mandibular first molar can be determined. For example, a pre-trained first dentition segmentation model is used to segment the dentition in the two-dimensional photo to obtain each single tooth, and the maxillary first molar and the mandibular first molar can be determined. In some embodiments, the first dentition segmentation model can be trained based on a CNN structure network such as FCIS, MaskRCNN, UNet series or a transformer structure network such as ISTR, Mask2Former, etc. It can be understood that the specific first dentition segmentation model in actual application can adopt an existing model or be trained separately, which will not be repeated here. Those skilled in the art will understand that in addition to using a trained dentition segmentation model for tooth segmentation, other methods can also be used for segmentation, which will not be listed here one by one. The segmentation results can be indicated by labels, specifically by visual labels, such as marking different teeth with different colors. The specific colors can be set as needed and are not limited here.

[0080] More specifically, the maxillary first molar and mandibular first molar obtained in this step can be marked by adding annotations, and the tooth image data can also be further processed: in some embodiments, the pixels of the tooth image data can be changed, and the pixels of the data other than the maxillary first molar and mandibular first molar can be set to 0. Figure 6 and Figure 7 As shown, Figure 6 Shows that different teeth are marked with different colors after separation. Figure 7 It shows that all pixels of the data other than the maxillary first molar and the mandibular first molar are set to 0. By changing the pixels, the feature-related area is focused. Since the method of setting the pixels to 0 is simple and fast, and can produce a large difference from the area where the pixels are not changed, the data of the upper and lower first molars are highlighted without the need for additional data production. Furthermore, only the area block including the maxillary first molar and the mandibular first molar can be cropped, and the pixels of the data other than the maxillary first molar and the mandibular first molar in the cropped area block can be changed, as shown in FIG. Figure 8 As shown. During recognition, excessive non-feature-related data not only affects the recognition speed but may also reduce the accuracy of the recognition results. Therefore, in this embodiment, to avoid excessive background interference, the area block including the maxillary first molar and the mandibular first molar is cropped, and the cropped area block is subsequently processed by changing the pixels. This ensures the integrity of the feature information while reducing the amount of data processing, effectively improving the accuracy of the recognition results.

[0081] Step 407: Use the trained neural network model to identify the maxillary first molar and the mandibular first molar to determine the Angle classification result.

[0082] Specifically, this step can use the neural network model trained in step 402 to identify the maxillary first molars and mandibular first molars obtained in step 406, thereby obtaining corresponding Angle's classification results.

[0083] As can be seen, this embodiment first identifies the ipsilateral first molars of the upper and lower jaws from the dental image data, and then determines the patient's Angle's classification result based on the first molars. Since the primary reference for Angle's classification is the ipsilateral first molars of the upper and lower jaws, the model recognition information is concentrated on the ipsilateral first molars of the upper and lower jaws, reducing information interference and making the neural network model's image analysis more accurate. Furthermore, instance segmentation is further limited to using a trained convolutional network. Due to the rich variety of convolutional networks, the specific implementation is more feasible and easier to generalize. Moreover, in the neural network recognition stage of this embodiment, due to the centralized data features, the training speed is fast, and the requirement for the number of sample sets is relatively low, so the neural network model structure that can be adopted can be simpler.

[0084] Another embodiment of the present application also relates to a method for automatic Angle classification recognition. This embodiment is substantially the same as the above embodiment, with the main difference being that the above embodiment uses a two-dimensional image as an example for explanation, while this embodiment uses a three-dimensional dental model as an example for explanation of the dental image data. A three-dimensional dental model is also a common data that needs to be collected in the early stages of orthodontics, and the information carried in a three-dimensional dental model is richer than that of a two-dimensional photograph. Therefore, in this embodiment, the use of a three-dimensional dental model as the dental data can, on the one hand, make the results of the automatic Angle classification recognition more accurate, and on the other hand, it can expand the application scenarios of this application and adapt to different needs.

[0085] Specifically, in this embodiment, the three-dimensional dental model can be obtained by oral scanning, specifically, the scanning equipment can be used to perform intraoral or extraoral scanning of the patient's upper and lower teeth, thereby obtaining a digital three-dimensional dental model, which can have high-precision crown information. It should also be noted that the dental model obtained by oral scanning in this embodiment has been occlusally matched, and is an upper and lower jaw model that maintains the patient's true occlusal relationship. In addition, in some embodiments, the three-dimensional dental model can be obtained by scanning a physical model, such as taking a mold of the patient's dentition in advance, making a plaster model, and then scanning the plaster model to obtain the above-mentioned three-dimensional dental model. It can be seen that there is more than one way to obtain a three-dimensional dental model, and it can be obtained according to actual needs, which is not limited here.

[0086] The process of the automatic recognition method of Angle's classification in this embodiment is similar to the process of the automatic recognition method in the aforementioned embodiment. Specifically, in the step of identifying the acquired dental image data and obtaining the maxillary first molar and the mandibular first molar therein, it includes: extracting the geometric features of the three-dimensional dental model, segmenting the three-dimensional dental model according to the geometric features, and determining the maxillary first molar and the mandibular first molar. In practical applications, it is also possible to classify teeth by combining geometric features and color features. Specific algorithms can adopt machine learning methods, spectral clustering methods, etc., which are not listed here one by one.

[0087] In some embodiments, independent image data can be constructed for the obtained maxillary first molar and mandibular first molar to obtain first image data; the use of the trained neural network model to identify the maxillary first molar and mandibular first molar includes: using the trained neural network model to identify the first image data. The method of producing independent image data is further adopted so that the neural network model can focus more on feature data during recognition, so as to improve recognition speed and accuracy. Specifically, after identifying the maxillary first molar and mandibular first molar, the three-dimensional dental model ( Figure 9 The maxillary first molar and the mandibular first molar are separated ( Figure 10 (shown), these two teeth are made into the first image data, i.e., the new model data. It can be understood that even if these two first molars are segmented from the overall dental model, their original spatial position relationship is still retained. Therefore, in the subsequent recognition process, the trained neural network model can be used only by recognition. Figure 10 The maxillary first molar and the mandibular first molar shown in the figure are automatically identified and their corresponding Angle classification results are automatically identified. In this way, during the recognition process, the features of the data to be recognized are concentrated on these two teeth, so the recognition speed is not only faster, but the results will also be more accurate.

[0088] More specifically, the neural network model trained in this embodiment can use classic models such as PointNet and PointNet++ to classify the generated point cloud data.

[0089] It can be understood from the above embodiments that this application not only covers the application scenarios of two-dimensional image recognition, but also covers the application scenarios of three-dimensional dental model recognition. It has wide applications and high accuracy, and contributes to improving the data accuracy in the orthodontic digital design stage.

[0090] It should be noted that the above examples in this embodiment are only illustrative for ease of understanding and do not limit the technical solutions of the present invention.

[0091] The steps of the various methods above are divided only for the purpose of clear description. During implementation, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the scope of protection of this patent.

[0092] Another embodiment of the present application relates to an electronic device, such as Figure 11 As shown, it includes: at least one processor 801; and a memory 802 that is communicatively connected to the at least one processor 801; wherein the memory 802 stores instructions that can be executed by the at least one processor 801, and the instructions are executed by the at least one processor 801 to enable the at least one processor 801 to execute the automatic recognition method of Angle's classification in the above-mentioned embodiments.

[0093] The memory and processor are connected using a bus, which can include any number of interconnected buses and bridges. The bus connects various circuits of one or more processors and memories. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits. These are all well known in the art and are therefore not described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single component or multiple components, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over a wireless medium via an antenna. Furthermore, the antenna receives data and transmits it to the processor.

[0094] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0095] Another embodiment of the present application relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0096] That is, those skilled in the art will understand that all or part of the steps in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a program, which is stored in a storage medium and includes a number of instructions for causing a device (which may be a single-chip microcomputer, chip, etc.) or a processor to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, etc., various media that can store program code.

[0097] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present application, and that in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present application.

Claims

1. An automatic recognition method for Angle's classification, characterized by: The identification method includes automatically performing the following steps by an electronic device: Acquire tooth image data to be identified, wherein the tooth image data includes: a two-dimensional photo and / or a three-dimensional dental model; Using the trained neural network model to identify the acquired dental image data, determining the patient's Angle's classification result; wherein the dental image data includes at least the patient's maxillary first molar and mandibular first molar; Before the tooth image data is obtained by using the trained neural network model for recognition, the following steps are included: Collect first-category tooth image data as first-category sample images, save the first-category sample images and corresponding Angle's classification labels to form a first sample set; construct a neural network model; use the first sample set to train the neural network model to obtain a trained neural network model for Angle's classification.

2. The automatic identification method of Angle's classification according to claim 1, characterized in that: The tooth image data obtained by using the trained neural network model for recognition includes: Identify the acquired tooth image data to obtain the maxillary first molar and the mandibular first molar; The trained neural network model is used to identify the maxillary first molar and the mandibular first molar to determine the Angle classification result.

3. The automatic identification method of Angle's classification according to claim 2, characterized in that: The step of identifying the acquired tooth image data to obtain the maxillary first molar and the mandibular first molar comprises: The acquired tooth image data is identified by instance segmentation to obtain the maxillary first molar and the mandibular first molar.

4. The automatic identification method of Angle's classification according to claim 3, characterized in that: The instance segmentation method includes: using a trained convolutional network to perform instance segmentation; Wherein, before identifying the acquired tooth image data, the method includes: Acquire second-category tooth image data as second-category sample images, wherein each tooth in the second-category sample images corresponds to a tooth classification label, forming a second sample set; construct a convolutional network for instance segmentation, and use the second sample set to train the neural network model to obtain a trained convolutional network for identifying the maxillary first molars and the mandibular first molars.

5. The automatic identification method of Angle's classification according to claim 4, characterized in that: The acquiring of the second type of tooth image data as the second type of sample images includes: acquiring each first type of tooth image from the first sample set as each second type of tooth image in the second sample set.

6. The automatic identification method of Angle's classification according to claim 2, characterized in that: If the dental image data is a two-dimensional photograph, then identifying the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes: using a pre-trained first dentition segmentation model to segment the dentition in the two-dimensional photograph to obtain each individual tooth, and determining the maxillary first molar and the mandibular first molar; If the dental image data is a three-dimensional dental model, then the identification of the acquired dental image data to obtain the maxillary first molar and the mandibular first molar therein includes: extracting the geometric features of the three-dimensional dental model, segmenting the three-dimensional dental model according to the geometric features, and determining the maxillary first molar and the mandibular first molar.

7. The automatic identification method of Angle's classification according to any one of claims 2 to 5, characterized in that: If the tooth image data is a two-dimensional photo, the step of identifying the acquired tooth image data to obtain the maxillary first molar and the mandibular first molar includes the following sub-steps: The pixels of the tooth image data are changed, and the pixels of the data other than the maxillary first molar and the mandibular first molar are set to 0.

8. The automatic identification method of Angle's classification according to claim 7, characterized in that: Before changing the pixels of the tooth image data, the method further includes: cutting out a region block including the maxillary first molar and the mandibular first molar; The changing of the pixels of the tooth image data includes: changing the pixels of data other than the maxillary first molar and the mandibular first molar in the cropped area block.

9. The automatic identification method of Angle's classification according to any one of claims 2 to 5, characterized in that: Before using the trained neural network model to identify the maxillary first molar and the mandibular first molar, the method includes: constructing independent image data for the obtained maxillary first molar and the mandibular first molar to obtain first image data; The identifying the maxillary first molar and the mandibular first molar by using the trained neural network model includes: identifying the first image data by using the trained neural network model.

10. The automatic identification method of Angle's classification according to any one of claims 2 to 5, characterized in that: The neural network model is one of the following: Resnet, DesNet, Inception, ViT, GAT or SR-GNN network.

11. The automatic identification method of Angle's classification according to claim 1, characterized in that: If the tooth image data is a two-dimensional photo, collecting the first type of tooth image data as a first type of sample image, saving the first type of sample image and the corresponding Angle's classification label to form a first sample set, includes: Processing the first type of sample images; including: segmenting the maxillary first molars and the mandibular first molars in the first type of sample images, adding backgrounds to the maxillary first molars and the mandibular first molars, and forming updated first type sample images.

12. The automatic identification method of Angle's classification according to claim 11, characterized in that: Adding backgrounds to the maxillary first molar and the mandibular first molar includes adding multiple different backgrounds to the maxillary first molar and the mandibular first molar to form multiple different first-category sample images.

13. The automatic identification method of Angle's classification according to any one of claims 1, 11 or 12, characterized in that: The neural network model is one of the following: CMAL-Net, ViT, SR-GNN, ResNet, DesNet, Yolo or EfficientNet network.

14. The automatic identification method of Angle's classification according to claim 1, characterized in that: The two-dimensional photograph is a side intraoral photograph.

15. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the automatic identification method of Angle's classification according to any one of claims 1 to 14.

16. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for automatic Angle's classification recognition according to any one of claims 1 to 14 is implemented.

Images