Tooth recognition classification method, electronic device, and storage medium

Abstract

The application provides a tooth recognition and classification method, an electronic device and a storage medium. The tooth recognition and classification method first samples segmented tooth data and pre-processes each sampling point to obtain spatial three-dimensional coordinates of sampling points corresponding to all teeth; then, for each tooth, according to a tooth region where the current tooth is located and the spatial three-dimensional coordinates of the sampling points corresponding to the current tooth, all sampling points belonging to the current tooth are cropped to obtain a cropped block corresponding to the current tooth; next, a preset tooth category query vector and all cropped blocks are input into a trained tooth category recognition model to obtain a preliminary recognition result of tooth categories of all teeth; finally, the preliminary recognition result of the tooth categories is post-processed by a bipartite graph matching method to obtain a final recognition result of tooth categories of all teeth. The application realizes automatic recognition of tooth categories, has high recognition accuracy and good generalization.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a method for tooth identification and classification, an electronic device, and a storage medium. Background Technology

[0002] Digital dental technology has developed rapidly in recent years with the increasing popularity of orthodontics. The main means of achieving this is to analyze and process data from digital dental models, including tooth type identification, tooth alignment, and the magnitude, direction, and point of application of occlusal forces, in conjunction with computer-aided design technology, ultimately resulting in a series of treatment plans.

[0003] Accurate identification of tooth types is a crucial component of digital dental systems. Precise annotation of segmented individual teeth is a vital prerequisite for subsequent dental diagnosis and orthodontic treatment. However, tooth identification is a complex task. Due to individual differences, malocclusion, and the possibility of missing teeth, traditional tooth identification methods suffer from varying degrees of annotation errors. For example, traditional 3D tooth model segmentation methods typically use predefined spatial geometric features such as curvature and normal vectors as reference information for tooth segmentation, matching tooth types using standard tooth templates. This approach is not robust to malocclusion and potential tooth loss, easily leading to unstable segmentation results and misidentification. Furthermore, some traditional methods require some manual interaction, resulting in low efficiency. With the continuous development of deep learning technology in natural and medical image segmentation, significant progress has been made in automatic tooth type identification. However, some methods require a fixed number of teeth for tooth identification and do not support the identification of missing teeth. Other methods, when identifying segmented teeth, do not combine global information of all teeth with local information of individual teeth, sometimes resulting in incorrect identification of teeth of different categories into the same category.

[0004] It should be noted that the information disclosed in the background section of this invention is intended only to enhance the understanding of the general background of this invention, and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address one or more of the problems existing in the prior art, such as low efficiency, low accuracy, and poor generalization of tooth category recognition, by providing a tooth recognition and classification method, electronic device, and storage medium. This invention can not only achieve fully automatic tooth category recognition, but also has high accuracy in tooth category recognition, and has good recognition effect on dentition such as malformed and missing teeth, and has good generalization.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a tooth identification and classification method, comprising:

[0007] The segmented tooth data is obtained, the segmented tooth data is sampled, and each sampling point is preprocessed to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth.

[0008] For each tooth, based on the spatial three-dimensional coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all the sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth; wherein, each clipping block includes a first preset number of sampling points;

[0009] The preset tooth category query vector and all the clipped blocks are input into the trained tooth category recognition model to obtain the preliminary recognition results of the tooth category of all teeth; wherein, the tooth category recognition model includes a neural network model based on an attention mechanism;

[0010] The preliminary identification results of the tooth categories are post-processed using a bipartite graph matching method to obtain the final identification results of the tooth categories for all teeth.

[0011] Optionally, the step of sampling the segmented tooth data and preprocessing each sampling point to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth includes:

[0012] The tooth data of all the segmented teeth are averaged to obtain a second preset number of sampling points; the second preset number is greater than the first preset number.

[0013] The spatial three-dimensional coordinates of the sampling points are standardized to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth; wherein, the standardization process includes making the mean of the spatial three-dimensional coordinates of the sampling points corresponding to all teeth 0 and the variance 1.

[0014] Optionally, for each tooth, based on the spatial three-dimensional coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth, including:

[0015] For each tooth, obtain the three-dimensional spatial coordinates of the tooth center of the current tooth;

[0016] Based on the spatial three-dimensional coordinates of the center of the current tooth and the spatial three-dimensional coordinates of all the sampling points belonging to the current tooth, calculate the distance between each of the sampling points belonging to the current tooth and the center of the current tooth.

[0017] The first preset number of sampling points closest to the center of the current tooth are used as the clipping blocks corresponding to the current tooth.

[0018] Optionally, the tooth category recognition model includes: a first mapping module, an attention encoding module, an attention decoding module, and a second mapping module;

[0019] The process involves inputting a preset tooth category query vector and all the cropped blocks into the trained tooth category recognition model to obtain preliminary recognition results for the tooth categories of all teeth, including:

[0020] The clipping blocks corresponding to all teeth are input into the first mapping module, and the first mapping module extracts and outputs the feature information of the sampling points corresponding to each tooth to the attention encoding module.

[0021] The attention encoding module extracts and outputs the feature information between each tooth and the surrounding teeth to the attention decoding module based on the feature information of the sampling points corresponding to each tooth.

[0022] The attention decoding module extracts and outputs the tooth category of each tooth and the feature information between the tooth categories of the surrounding teeth to the second mapping module based on the preset tooth category query vector and the feature information between each tooth and the surrounding teeth.

[0023] The second mapping module obtains a preliminary identification result of the tooth category of all teeth based on the feature information between the tooth category of each tooth and the tooth categories of the surrounding teeth.

[0024] Optionally, the attention encoding module includes a self-attention module, and the attention decoding module includes a mutual attention module.

[0025] Optionally, the preset tooth category query vector includes feature sequence vectors of m tooth categories; where m represents the number of tooth categories;

[0026] The preliminary identification results of the tooth categories of all teeth include: for each of the m tooth categories, the probability value of each segmented tooth belonging to the current tooth category; m is greater than or equal to the number of segmented teeth.

[0027] Optionally, before inputting the preset tooth category query vector and all the clipped blocks into the trained tooth category recognition model, the tooth recognition and classification method further includes training the tooth category recognition model using the following steps:

[0028] Obtain a tooth category query vector and training samples for training; wherein each training sample includes segmented tooth data for training and label information of the tooth data for training; the label information includes the tooth category corresponding to each tooth of the tooth data for training.

[0029] The initial values ​​of the model parameters of the attention-based neural network model are set; and the pre-built attention-based neural network model is trained using the training samples according to the initial values ​​of the model parameters of the attention-based neural network model until the preset training termination condition is met, so as to obtain the tooth category recognition model.

[0030] Optionally, the preset training termination condition includes: the loss value between the tooth category recognition model's recognition result for each tooth category in the training tooth data and the true value of the tooth category corresponding to each tooth in the training tooth data is less than a preset error threshold, or the number of training iterations exceeds a preset number of training iterations; the loss value is calculated using the following formula:

[0031]

[0032] In the formula, l BCE Let m represent the number of tooth categories, n represent the number of segmented teeth, and p represent the number of segments. ij y represents the probability that the predicted tooth category i corresponds to the j-th segmented tooth, as determined by the tooth category recognition model. ij It is calculated in the following way:

[0033]

[0034] y ij This represents the truth value for the stated tooth category.

[0035] Optionally, the post-processing of the bipartite graph matching method includes: using a 0-1 programming model to determine the final identification result of the tooth category for all teeth.

[0036] To achieve the above objectives, the present invention also provides an electronic device, the electronic device including a processor and a memory, the memory storing a computer program, which, when executed by the processor, implements the tooth identification and classification method described in any of the above claims.

[0037] To achieve the above objectives, the present invention also provides a readable storage medium storing a computer program, which, when executed by a processor, implements the tooth identification and classification method described in any of the preceding claims.

[0038] Compared with existing technologies, the tooth identification and classification method, electronic device, and storage medium provided by this invention have the following advantages:

[0039] The tooth recognition and classification method provided by this invention first acquires segmented tooth data, samples the segmented tooth data, and preprocesses each sampling point to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth. Then, for each tooth, based on the tooth region where the current tooth is located and the spatial three-dimensional coordinates of the sampling points corresponding to the current tooth, all sampling points belonging to the current tooth are cropped to obtain a cropped block corresponding to the current tooth. Each cropped block includes a first preset number of sampling points. Next, a preset tooth category query vector and all the cropped blocks are input into a trained tooth category recognition model to obtain preliminary recognition results of the tooth categories of all teeth. The tooth category recognition model includes a neural network model based on an attention mechanism. Finally, the preliminary recognition results of the tooth categories are post-processed using a bipartite graph matching method to obtain the final recognition results of the tooth categories of all teeth. Therefore, the tooth identification and classification method provided by this invention, without affecting the accuracy of tooth category identification and classification, significantly reduces the amount of data involved in tooth category identification and classification by sampling the segmented tooth data, preprocessing each sampling point, and cropping all sampling points belonging to the current tooth according to the spatial three-dimensional coordinates of the current tooth's region and the sampling point corresponding to the current tooth, thereby improving the efficiency of tooth identification and classification. Furthermore, this invention inputs a preset tooth category query vector and all the cropped blocks into a trained tooth category identification model for identification and classification prediction, and performs post-processing on the preliminary identification results of the tooth category using a bipartite graph matching method. The entire process requires no manual intervention, achieving fully automatic tooth category classification and identification. Moreover, the tooth category identification model is based on an attention mechanism neural network model, which has high accuracy in tooth category identification and good recognition effects for malformed and missing teeth, and has good generalization ability.

[0040] Furthermore, since the electronic device and storage medium provided by this invention belong to the same inventive concept as the tooth identification and classification method provided by this invention, the electronic device and storage medium provided by this invention have at least all the advantages of the tooth identification and classification method provided by this invention. For more detailed information on the advantages of the electronic device and storage medium provided by this invention, please refer to the description related to the tooth identification and classification method above, which will not be repeated here. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the overall process of a tooth identification and classification method provided in one embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the structure of a tooth category recognition model provided in one embodiment of the present invention;

[0043] Figure 3 This is a specific example diagram of the data processing flow for applying the tooth identification and classification method provided by the present invention;

[0044] Figure 4 for Figure 3 Enlarged schematic diagram of each tooth and its corresponding cutting block;

[0045] Figure 5 This is a block diagram of an electronic device according to one embodiment of the present invention.

[0046] The reference numerals in the attached figures are as follows:

[0047] First mapping module - 110, attention encoding module - 120, attention decoding module - 130, second mapping module - 140; clipping block - 300;

[0048] Processor-210, communication interface-220, memory-230, communication bus-240. Detailed Implementation

[0049] The following detailed description, in conjunction with the accompanying drawings, further illustrates the tooth identification and classification method, electronic device, and storage medium proposed in this invention. The advantages and features of this invention will become clearer from the following description. It should be noted that the drawings are in a very simplified form and use non-precise proportions, intended only to facilitate and clearly illustrate the embodiments of this invention. Please refer to the drawings to make the objectives, features, and advantages of this invention more apparent and understandable. It should be understood that the structures, proportions, sizes, etc., depicted in the accompanying drawings are only for illustrative purposes and to enable those skilled in the art to understand and read them, and are not intended to limit the implementation conditions of this invention. Any modifications to the structure, changes in proportions, or adjustments to the size, provided they produce the same or similar effects and achieve the same objectives as this invention, should still fall within the scope of the technical content disclosed in this invention. Specific design features of the invention disclosed herein, including, for example, specific dimensions, orientations, positions, and shapes, will be determined in part by the specific application and usage environment. Furthermore, in the embodiments described below, the same reference numerals are sometimes used across different drawings to denote the same parts or parts with the same function, omitting repeated descriptions. In this specification, similar reference numerals and letters are used to denote similar items; therefore, once an item is defined in one figure, it need not be discussed further in subsequent figures. Furthermore, if the methods described herein involve a series of steps, and the order of these steps presented herein is not necessarily the only possible order in which they can be performed, some of the described steps may be omitted and / or other steps not described herein may be added to the method.

[0050] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The singular forms “a,” “an,” and “the” include plural objects. The term “or” is generally used to mean “and / or,” the term “several” is generally used to mean “at least one,” and the term “at least two” is generally used to mean “two or more.” Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated.

[0051] The core idea of ​​this invention is to provide a tooth identification and classification method, an electronic device, and a storage medium to achieve fully automatic tooth category identification while improving the accuracy of tooth category identification. It also demonstrates good identification performance for malformed or missing teeth and exhibits excellent generalization ability. It should be noted that the tooth identification and classification method provided in this invention can be applied to the electronic device provided in this invention. The electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, tablet computer, or other hardware device with various operating systems.

[0052] To achieve the above-mentioned goals, this invention provides a tooth identification and classification method, please refer to... Figure 1 The diagram illustrates the overall flow of a tooth identification and classification method according to an embodiment of the present invention. Figure 1 As can be seen, the tooth identification and classification method provided in this embodiment includes:

[0053] S100: Obtain the segmented tooth data, sample the segmented tooth data and preprocess each sampling point to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth.

[0054] S200: For each tooth, based on the spatial three-dimensional coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all the sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth; wherein, each clipping block includes a first preset number of sampling points;

[0055] S300: Input the preset tooth category query vector and all the clipping blocks into the trained tooth category recognition model to obtain the preliminary recognition results of the tooth category of all teeth; wherein, the tooth category recognition model includes a neural network model based on an attention mechanism;

[0056] S400: The preliminary identification results of the tooth categories are processed by the bipartite graph matching method to obtain the final identification results of the tooth categories of all teeth.

[0057] The tooth identification and classification method provided in this embodiment, without affecting the accuracy of tooth category identification and classification, significantly reduces the amount of data involved in tooth category identification and classification by sampling the segmented tooth data, preprocessing each sampling point, and cropping all sampling points belonging to the current tooth according to the spatial three-dimensional coordinates of the current tooth's region and the sampling point corresponding to the current tooth, thereby improving the efficiency of tooth identification and classification. Furthermore, this embodiment inputs a preset tooth category query vector and all the cropped blocks into a trained tooth category identification model for identification and classification prediction, and performs post-processing on the preliminary identification results of the tooth category using a bipartite graph matching method. The entire process requires no manual intervention, achieving fully automatic tooth category classification and identification. Moreover, the tooth category identification model is based on an attention mechanism neural network model, which has high accuracy in tooth category identification and good recognition effects for malformed and missing teeth, and has good generalization ability.

[0058] It should be noted that, as those skilled in the art will understand, in order to improve the efficiency of tooth category recognition, the premise for tooth category recognition is based on segmented tooth data. The segmented tooth data includes the number of segmented teeth and the tooth data corresponding to each tooth. The tooth data corresponding to each tooth can be point cloud data or 3D model data reconstructed from point cloud data; this invention does not limit this. However, preferably, the segmented tooth data is point cloud data. Further, in some preferred embodiments, the tooth data corresponding to each tooth includes the spatial 3D coordinates of each point cloud data point and the labeling information of each point cloud data point, the labeling information including whether the point cloud data point is a gingival point or a tooth point. Even further, in some preferred embodiments, the segmented tooth data may also include the spatial 3D coordinates of the center of each tooth.

[0059] In some exemplary embodiments, step S100, which involves sampling the segmented tooth data and preprocessing each sampling point to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth, includes:

[0060] S110: The tooth data of all the segmented teeth are averaged to obtain a second preset number of sampling points; the second preset number is greater than the first preset number;

[0061] S120: Standardize the spatial three-dimensional coordinates of the sampling points to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth; wherein, the standardization process includes making the mean of the spatial three-dimensional coordinates of the sampling points corresponding to all teeth 0 and the variance 1.

[0062] The tooth identification and classification method provided in this embodiment obtains a second preset number of sampling points by averaging the tooth data of all segmented teeth. Therefore, this invention does not limit the number of teeth or their arrangement (e.g., normal dentition, malocclusion, missing teeth, etc.) when performing tooth identification, exhibiting good robustness. Furthermore, this invention standardizes the spatial three-dimensional coordinates of the sampling points, ensuring that the mean of the spatial three-dimensional coordinates of all tooth-corresponding sampling points is 0 and the variance is 1. That is, by normalizing the spatial three-dimensional coordinates of the tooth-corresponding sampling points, the spatial three-dimensional coordinates of all tooth-corresponding sampling points are mapped to the same scale. This ensures that the influence of each sampling point is the same when extracting feature information using the tooth category identification model for subsequent prediction, thereby improving prediction accuracy.

[0063] Specifically, in one preferred embodiment, the average sampling refers to averaging the segmented tooth data of all teeth according to the values ​​of their three-dimensional spatial coordinates. This makes the distribution of the sampling points more balanced, thereby minimizing the adverse effects on tooth classification and identification caused by unreasonable selection of sampling points. Furthermore, the present invention does not limit the specific process of standardization. In one preferred embodiment, the mean of the three-dimensional spatial coordinates of all sampling points corresponding to all teeth can be first reduced to 0, and then the variance of the three-dimensional spatial coordinates of all sampling points corresponding to all teeth can be reduced to 1 based on the mean of 0. For more detailed information, please refer to the relevant descriptions of data normalization in the prior art; they will not be elaborated upon here.

[0064] In some exemplary embodiments, in step S200, for each tooth, based on the spatial three-dimensional coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth, including:

[0065] S210: For each tooth, obtain the three-dimensional spatial coordinates of the tooth center of the current tooth;

[0066] S220: Based on the spatial three-dimensional coordinates of the center of the current tooth and the spatial three-dimensional coordinates of all the sampling points belonging to the current tooth, calculate the distance between each of the sampling points belonging to the current tooth and the center of the current tooth.

[0067] S230: The first preset number of sampling points closest to the center of the current tooth are used as the clipping blocks corresponding to the current tooth.

[0068] The tooth recognition and classification method provided in this embodiment obtains a first preset number of sampling points closest to the center of each tooth by cropping, thus obtaining a cropped block corresponding to the current tooth. Therefore, while ensuring the accuracy of tooth category classification, the amount of data involved in tooth category recognition and classification can be significantly reduced, thereby improving the efficiency of tooth recognition and classification. Furthermore, this embodiment cropps the sampling points according to their distance from the center of each tooth to obtain cropped blocks corresponding to each tooth, thus laying a good foundation for inputting a preset tooth category query vector and all the cropped blocks into a trained tooth category recognition model for recognition, classification, and prediction.

[0069] It should be noted that, as those skilled in the art will understand, this invention does not limit the specific values ​​of the first preset quantity and the second preset quantity. Generally, the larger the values ​​of the first preset quantity and the second preset quantity (i.e., the more sampling points), the higher the recognition accuracy of the tooth category. However, this also increases the requirements for computing resources (including but not limited to storage space, such as memory, higher processor performance, or longer recognition time) during recognition. Conversely, the smaller the values ​​of the first preset quantity and the second preset quantity, especially the first preset quantity, the lower the requirements for computing resources during classification and recognition. However, an excessively small value of the first preset quantity may affect recognition accuracy. In practical applications, these values ​​should be set reasonably according to the actual configuration and requirements of computing resources. Obviously, as those skilled in the art will understand, the value of the first preset quantity is less than the value of the second preset quantity.

[0070] In some exemplary embodiments, please refer to Figures 2 to 4 ,in, Figure 2 This is a schematic diagram of the structure of a tooth category recognition model provided in one embodiment of the present invention; Figure 3 This is a specific example diagram of the data processing flow for applying the tooth identification and classification method provided by the present invention; Figure 4 for Figure 3 Enlarged schematic diagram of each tooth and its corresponding cutting block. From Figure 2 and Figure 3 It can be seen that the tooth category recognition model includes: a first mapping module 110, an attention encoding module 120, an attention decoding module 130, and a second mapping module 140. Correspondingly, step S300, which involves inputting the preset tooth category query vector and all the clipping blocks 300 into the trained tooth category recognition model to obtain preliminary recognition results for the tooth categories of all teeth, includes:

[0071] S310: Input the clipping blocks 300 corresponding to all teeth into the first mapping module 110, and the first mapping module 110 extracts and outputs the feature information of the sampling points corresponding to each tooth to the attention encoding module 120;

[0072] S320: The attention encoding module 120 extracts and outputs the feature information between each tooth and the surrounding teeth to the attention decoding module 130 based on the feature information of the sampling point corresponding to each tooth;

[0073] S330: The attention decoding module 130 extracts and outputs the tooth category of each tooth and the tooth category of the surrounding teeth to the second mapping module 140 based on the preset tooth category query vector and the feature information between each tooth and the surrounding teeth.

[0074] S340: The second mapping module 140 obtains a preliminary identification result of the tooth category of all teeth based on the feature information between the tooth category of each tooth and the tooth categories of the surrounding teeth.

[0075] The tooth recognition and classification method provided in this embodiment extracts the feature information (including the relative position information of each tooth) of the sampling points corresponding to each tooth through the first mapping module 110, thereby obtaining the local information of a single tooth; then, the attention encoding module 120 encodes the feature information between each tooth and surrounding teeth (including the relative position information of each tooth and the similarity information between each tooth and its adjacent teeth) based on the feature information of the sampling points corresponding to each tooth, thereby obtaining the global information of all teeth; then, the attention decoding module 130 uses the preset tooth category query vector to... The method combines the local information of a single tooth with the global information of all teeth, thereby avoiding classifying teeth belonging to different categories into the same category. This improves the accuracy of tooth category recognition and has better robustness, showing good recognition performance for different teeth of different patients.

[0076] Specifically, in some preferred embodiments, the cut blocks 300 obtained after cutting different tooth regions are mapped by the first mapping module 110 to obtain a series of feature sequences X = [X1, ..., X...]. n ], where n represents the number of teeth obtained after trimming. More specifically, the first mapping module 110 and the second mapping module 140 include a multilayer perceptron (MLP). The model parameters of the first mapping module 110 and the second mapping module 140 may be different; please refer to [link to documentation]. Figure 2 and Figure 3The main function of the first mapping module 110 is to extract the feature information of the corresponding sampling points of a single tooth. The main function of the second mapping module 140 is to obtain the classification of each tooth by mapping the feature after decoding by the attention decoding module 130. It should be noted that, as those skilled in the art will understand, a multilayer perceptron (MLP), also called an artificial neural network (ANN), can have multiple hidden layers between the input and output layers, in addition to the input and output layers. The layers of a multilayer perceptron are fully connected. The number of hidden layers can be reasonably set according to actual needs, and this invention does not limit this. Further, the attention encoding module 120 and the attention decoding module 130 are mainly composed of multi-head attention modules. Further, the attention encoding module 120 includes a self-attention module, and the attention decoding module 130 includes a cross-attention module. As mentioned above, the attention encoding module 120 extracts the feature information between each tooth and the surrounding teeth based on the feature information of the sampling points corresponding to each tooth; the attention decoding module 130 extracts the feature information between the tooth category of each tooth and the tooth category of the surrounding teeth based on the preset tooth category query vector and the feature information between each tooth and the surrounding teeth, thereby laying the foundation for the second mapping module 140 to obtain the preliminary recognition results of the tooth category of all teeth.

[0077] More specifically, in one exemplary embodiment, the preset tooth category query vector includes m feature sequence vectors Q = [Q1, ..., Q...] for each tooth category. m In the formula, m represents the number of tooth categories, preferably the maximum number of tooth categories contained in the segmented tooth data, for example, m is 16 (16 is the maximum number of tooth categories contained in the entire set of teeth). Preferably, the preliminary identification result of the tooth categories of all teeth includes: for each of the m tooth categories, the probability value of each segmented tooth belonging to the current tooth category; m is greater than or equal to the number of teeth after segmentation.

[0078] In some exemplary embodiments, before step S300 inputting the preset tooth category query vector and all the clipped blocks into the trained tooth category recognition model, the tooth recognition and classification method further includes training the tooth category recognition model using the following steps:

[0079] S301: Obtain a tooth category query vector and training samples for training; wherein each training sample includes segmented tooth data for training and label information of the tooth data for training; the label information includes the tooth category corresponding to each tooth of the tooth data for training.

[0080] S302: Set the initial values ​​of the model parameters of the attention-based neural network model; and train the pre-built attention-based neural network model using the training samples according to the initial values ​​of the model parameters of the attention-based neural network model until the preset training termination condition is met, thereby obtaining the tooth category recognition model.

[0081] Specifically, preferably, the preset training termination condition includes: the loss value between the tooth category recognition model's recognition result for each tooth category in the training tooth data and the true value of the tooth category corresponding to each tooth in the training tooth data is less than a preset error threshold, or the number of training iterations exceeds a preset number of training iterations; the loss value is calculated using the following formula:

[0082]

[0083] In the formula, l BCE Let m represent the number of tooth categories, n represent the number of segmented teeth, and p represent the number of segments. ij y represents the probability that the predicted tooth category i corresponds to the j-th segmented tooth, as determined by the tooth category recognition model. ij It is calculated in the following way:

[0084]

[0085] y ij This represents the truth value of the tooth category, that is, when the tooth category of tooth j is i, y ij The value of y is 1; otherwise, y ij The value of is 0.

[0086] Specifically, the present invention does not limit the numbering order of each tooth. For example, in one exemplary embodiment, the teeth arranged continuously from left to right can be numbered 0, 1, 2, 3... in sequence.

[0087] In particular, since the model training process is actually a process of minimizing the loss function, it is preferable to use gradient descent to optimize the difference, which can quickly and easily achieve the training of the tooth category recognition model.

[0088] It should be noted that the present invention does not limit the order of steps S301 and S302 and steps S100 and S200, that is, it does not limit the training time of the tooth category recognition model. Steps S301 and S302 can be before or after steps S100 and S200.

[0089] In one exemplary embodiment, step S400, the post-processing of the bipartite graph matching method, includes using a 0-1 programming model to determine the final identification result of the tooth category for all teeth. Therefore, the tooth identification and classification method provided in this embodiment, by first using an attention-based neural network model to identify and predict the segmented tooth results, and then performing bipartite graph matching post-processing on the classification probabilities, eliminates the need for manual construction of objective functions, setting thresholds, or other manual interactions, directly obtaining the identification result and achieving fully automatic tooth classification and identification.

[0090] Specifically, the final identification result of tooth j corresponding to tooth category i is c. ij In other words, when the final identification result of tooth j is the i-th category, c ij =1. Otherwise, c ij =0.

[0091] More specifically, the final recognition result c ij The following planning model can be used to solve the problem:

[0092]

[0093]

[0094] In the above formula, p ij This represents the probability that the predicted tooth category i corresponds to the j-th segmented tooth by the tooth category recognition model.

[0095] Another embodiment of the present invention also provides an electronic device, please refer to Figure 5 The diagram illustrates a block structure of an electronic device according to an embodiment of the present invention. Figure 5As shown, the electronic device includes a processor 210 and a memory 230. The memory 230 stores a computer program. When the computer program is executed by the processor 210, it implements the tooth identification and classification method described in any of the above embodiments. Since the electronic device provided in this embodiment belongs to the same inventive concept as the tooth identification and classification methods provided in the above embodiments, the electronic device provided in this embodiment has at least all the advantages of the tooth identification and classification methods provided in the above embodiments. For more detailed information on the advantages of the electronic device provided in this embodiment, please refer to the description related to the tooth identification and classification methods above, which will not be repeated here.

[0096] like Figure 5 As shown, the electronic device also includes a communication interface 220 and a communication bus 240, wherein the processor 210, the communication interface 220, and the memory 230 communicate with each other via the communication bus 240. The communication bus 240 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 240 can be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus. The communication interface 220 is used for communication between the aforementioned electronic device and other devices.

[0097] The processor 210 referred to in this invention can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor 210 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines.

[0098] The memory 230 can be used to store the computer program. The processor 210 implements various functions of the electronic device by running or executing the computer program stored in the memory 230 and calling the data stored in the memory 230.

[0099] The memory 230 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0100] Another embodiment of the present invention provides a computer-readable storage medium storing a computer program. When executed by a processor, the computer program can implement the tooth identification and classification method described in any of the above embodiments. Since the computer-readable storage medium provided in this embodiment belongs to the same inventive concept as the tooth identification and classification methods provided in the above embodiments, it possesses at least all the advantages of the tooth identification and classification methods provided in the above embodiments. For more detailed information on the advantages of the computer-readable storage medium provided in this embodiment, please refer to the description related to the tooth identification and classification methods above; further details will not be repeated here.

[0101] The readable storage medium of embodiments of the present invention can be any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. Computer-readable storage media can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: electrical connections having one or more wires, portable computer hard disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, apparatus, or device.

[0102] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0103] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] In summary, compared with the prior art, the tooth recognition and classification method, electronic device, and storage medium provided by this invention have the following advantages: The tooth recognition and classification method provided by this invention, without affecting the accuracy of tooth category recognition and classification, significantly reduces the amount of data involved in tooth category recognition and classification by sampling the segmented tooth data, preprocessing each sampling point, and cropping all sampling points belonging to the current tooth according to the spatial three-dimensional coordinates of the current tooth's region and the sampling point corresponding to the current tooth, thereby improving the efficiency of tooth recognition and classification. Furthermore, this invention inputs a preset tooth category query vector and all the cropped blocks into a trained tooth category recognition model for recognition and classification prediction, and performs post-processing on the preliminary recognition results of the tooth category using a bipartite graph matching method. The entire process requires no manual intervention, achieving fully automatic tooth category classification and recognition. Moreover, the tooth category recognition model is based on an attention-based neural network model, exhibiting high accuracy in tooth category recognition and good recognition effects for various dentitions such as malformed and missing teeth, demonstrating excellent generalization ability.

[0105] Furthermore, since the electronic device and storage medium provided by this invention belong to the same inventive concept as the tooth identification and classification method provided by this invention, the electronic device and storage medium provided by this invention have at least all the advantages of the tooth identification and classification method provided by this invention. For more detailed information on the advantages of the electronic device and storage medium provided by this invention, please refer to the description related to the tooth identification and classification method above, which will not be repeated here.

[0106] It should be noted that the apparatus and methods disclosed in the embodiments herein can also be implemented in other ways. The apparatus embodiments described above are merely illustrative; for example, the flowcharts and block diagrams in the accompanying drawings show the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, program, or part of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system to perform the specified function or action, or can be implemented using a combination of dedicated hardware and computer instructions.

[0107] In addition, the functional modules in the various embodiments of this article can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0108] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure are within the protection scope of the present invention. Obviously, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A method for identifying and classifying teeth, characterized in that, include: The segmented tooth data is obtained, the segmented tooth data is sampled, and each sampling point is preprocessed to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth. For each tooth, based on the three-dimensional spatial coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all the sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth; wherein, each clipping block includes a first preset number of sampling points; The preset tooth category query vector and all the clipped blocks are input into the trained tooth category recognition model to obtain the preliminary recognition results of the tooth category of all teeth; wherein, the tooth category recognition model includes a neural network model based on an attention mechanism; The preliminary identification results of the tooth categories are post-processed using a bipartite graph matching method to obtain the final identification results of the tooth categories for all teeth. The tooth category recognition model includes: a first mapping module, an attention encoding module, an attention decoding module, and a second mapping module; The process involves inputting a preset tooth category query vector and all the cropped blocks into the trained tooth category recognition model to obtain preliminary recognition results for the tooth categories of all teeth, including: The clipping blocks corresponding to all teeth are input into the first mapping module, and the first mapping module extracts and outputs the feature information of the sampling points corresponding to each tooth to the attention encoding module. The attention encoding module extracts and outputs the feature information between each tooth and the surrounding teeth to the attention decoding module based on the feature information of the sampling points corresponding to each tooth. The attention decoding module extracts and outputs the tooth category of each tooth and the feature information between the tooth categories of the surrounding teeth to the second mapping module based on the preset tooth category query vector and the feature information between each tooth and the surrounding teeth. The second mapping module obtains a preliminary identification result of the tooth category of all teeth based on the feature information between the tooth category of each tooth and the tooth categories of the surrounding teeth.

2. The tooth identification and classification method according to claim 1, characterized in that, The step of sampling the segmented tooth data and preprocessing each sampling point to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth includes: The tooth data of all the segmented teeth are averaged to obtain a second preset number of sampling points; the second preset number is greater than the first preset number. The spatial three-dimensional coordinates of the sampling points are standardized to obtain the spatial three-dimensional coordinates of the sampling points corresponding to all teeth; wherein, the standardization process includes making the mean of the spatial three-dimensional coordinates of the sampling points corresponding to all teeth 0 and the variance 1.

3. The tooth identification and classification method according to claim 1, characterized in that, For each tooth, based on the spatial three-dimensional coordinates of the tooth region where the current tooth is located and the sampling point corresponding to the current tooth, all sampling points belonging to the current tooth are clipped to obtain a clipping block corresponding to the current tooth, including: For each tooth, obtain the three-dimensional spatial coordinates of the tooth center of the current tooth; Based on the spatial three-dimensional coordinates of the center of the current tooth and the spatial three-dimensional coordinates of all the sampling points belonging to the current tooth, calculate the distance between each sampling point belonging to the current tooth and the center of the current tooth. The first preset number of sampling points closest to the center of the current tooth are used as the clipping blocks corresponding to the current tooth.

4. The tooth identification and classification method according to claim 1, characterized in that, The attention encoding module includes a self-attention module, and the attention decoding module includes a mutual attention module.

5. The tooth identification and classification method according to claim 1, characterized in that, The preset tooth category query vector includes feature sequence vectors of m tooth categories; where m represents the number of tooth categories. The preliminary identification results of the tooth categories of all teeth include: for each of the m tooth categories, the probability value of each segmented tooth belonging to the current tooth category; m is greater than or equal to the number of segmented teeth.

6. The tooth identification and classification method according to claim 1, characterized in that, Before inputting the preset tooth category query vector and all the clipped blocks into the trained tooth category recognition model, the tooth recognition and classification method further includes training the tooth category recognition model using the following steps: Obtain a tooth category query vector and training samples for training; wherein each training sample includes segmented tooth data for training and label information of the tooth data for training; the label information includes the tooth category corresponding to each tooth of the tooth data for training. The initial values ​​of the model parameters of the attention-based neural network model are set; and the pre-built attention-based neural network model is trained using the training samples according to the initial values ​​of the model parameters of the attention-based neural network model until the preset training termination condition is met, so as to obtain the tooth category recognition model.

7. The tooth identification and classification method according to claim 6, characterized in that, The preset training termination conditions include: the loss value between the tooth category recognition model's recognition result for each tooth category in the training tooth data and the true value of the tooth category corresponding to each tooth in the training tooth data is less than a preset error threshold, or the number of training iterations exceeds a preset number of training iterations; the loss value is calculated using the following formula: In the formula, This represents the loss value. Indicates the number of tooth categories. This indicates the number of teeth after the segmentation. This represents the predicted first tooth category by the tooth category recognition model. The tooth category of class corresponds to the first The probability of a segmented tooth. It is calculated in the following way: This represents the truth value for the stated tooth category.

8. The tooth identification and classification method according to claim 1, characterized in that, The post-processing of the bipartite graph matching method includes: using a 0-1 programming model to determine the final identification result of the tooth category of all teeth.

9. An electronic device, characterized in that, It includes a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the tooth identification and classification method according to any one of claims 1 to 8.

10. A readable storage medium, characterized in that, The readable storage medium stores a computer program, which, when executed by a processor, implements the tooth identification and classification method according to any one of claims 1 to 8.

Images