Residual crown identification method, model training method, device, equipment and storage medium

Abstract

The present application provides a residual crown recognition method, a model training method, an apparatus, a device and a storage medium. The residual crown recognition method comprises: obtaining a first single tooth model to be recognized and the spatial information of the first single tooth model in the first tooth model; using a pre-trained residual crown recognition model to perform residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model, and obtaining a residual crown recognition result corresponding to the first single tooth model; wherein the residual crown recognition model is obtained through supervised training based on a number of training samples, and the training samples include a second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model. The residual crown recognition efficiency and recognition accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a residual crown recognition method, a residual crown recognition model training method, a residual crown recognition device, an electronic device and a computer-readable storage medium. Background Art

[0002] In digital oral scanning scenarios, for example, in the detection and identification process of common oral diseases such as residual crowns, doctors will generate corresponding oral inspection reports based on the patient's 3D tooth model in order to analyze and summarize the patient's oral problems. This process usually requires doctors to manually rotate the 3D tooth model, identify and judge oral diseases based on their own experience, and manually take screenshots and feedback to the patient. This method requires doctors to have high clinical experience, is cumbersome to operate, and requires a lot of time and labor costs. Summary of the invention

[0003] In view of this, the present application provides a method for identifying a residual crown, a training method for a residual crown recognition model, a residual crown recognition device, an electronic device and a computer-readable storage medium.

[0004] Specifically, the present application is implemented through the following technical solutions:

[0005] According to a first aspect of an embodiment of the present application, a method for identifying a residual crown is provided, comprising:

[0006] Acquire a first single tooth model to be identified and spatial information of the first single tooth model in the first tooth model;

[0007] Using a pre-trained residual crown recognition model, performing residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model to obtain a residual crown recognition result corresponding to the first single tooth model;

[0008] The residual crown recognition model is obtained through supervised training based on several training samples, wherein the training samples include a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

[0009] According to a second aspect of an embodiment of the present application, a training method for a residual crown recognition model is provided, comprising:

[0010] Acquire a plurality of training samples, wherein the training samples include a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating residual crown information of the second single tooth model; different training samples correspond to different second single tooth models;

[0011] inputting the training samples into the residual crown recognition model to be trained, so that the residual crown recognition model to be trained performs residual crown recognition processing based on the second single tooth model and the spatial information of the second single tooth model to obtain a residual crown recognition result corresponding to the second single tooth model, and training the residual crown recognition model to be trained with minimizing the difference between the residual crown recognition result and the label as the optimization goal;

[0012] Among them, the trained residual crown recognition model is applied to the residual crown recognition method as described in the first aspect.

[0013] According to a third aspect of an embodiment of the present application, a residual crown identification device is provided, comprising:

[0014] A model to be identified and spatial information acquisition module, used to acquire a first single tooth model to be identified and spatial information of the first single tooth model in the first tooth model;

[0015] a residual crown recognition module, configured to perform residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model using a pre-trained residual crown recognition model, and obtain a residual crown recognition result corresponding to the first single tooth model;

[0016] The residual crown recognition model is obtained through supervised training based on several training samples, wherein the training samples include a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

[0017] According to a fourth aspect of an embodiment of the present application, a training device for a residual crown recognition model is provided, comprising:

[0018] A training sample acquisition module, used to acquire a plurality of training samples, wherein the training samples include a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating residual crown information of the second single tooth model; different training samples correspond to different second single tooth models;

[0019] a training module, used for inputting the training samples into the residual crown recognition model to be trained, so that the residual crown recognition model to be trained performs residual crown recognition processing based on the second single tooth model and the spatial information of the second single tooth model, obtains the residual crown recognition result corresponding to the second single tooth model, and trains the residual crown recognition model to be trained with minimizing the difference between the residual crown recognition result and the label as the optimization goal;

[0020] Among them, the trained residual crown recognition model is applied to the residual crown recognition method as described in the first aspect.

[0021] According to a fifth aspect of an embodiment of the present application, there is provided an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0022] Wherein, when the processor executes the executable instructions, the steps of the method described in the first aspect are implemented.

[0023] According to a sixth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the computer instructions are executed by a processor, the steps of the method described in the first aspect are implemented.

[0024] According to a seventh aspect of an embodiment of the present application, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the method described in the first aspect when executed by a processor.

[0025] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0026] The training method of the residual crown recognition model provided in the embodiment of the present application constructs a rich training data set by acquiring multiple different second single tooth models and the spatial information of the second single tooth models in the corresponding second tooth models, and labeling each second single tooth model with the residual crown information, so that the model can comprehensively learn the surface features and spatial features of different single teeth, thereby enhancing the recognition accuracy and effect of the model in the residual crown recognition task.

[0027] The residual crown identification method provided in the embodiment of the present application is based on the multi-dimensional feature information of a single tooth and combined with the use of a pre-trained residual crown identification model to automatically realize the rapid and accurate identification of residual crowns, significantly improve the accuracy and reliability of residual crown identification, and provide doctors with more powerful residual crown diagnosis support.

[0028] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings, which are incorporated in the specification and constitute a part of this application, illustrate embodiments consistent with the application and, together with the description, serve to explain the principles of the application.

[0030] Figure 1 It is a flowchart of a training method for a residual crown recognition model shown in an exemplary embodiment of the present application.

[0031] Figure 2 It is a schematic diagram showing segmentation processing of a three-dimensional tooth model according to an exemplary embodiment of the present application.

[0032] Figure 3It is a schematic diagram showing a completion process for a single tooth model according to an exemplary embodiment of the present application.

[0033] Figure 4 It is a schematic diagram of recording the residual crown tooth number input by the user, shown in an exemplary embodiment of the present application.

[0034] Figure 5 It is a schematic diagram of a residual crown recognition model performing residual crown recognition processing as shown in an exemplary embodiment of the present application.

[0035] Figure 6 It is a flowchart of a residual crown identification method shown in an exemplary embodiment of the present application.

[0036] Figure 7 It is a schematic diagram showing an exemplary embodiment of the present application for automatically taking screenshots of the residual crown identified in the tooth model.

[0037] Figure 8 It is a structural schematic diagram of a training device for a residual crown recognition model shown in an exemplary embodiment of the present application.

[0038] Fig. 9 It is a structural schematic diagram of a residual crown identification device shown in an exemplary embodiment of the present application.

[0039] Fig.10 It is a structural schematic diagram of an electronic device shown in an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0040] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0041] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0042] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0043] In digital oral scanning scenarios, for example, in the detection and identification process of common oral diseases such as residual crowns, doctors will generate corresponding oral inspection reports based on the patient's 3D tooth model in order to analyze and summarize the patient's oral problems. This process usually requires doctors to manually rotate the 3D tooth model, identify and judge oral diseases based on their own experience, and manually take screenshots and feedback to the patient. This method requires doctors to have high clinical experience, is cumbersome to operate, and requires a lot of time and labor costs, and there may be cases of misjudgment and missed detection of oral diseases.

[0044] Here, the process of generating a 3D tooth model is described by way of example. An oral scanner can be used to obtain a 3D tooth model of a patient. A dentist operates the oral scanner and slowly moves the scanning head of the oral scanner in the patient's mouth to capture the image of the teeth. The scanning head emits a low-power light beam to illuminate the teeth. After the light beam is reflected back, it is captured by the scanner's sensor and the precise position of each point on the tooth surface is calculated through the triangulation principle. A 3D tooth model can be generated based on the data collected by the scanner. However, this is not limited to the above, and other methods can also be used to obtain a 3D tooth model.

[0045] In the field of stomatology, residual crown is defined as a tooth with a missing part of more than half, and the treatment plan tends to be tooth extraction. The biggest difference from the diagnostic criteria of other general oral diseases (such as caries, tartar, pigmentation) is that residual crown has a limit on the volume of the missing part, and only a part with a missing part of more than half can be diagnosed as a residual crown, and the relevant technology does not provide an automatic recognition solution for residual crowns.

[0046] Based on this, the embodiments of the present application provide a training method for a residual crown recognition model and a residual crown recognition method, which can assist doctors in quickly and accurately locating and identifying residual crowns with the help of a pre-trained residual crown recognition model, and use a single tooth model and the spatial information of the single tooth model on the tooth model for AI (Artificial Intelligence) prediction, thereby reducing the doctor's operating steps and improving recognition efficiency. In addition, by referring to the spatial information of the single tooth model on the tooth model, the missing parts of the tooth can be fully and comprehensively observed and identified, thereby improving the recognition accuracy of residual crown recognition.

[0047] Next, the training process of the residual crown recognition model is illustrated as follows:

[0048] The training method of the residual crown recognition model provided in the embodiment of the present application can be executed by electronic devices, including but not limited to physical servers, server machines, cloud servers, smart phones / mobile phones, tablet computers, personal digital assistants (PDAs), laptop computers, and desktop computers.

[0049] See also Figure 1 , a flow chart of a training method for a residual crown recognition model is shown, the method comprising:

[0050] In S101, a number of training samples are obtained, the training samples including a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating residual crown information of the second single tooth model; different training samples correspond to different second single tooth models.

[0051] By acquiring multiple different second single tooth models and the spatial information of each second single tooth model in the corresponding second tooth model, and labeling the residual crown information for each second single tooth model, a rich training data set was constructed. This training method directly targeting single tooth models can effectively avoid the problem of mutual occlusion between teeth in a complete tooth model, thereby ensuring that the model can fully capture and learn the surface features of each single tooth, and enhance the adaptability and performance of the model in the residual crown recognition task.

[0052] In addition, considering that in the identification of residual crowns, in addition to the surface features of a single tooth, the spatial features of a single tooth are equally important, the present embodiment also incorporates the spatial information of the second single tooth model on the corresponding second tooth model into the training samples, so that the model can also accurately identify and understand the spatial features of a single tooth, and then evaluate the health status of the teeth from multiple dimensions, thereby improving the accuracy and effectiveness of residual crown identification.

[0053] Each second single-tooth model is equipped with an accurate residual crown label, which is used as true value data to guide the training of the model, so that the model can learn to identify single-tooth residual crowns with different characteristics, thereby improving the accuracy and reliability of recognition. For example, if the volume of the defective part of the second single-tooth model exceeds half, its corresponding label is used to indicate that the second single-tooth model is a residual crown; otherwise, its corresponding label is used to indicate that the second single-tooth model is not a residual crown.

[0054] In S102, the training samples are input into the residual crown recognition model to be trained, so that the residual crown recognition model to be trained performs residual crown recognition processing based on the second single tooth model and the spatial information of the second single tooth model to obtain the residual crown recognition result corresponding to the second single tooth model, and the residual crown recognition model to be trained is trained with minimizing the difference between the residual crown recognition result and the label as the optimization goal; wherein the trained residual crown recognition model is used to perform residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model, so as to obtain the residual crown recognition result corresponding to the first single tooth model.

[0055] It can be understood that, for the sake of convenience of distinction, in the embodiment of the present application, the single-tooth model used as a training sample in the model training process is called the second single-tooth model, and the single-tooth model to be identified in the model application process is called the first single-tooth model.

[0056] Exemplarily, the spatial information of the second single tooth model includes at least one of the volume of the second single tooth model and the tooth position number of the second single tooth model in the second tooth model.

[0057] Among them, volume can help evaluate the degree of tooth defect. For example, when the missing part of a tooth exceeds a certain proportion of its original volume (such as more than one half), this usually means that the tooth structure is severely damaged and may meet the identification criteria of residual crowns. Therefore, volume can be used as key spatial information for the model to identify residual crowns; and the tooth position number can provide specific position information of the tooth in the mouth, because the identification criteria of residual crowns may often vary depending on the tooth position. For example, the identification criteria of residual crowns for front teeth and back teeth may be different. Through the tooth position number, the model can identify the specific position of the tooth, and thus identify the residual crown according to the specific criteria of the position, thereby improving the accuracy of model recognition.

[0058] In some embodiments, the second single tooth model included in each training sample and the process of obtaining the tooth position number and volume of the second single tooth model are exemplarily described:

[0059] The electronic device first obtains the second tooth model to be identified, identifies and segments each tooth in the second tooth model, and obtains multiple second single tooth models and tooth position numbers corresponding to the multiple second single tooth models. Specifically, the tooth model can be segmented using a pre-trained instance segmentation model or other types of segmentation methods, and this embodiment does not impose any restrictions on this. Among them, the instance segmentation model can be any deep learning model that can identify and segment different objects in the image. For example, in the field of stomatology, a convolutional neural network (CNN) architecture is usually used, such as U-Net, Mask R-CNN, etc. These models are trained with a large amount of tooth image data and can accurately identify the contours of each tooth and segment them from the complete tooth model to form an independent single tooth model, thereby avoiding the problem of mutual occlusion between teeth in the tooth model. As for the tooth position number, it is the position identifier of the tooth in the mouth, and each tooth can have a unique corresponding tooth position number.

[0060] See for example Figure 2 , showing a schematic diagram of segmenting a three-dimensional tooth model to obtain multiple single teeth.

[0061] Next, the electronic device completes the second single tooth model obtained by segmentation, so that the completed second single tooth model forms a closed solid model, and the volume of the completed second single tooth model is determined as the volume of the second single tooth model. Since the single tooth model is obtained by segmentation, it may be defective or incomplete. At this time, directly calculating the volume cannot accurately reflect the actual condition of the tooth, which in turn affects the accuracy of residual crown identification. Therefore, in order to improve the accuracy of model recognition, the single tooth model needs to be completed in advance. The completion method can use a computer to automatically reconstruct the missing part of the tooth, or it can be manually completed by the doctor using an interactive device (such as a mouse, keyboard, touch screen, etc.). This embodiment does not impose any restrictions on this.

[0062] See for example Figure 3 , showing a schematic diagram of completing a single tooth model.

[0063] In some embodiments, the process of obtaining a label indicating the residual crown information of the second single tooth model in the training sample is exemplarily described:

[0064] The electronic device can display the second tooth model to be annotated in the display interface, so that the user can quickly view and annotate. The user can determine the residual crown and the position of the residual crown in the second tooth model based on professional experience, and then input the corresponding tooth position number to the electronic device through an input device (such as a mouse, keyboard, touch screen, etc.). In response to the tooth position number input operation for the second tooth model, the electronic device obtains at least one tooth position number for indicating the residual crown in the second tooth model, and then for the multiple second single tooth models obtained by segmenting the second tooth model, the label of the second single tooth model corresponding to the input tooth position number is determined as indicating that the second single tooth model is a residual crown, and the labels of the remaining other second single tooth models are determined as indicating that the second single tooth model is not a residual crown.

[0065] Exemplarily, the tool for the user to view the second tooth model may be various three-dimensional software, such as MeshLab, ParaView, etc., and the tool for recording the residual crown tooth position number input by the user may be a spreadsheet software such as Microsoft Excel. The marking method provided in this embodiment only needs to mark the tooth position number, which is simple and direct, which is conducive to simplifying the marking steps of the user and improving the marking efficiency.

[0066] See for example Figure 4 , showing a schematic diagram of recording the residual crown tooth number input by the user. Figure 4 The data name in refers to the name of each second tooth model, and the residual crown tooth position number is the tooth position number of the tooth belonging to the residual crown in each second tooth model input by the user.

[0067] In some embodiments, after obtaining the training samples based on the above method, the electronic device can input the training samples into the residual crown recognition model to be trained, and execute the following process in a loop until the optimization target is met: using the current residual crown recognition model to perform residual crown recognition based on the second single tooth model and the spatial information of the second single tooth model, and obtain the residual crown recognition result corresponding to the second single tooth model; based on the difference between the residual crown recognition result and the label corresponding to the second single tooth model, determine whether the optimization target is met; if the optimization target is not met, based on the difference between the second single tooth model and the label corresponding to the second single tooth model, adjust the model parameters of the current residual crown recognition model, and use the adjusted residual crown recognition model as the current residual crown recognition model in the next cycle. Among them, to determine whether the optimization target is met, the loss function of the model can be used for judgment. The loss function can include but is not limited to the cross entropy loss function, the Dice loss function, the mean square error loss function, the IoU loss function (Intersection over Union Loss) or the bounding box regression loss, etc. This embodiment does not limit the loss function used in the training process.

[0068] During the training process, through the second single tooth model combined with its corresponding spatial information, the residual crown recognition model can learn the surface features and spatial features of a single tooth more comprehensively and accurately, and integrate these features to perform residual crown recognition, thereby improving the accuracy and reliability of recognition.

[0069] In this embodiment, the trained residual crown recognition model can greatly improve the efficiency and accuracy of doctors in diagnosing residual crowns. The residual crown recognition model can quickly process single tooth models and their spatial information, automatically identify residual crowns, and thus provide powerful auxiliary support for doctors.

[0070] In some embodiments, see Figure 5 The residual crown recognition model includes a first classification network and a second classification network. The following is an exemplary description of the process of performing residual crown recognition processing on the residual crown recognition model based on the second single tooth model and the spatial information of the second single tooth model:

[0071] The electronic device inputs the second single tooth model and the spatial information of the second single tooth model into the residual crown recognition model, and firstly extracts the features of the second single tooth model by the first classification network in the residual crown recognition model to obtain the tooth surface features. Exemplarily, the tooth surface features include at least one of the texture features and geometric features of the second single tooth model. The geometric features may include but are not limited to the curvature, depth, normal vector, etc. of the three-dimensional point in the single tooth model.

[0072] After the tooth surface features are extracted by the first classification network, the second classification network in the residual crown recognition model performs residual crown recognition processing based on the obtained tooth surface features and the input spatial information.

[0073] In this embodiment, the residual crown recognition model adopts a two-level network structure, so that different network structures can focus on processing different tasks, thereby reducing the complexity of the model, reducing the risk of overfitting, and ensuring the stability of the model. Among them, the first classification network focuses on feature extraction; the second classification network focuses on combining tooth surface features and spatial information to classify and identify residual crowns. The residual crown recognition result finally output by the second classification network can be a probability value, which is used to indicate the probability that the input second single tooth model is a residual crown. For example, if the output probability value exceeds 0.5, the second single tooth model is considered to be a residual crown; or, the second classification network can also output a classification label, directly indicating whether the single tooth model is a residual crown. For example, this label can be a binary classification (residual crown / non-residual crown) and / or a more complex classification (different degrees or categories of residual crowns, etc.).

[0074] In some embodiments, the spatial information of the second single tooth model includes the volume of the second single tooth model; in the process of the electronic device performing residual crown recognition processing by the second classification network in the residual crown recognition model based on the tooth surface features and the spatial information of the second single tooth model, the second classification network can first calculate the defect volume of the second single tooth model, and if the defect volume and tooth surface features of the second single tooth model meet at least one of the following conditions, the residual crown recognition result indicating that the second single tooth model is a residual crown is output: the defect volume exceeds half of the volume of the second single tooth model; the texture features belong to the residual crown color range; the geometric features show a residual crown change trend.

[0075] Residual crowns are generally caused by caries, and can be determined by analyzing and identifying the volume of single tooth defects and tooth surface features. Specifically, if the volume of a single tooth defect exceeds half of its total volume, it can be considered severely damaged, and the tooth is determined to be a residual crown. Alternatively, if the texture characteristics of the tooth are significantly different in color from normal teeth, such as showing a reddish color range that is unique to residual crowns, the tooth can also be identified as a residual crown. Alternatively, residual crowns usually also show an uneven surface or pits, and have greater curvature and depth information than normal teeth. On the other hand, the normal vector of a normal tooth usually changes smoothly and continuously, while the normal vector of a residual crown may mutate. If the geometric characteristics of the tooth show the above-mentioned trend of residual crown changes, the tooth can also be identified as a residual crown.

[0076] After outputting the residual crown recognition result indicating that the second single tooth model is a residual crown through the above embodiment, the electronic device may further determine the residual crown category of the second single tooth model according to the defect volume, texture features and geometric features.

[0077] Specifically, the severity of the residual crown can be determined based on the defect volume, texture features and geometric features, and the residual crown can be divided into different levels of residual crowns such as primary residual crowns and secondary residual crowns. For example, a primary residual crown may refer to a relatively mild tooth defect, for example, the defect volume does not exceed half of the total volume, the texture feature changes are not obvious, and the geometric feature changes are small, such as the curvature does not increase significantly, the pit depth is shallow, and the normal vector changes gently. This type of residual crown may restore the tooth function through simple repair or filling treatment. A secondary residual crown may indicate a more serious tooth defect, for example, the defect volume exceeds half of the total volume, the texture features change significantly, the color is significantly different from that of normal teeth, and the geometric features change significantly, such as the curvature increases significantly, the pit depth is large, and the normal vector has a mutation. This type of residual crown may require more complex treatment options, such as crown restoration or root canal treatment, to restore the structure and function of the tooth. By further determining the category of residual crowns, electronic devices can provide more precise guidance for subsequent clinical treatment.

[0078] It is understandable that this embodiment does not limit the specific network architecture of the residual crown recognition model and the output form of the second classification network. The first classification network and the second classification network can be a convolutional neural network (CNN), a recurrent neural network (RNN), a long short-term memory network (LSTM), a residual network (ResNet), and an attention mechanism network, but are not limited thereto.

[0079] It is understandable that the electronic device used to train the residual crown recognition model and the electronic device used to perform the residual crown recognition method can be the same device or different devices, and this embodiment does not impose any restrictions on this. For example, the residual crown recognition model can be trained by a certain electronic device, and then the trained tooth residual crown recognition model can be transplanted to another electronic device used to perform the residual crown recognition method.

[0080] The following is an exemplary description of the residual crown identification method provided in the embodiment of the present application:

[0081] See also Figure 6 , a schematic flow chart of a residual crown identification method is shown, which can be performed by an electronic device, including but not limited to a physical server, a server machine, a cloud server, a smart phone / mobile phone, a tablet computer, a personal digital assistant (PDA), a laptop computer, and a desktop computer. The method includes:

[0082] In S601, a first single tooth model to be identified and spatial information of the first single tooth model in the first tooth model are obtained;

[0083] By acquiring the first single tooth model and the spatial information of the first single tooth model in the first tooth model, comprehensive and rich surface feature and spatial feature information of a single tooth can be provided, which in turn helps to comprehensively evaluate the health status of the teeth from multiple dimensions and improve the accuracy and effectiveness of residual crown identification.

[0084] In S602, a pre-trained residual crown recognition model is used to perform residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model to obtain a residual crown recognition result corresponding to the first single tooth model; wherein the residual crown recognition model is obtained through supervised training based on a number of training samples, and the training samples include the second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

[0085] This embodiment uses the trained residual crown recognition model to perform residual crown recognition processing, which can effectively avoid the tedious manual operations in the traditional manual recognition method, reduce the time cost and labor cost, and realize fast and accurate residual crown recognition automatic processing. In addition, the automatic processing can also ensure the consistency and repeatability of the recognition process, provide doctors with fast and accurate diagnosis support, thereby improving the overall clinical work efficiency and patient satisfaction.

[0086] The residual crown identification method provided in this embodiment can quickly and accurately identify the residual crown based on the multi-dimensional feature information of a single tooth and combined with a high-precision, automated residual crown identification model, significantly improving the accuracy and reliability of residual crown identification and providing doctors with more powerful residual crown diagnosis support.

[0087] Exemplarily, the spatial information of the first single tooth model includes at least one of the volume of the first single tooth model and the tooth position number of the first single tooth model in the first tooth model. In this embodiment, the volume and the tooth position number are used as the spatial information of the single tooth model, which helps the model to accurately identify the degree of tooth defect and position information, thereby improving the accuracy and reliability of residual crown identification.

[0088] In some embodiments, for the first single tooth model and the process of acquiring the tooth position number and volume of the first single tooth model, similar to the training process, the electronic device can first acquire the first tooth model to be identified, identify and segment each tooth in the first tooth model, and obtain multiple first single tooth models and the tooth position numbers corresponding to the multiple first single tooth models. Then, the electronic device completes the segmented first single tooth model so that the completed first single tooth model forms a closed solid model, and determines the volume of the completed first single tooth model as the volume of the first single tooth model. This embodiment can divide the first tooth model into multiple independent single tooth models through an instance segmentation model, avoiding the problem of mutual occlusion between teeth in the tooth model, ensuring that the model can fully capture the surface and spatial features of each single tooth, and significantly improving the accuracy of residual crown recognition. At the same time, this embodiment also completes the single tooth model to ensure the accuracy of volume calculation, further improving the reliability of residual crown recognition.

[0089] In some embodiments, the residual crown recognition model includes a first classification network and a second classification network. For the process of the residual crown recognition model performing residual crown recognition processing based on the first single tooth model and the spatial information of the first single tooth model, similar to the training process, the electronic device inputs the first single tooth model and the spatial information of the first single tooth model into the residual crown recognition model, and the first classification network in the residual crown recognition model first extracts features of the first single tooth model to obtain tooth surface features. Exemplarily, the tooth surface features include at least one of the texture features and geometric features of the first single tooth model. Geometric features may include but are not limited to the curvature, depth, normal vector, etc. of a three-dimensional point in the single tooth model. After the tooth surface features are extracted by the first classification network, the second classification network in the residual crown recognition model performs residual crown recognition processing based on the obtained tooth surface features and the input spatial information of the first single tooth model. The residual crown recognition model of this embodiment adopts a two-level network architecture, so that different levels of network structures can focus on processing different tasks, thereby reducing the complexity of the model, reducing the risk of overfitting, and ensuring the stability of the model.

[0090] In some embodiments, the spatial information of the first single tooth model includes the volume of the first single tooth model; in the process of the electronic device performing residual crown recognition processing by the second classification network in the residual crown recognition model based on the tooth surface features and the spatial information of the first single tooth model, the second classification network can first calculate the defect volume of the first single tooth model, and if the defect volume and tooth surface features of the first single tooth model meet at least one of the following conditions, the residual crown recognition result indicating that the first single tooth model is a residual crown is output: the defect volume exceeds half of the volume of the first single tooth model; the texture features belong to the residual crown color range; the geometric features show a residual crown change trend.

[0091] After outputting the residual crown recognition result indicating that the second single tooth model is a residual crown through the above embodiment, the electronic device can further determine the residual crown category of the first single tooth model according to the defect volume, texture features and geometric features.

[0092] In some embodiments, in order to facilitate the doctor to more intuitively show the identified residual crown to the patient, the residual crown identified in the tooth model can also be automatically screenshotted and displayed. The following is an exemplary description of the process of automatically screenshotting and displaying the residual crown identified in the tooth model:

[0093] The electronic device may determine the position of the first single tooth model in the first tooth model when the residual crown recognition result indicates that the first single tooth model is a residual crown. Exemplarily, the electronic device may determine the position of the first single tooth model in the first tooth model by generating a three-dimensional bounding box indicating the first single tooth model and determining the position of the three-dimensional bounding box in the first tooth model. The three-dimensional bounding box may be a cube or a cuboid of the minimum volume surrounding the first single tooth model, which is generated by calculating the boundary points of the single tooth model, and the position of the three-dimensional bounding box may be determined by calculating the relative position of the center point or specific corner point of the three-dimensional bounding box and the entire tooth model.

[0094] Next, the electronic device can determine the projection viewing angle of the first tooth model according to the position of the first single tooth model in the first tooth model, and project the first tooth model from the three-dimensional space to the two-dimensional space according to the projection viewing angle to obtain a tooth image, and the projection viewing angle is used to make the tooth image display the single tooth area corresponding to the first single tooth model, and display the tooth image in the display interface. Among them, the projection viewing angle can be determined by analyzing the position of the first single tooth model in the first tooth model, using preset rules or algorithms, to determine one or more optimal observation viewing angles that can maximize the display of the characteristics of the single tooth area, and then, according to the projection viewing angle, the points on the three-dimensional model are respectively mapped to the two-dimensional space to obtain a tooth image containing the single tooth area corresponding to the first single tooth model, and the tooth image is a two-dimensional screenshot automatically generated by the electronic device containing the identified residual crown.

[0095] Exemplarily, when the electronic device generates a three-dimensional bounding box indicating the first single tooth model, the electronic device can also project the three-dimensional bounding box from the three-dimensional space to the two-dimensional space according to the projection angle to obtain a two-dimensional bounding box; wherein the size of the two-dimensional bounding box is determined based on the size of the three-dimensional bounding box, and the display position of the two-dimensional bounding box in the tooth image is the same as the display position of the single tooth area; and the tooth image with the two-dimensional bounding box superimposed on the single tooth area is displayed in the display interface. Similar to the aforementioned projection process, the three-dimensional bounding box is projected from the three-dimensional space to the two-dimensional space at a determined projection angle to generate a two-dimensional bounding box, so that the residual crown is clearly identified in the tooth image through the two-dimensional bounding box, and the identified residual crown situation is further clearly displayed to the doctor and the patient.

[0096] See for example Figure 7 , showing a schematic diagram of automatically taking screenshots of the residual crowns identified in the tooth model.

[0097] It is not difficult to understand that the schemes described in the above embodiments can be combined when there is no conflict, and they are not listed one by one in the embodiments of the present application.

[0098] In some embodiments, see Figure 8The present application also provides a training device for a residual crown recognition model, including:

[0099] The training sample acquisition module 801 is used to acquire a plurality of training samples, wherein the training samples include the second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model; different training samples correspond to different second single tooth models;

[0100] The model training module 802 is used to input the training samples into the residual crown recognition model to be trained, so that the residual crown recognition model to be trained performs residual crown recognition processing based on the second single tooth model and the spatial information of the second single tooth model, and obtains the residual crown recognition result corresponding to the second single tooth model, and trains the residual crown recognition model to be trained with minimizing the difference between the residual crown recognition result and the label as the optimization goal; wherein the trained residual crown recognition model is applied to the above-mentioned residual crown recognition method.

[0101] In one implementation, the training sample acquisition module 801 is specifically used to: display the second tooth model to be labeled in a display interface; in response to a tooth position number input operation for the second tooth model, obtain at least one tooth position number for indicating a residual crown in the second tooth model; for multiple second single tooth models obtained by segmenting the second tooth model, determine the label of the second single tooth model corresponding to the tooth position number as indicating the residual crown information of the second single tooth model, and determine the labels of the remaining other second single tooth models as indicating the non-residual crown information of the second single tooth model.

[0102] In some embodiments, see Fig. 9 The present application also provides a residual crown identification device, including:

[0103] The model to be identified and the spatial information acquisition module 901 is used to acquire the first single tooth model to be identified and the spatial information of the first single tooth model in the first tooth model;

[0104] The residual crown recognition module 902 is used to use a pre-trained residual crown recognition model to perform residual crown recognition processing on the first single tooth model and the spatial information of the first single tooth model, and obtain a residual crown recognition result corresponding to the first single tooth model; wherein the residual crown recognition model is obtained through supervised training based on a number of training samples, and the training samples include the second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

[0105] In one implementation, the spatial information of the first single tooth model includes at least one of the volume of the first single tooth model and the tooth position number of the first single tooth model in the first tooth model.

[0106] In one implementation, the model to be identified and spatial information acquisition module 901 is specifically used to: obtain the first tooth model to be identified; identify and segment each tooth in the first tooth model to obtain multiple first single tooth models and tooth position numbers corresponding to the multiple first single tooth models; and / or complete the first single tooth model so that the completed first single tooth model forms a closed solid model, and the volume of the completed first single tooth model is determined as the volume of the first single tooth model.

[0107] In one implementation, the residual crown recognition model includes a first classification network and a second classification network; the residual crown recognition module 902 is specifically used to: input the first single tooth model and the spatial information of the first single tooth model into the residual crown recognition model, so that the first classification network in the residual crown recognition model performs feature extraction on the first single tooth model to obtain tooth surface features, and the second classification network in the residual crown recognition model performs residual crown recognition processing based on the tooth surface features and the spatial information of the first single tooth model; wherein the tooth surface features include at least one of the texture features and geometric features of the first single tooth model.

[0108] In one implementation, the spatial information of the first single tooth model includes the volume of the first single tooth model; the residual crown recognition module 902 is specifically used to: calculate the defect volume of the first single tooth model by the second classification network, and if the defect volume and tooth surface features of the first single tooth model meet at least one of the following conditions, then output the residual crown recognition result indicating that the first single tooth model is a residual crown: the defect volume exceeds half of the volume of the first single tooth model; the texture features belong to the residual crown color range; the geometric features show a trend of residual crown change.

[0109] In one implementation, after outputting the residual crown identification result indicating that the first single tooth model is a residual crown, the residual crown identification module 902 is further specifically used to: determine the residual crown category of the first single tooth model according to the defect volume, texture features and geometric features.

[0110] In one implementation, it also includes a screenshot module, which is used to determine the position of the first single tooth model in the first tooth model if the residual crown recognition result indicates that the first single tooth model is a residual crown; determine the projection viewing angle of the first tooth model according to the position of the first single tooth model in the first tooth model; project the first tooth model according to the projection viewing angle to obtain a tooth image, and the projection viewing angle is used to make the tooth image show a single tooth area corresponding to the first single tooth model; and display the tooth image in the display interface.

[0111] In one implementation, the screenshot module is specifically used to: generate a three-dimensional bounding box for indicating a first single tooth model, and determine the position of the three-dimensional bounding box in the first tooth model; project the three-dimensional bounding box according to the projection perspective to obtain a two-dimensional bounding box; wherein the size of the two-dimensional bounding box is determined based on the size of the three-dimensional bounding box, and the display position of the two-dimensional bounding box in the tooth image is the same as the display position of the single tooth area; and display a tooth image with a two-dimensional bounding box superimposed on the single tooth area in the display interface.

[0112] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The device embodiments described above are only illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units.

[0113] In some embodiments, embodiments of the present application further provide an electronic device, including a memory, a processor, and executable instructions stored in the memory and executable on the processor;

[0114] Wherein, the processor implements the steps in the above method when executing the executable instructions.

[0115] Exemplarily, the processor includes but is not limited to a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), or a field-programmable gate array (FPGA).

[0116] Exemplarily, the memory may include at least one type of storage medium, the storage medium including flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc.

[0117] For example, Fig.10 is a schematic structural diagram of a device provided by an exemplary embodiment. Fig.10At the hardware level, the device includes a processor 1002, an internal bus 1004, a network interface 1006, a memory 1008, and a non-volatile memory 1010, and may also include hardware required for other services. One or more embodiments of this specification may be implemented based on software, such as the processor 1002 reading the corresponding computer program from the non-volatile memory 1010 into the memory 1008 and then running it. Of course, in addition to the software implementation, one or more embodiments of this specification do not exclude other implementations, such as logic devices or a combination of software and hardware, etc., that is, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0118] In some embodiments, the embodiments of the present application further provide a computer program product, including a computer program, which is used to implement the above method when executed by a processor.

[0119] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of the device to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0120] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal, enables the terminal to execute the above method.

[0121] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0122] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0123] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for identifying a residual crown, characterized in that: include: Acquire a first single tooth model to be identified and spatial information of the first single tooth model in the first tooth model, wherein the spatial information of the first single tooth model includes at least one of the volume of the first single tooth model and the tooth position number of the first single tooth model in the first tooth model, so as to determine the degree of defect of the first single tooth model based on the volume of the first single tooth model, and / or determine the residual crown identification standard corresponding to the first single tooth model based on the tooth position number of the first single tooth model; wherein the first single tooth model and the tooth position number of the first single tooth model are obtained by identifying and segmenting each tooth in the first tooth model; and / or the volume of the first single tooth model is the volume of the completed first single tooth model obtained by completing the first single tooth model, and the completed first single tooth model forms a closed solid model; The first single tooth model is subjected to feature extraction using a pre-trained residual crown recognition model to obtain tooth surface features, and residual crown recognition processing is performed based on the tooth surface features and the spatial information of the first single tooth model to obtain a residual crown recognition result corresponding to the first single tooth model; the tooth surface features include at least one of texture features and geometric features of the first single tooth model; wherein the residual crown recognition model is obtained through supervised training based on a number of training samples, and the training samples include a second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

2. The method according to claim 1, characterized in that: The residual crown recognition model includes a first classification network and a second classification network; The method of extracting features of the first single tooth model using the pre-trained residual crown recognition model to obtain tooth surface features, and performing residual crown recognition processing based on the tooth surface features and spatial information of the first single tooth model, includes: The first single tooth model and the spatial information of the first single tooth model are input into the residual crown recognition model, so that the first classification network in the residual crown recognition model performs feature extraction on the first single tooth model to obtain tooth surface features, and the second classification network in the residual crown recognition model performs residual crown recognition processing based on the tooth surface features and the spatial information of the first single tooth model.

3. The method according to claim 2, characterized in that The spatial information of the first single tooth model includes the volume of the first single tooth model; The second classification network in the residual crown recognition model performs residual crown recognition processing based on the tooth surface features and the spatial information of the first single tooth model, including: The defect volume of the first single tooth model is calculated by the second classification network, and if the defect volume of the first single tooth model and the tooth surface feature meet at least one of the following conditions, the residual crown recognition result indicating that the first single tooth model is a residual crown is output: The defect volume exceeds half of the volume of the first single tooth model; The texture features belong to the residual crown color range; The geometric features show a trend of residual crown changes.

4. The method according to claim 3, characterized in that After outputting the residual crown recognition result indicating that the first single tooth model is a residual crown, the method further includes: The residual crown category of the first single tooth model is determined according to the defect volume, the texture features and the geometric features.

5. The method according to claim 1, characterized in that: Also includes: If the residual crown recognition result indicates that the first single tooth model is a residual crown, determining a position of the first single tooth model in the first tooth model; determining a projection viewing angle of the first tooth model according to a position of the first single tooth model in the first tooth model; Projecting the first tooth model according to the projection viewing angle to obtain a tooth image, wherein the projection viewing angle is used to make the tooth image display a single tooth area corresponding to the first single tooth model; The tooth image is displayed in a display interface.

6. The method according to claim 5, characterized in that The determining the position of the first single tooth model in the first tooth model comprises: generating a three-dimensional bounding box for indicating the first single tooth model, and determining a position of the three-dimensional bounding box in the first tooth model; The method further comprises: Projecting the three-dimensional bounding box according to the projection viewing angle to obtain a two-dimensional bounding box; wherein the size of the two-dimensional bounding box is determined based on the size of the three-dimensional bounding box, and the display position of the two-dimensional bounding box in the tooth image is the same as the display position of the single tooth area; The displaying of the tooth image in the display interface includes: A tooth image with the two-dimensional bounding box superimposed on the single tooth region is displayed in the display interface.

7. A training method for a residual crown recognition model, characterized in that: include: Acquire a plurality of training samples, wherein the training samples include a second single tooth model and spatial information of the second single tooth model in the second tooth model, and a label for indicating residual crown information of the second single tooth model; Different training samples correspond to different second single tooth models; the spatial information of the second single tooth model includes at least one of the volume of the second single tooth model and the tooth position number of the second single tooth model in the second tooth model, so as to determine the degree of defect of the second single tooth model based on the volume of the second single tooth model, and / or determine the residual crown identification standard corresponding to the second single tooth model based on the tooth position number of the second single tooth model; wherein the second single tooth model and the tooth position number of the second single tooth model are obtained by identifying and segmenting each tooth in the second tooth model; and / or, the volume of the second single tooth model is the volume of the completed second single tooth model obtained by completing the second single tooth model, and the completed second single tooth model forms a closed solid model; The training samples are input into the residual crown recognition model to be trained, so that the residual crown recognition model to be trained performs feature extraction on the second single tooth model to obtain tooth surface features, and performs residual crown recognition processing based on the tooth surface features of the second single tooth model and the spatial information of the second single tooth model to obtain a residual crown recognition result corresponding to the second single tooth model, and trains the residual crown recognition model to be trained with minimizing the difference between the residual crown recognition result and the label as the optimization goal; the tooth surface features include at least one of the texture features and geometric features of the second single tooth model; wherein the trained residual crown recognition model is applied to the residual crown recognition method as described in any one of claims 1 to 6.

8. The method according to claim 7, characterized in that The step of obtaining a label indicating the residual crown information of the second single tooth model includes: Displaying the second tooth model to be annotated in the display interface; In response to a tooth position number input operation for the second tooth model, obtaining at least one tooth position number for indicating a residual crown in the second tooth model; For multiple second single tooth models obtained by segmenting the second tooth model, the label of the second single tooth model corresponding to the tooth position number is determined as indicating the residual crown information of the second single tooth model, and the labels of the remaining other second single tooth models are determined as indicating the non-residual crown information of the second single tooth model.

9. A residual crown identification device, characterized in that: include: A model to be identified and spatial information acquisition module, used for acquiring a first single tooth model to be identified and spatial information of the first single tooth model in the first tooth model, wherein the spatial information of the first single tooth model includes at least one of the volume of the first single tooth model and the tooth position number of the first single tooth model in the first tooth model, so as to determine the degree of defect of the first single tooth model based on the volume of the first single tooth model, and / or determine the residual crown recognition standard corresponding to the first single tooth model based on the tooth position number of the first single tooth model; wherein the first single tooth model and the tooth position number of the first single tooth model are obtained by identifying and segmenting each tooth in the first tooth model; and / or the volume of the first single tooth model is the volume of the completed first single tooth model obtained by completing the first single tooth model, and the completed first single tooth model forms a closed solid model; A residual crown recognition module is used to use a pre-trained residual crown recognition model to perform feature extraction on the first single tooth model to obtain tooth surface features, and perform residual crown recognition processing based on the tooth surface features and the spatial information of the first single tooth model to obtain a residual crown recognition result corresponding to the first single tooth model; the tooth surface features include at least one of the texture features and geometric features of the first single tooth model; wherein the residual crown recognition model is obtained through supervised training based on a number of training samples, and the training samples include a second single tooth model and the spatial information of the second single tooth model in the second tooth model, and a label for indicating the residual crown information of the second single tooth model.

10. An electronic device, characterized in that: including a memory, a processor, and executable instructions stored in the memory and executable on the processor; Wherein, when the processor executes the executable instructions, the steps of the method as claimed in any one of claims 1 to 8 are implemented.

11. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images