Tooth segmentation method, display method, device, equipment and storage medium

Abstract

This application discloses a tooth segmentation method, display method, apparatus, device, and storage medium. The tooth segmentation method includes: obtaining a root bone scan image; inputting the root bone scan image into a localization model to obtain first localization information for N teeth, the first localization information being used to determine the position information of the teeth in the root bone scan image; identifying the center information of different teeth in the root bone scan image to obtain second localization information for M teeth; N and M are positive integers greater than 1; determining the region of interest for tooth segmentation based on the first localization information of the N teeth and the second localization information of the M teeth; and inputting the image corresponding to the segmented region of interest into the segmentation model to obtain the tooth segmentation result. The tooth segmentation method provided in this application balances comprehensiveness and accuracy in segmentation.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a tooth segmentation method, display method, device, equipment and storage medium. Background Technology

[0002] With societal development and rising demands for quality of life, oral health maintenance programs, including oral hygiene and aesthetics, have gradually become a focus of attention. In the dental field, particularly in orthodontics and implantology, dental information such as 3D models of teeth serves as crucial intermediate data, showcasing the current state of a user's teeth and assisting practitioners in making more accurate diagnoses and treatment plans.

[0003] Distinguishing between different teeth is called tooth segmentation. In practice, the oral cavity of the test subject may have teeth with inconsistent arrangement, and there may be supernumerary teeth, impacted teeth, and other teeth that are difficult to observe directly. This makes conventional tooth segmentation algorithms, such as thresholding, clustering, and level set methods based on image grayscale information, and segmentation methods based on deep neural networks, difficult to handle. In particular, methods based on image grayscale information often require manual interaction, which is inefficient, and this method is difficult to handle complex situations such as blurred tooth roots and uneven grayscale distribution, resulting in poor segmentation results. Methods based on deep neural networks apply a deep model to the entire slice image or 3D image. Due to the reliance on a single tooth detection and segmentation model and the lack of a large amount of training data, it is difficult to achieve high segmentation accuracy and high tooth detection rate. It is evident that existing technologies have low accuracy when dealing with tooth segmentation under complex tooth root and jawbone conditions. Summary of the Invention

[0004] One of the purposes of this application is to provide a tooth segmentation method to solve the technical problems of incomplete, unrealistic, and inaccurate tooth segmentation in the prior art, which makes it unsuitable for auxiliary treatment.

[0005] One of the purposes of this application is to provide a display method.

[0006] One of the purposes of this application is to provide a tooth-splitting device.

[0007] One of the purposes of this application is to provide an electronic device.

[0008] One of the purposes of this application is to provide a storage medium.

[0009] To achieve one of the above objectives, one embodiment of this application provides a tooth segmentation method, comprising: obtaining a root bone scan image; inputting the root bone scan image into a localization model to obtain first localization information for N teeth, wherein the first localization information is used to determine the position information of the teeth in the root bone scan image; identifying the center information of different teeth in the root bone scan image to obtain second localization information for M teeth; wherein N and M are positive integers greater than 1; determining the region of interest for tooth segmentation based on the first localization information of N teeth and the second localization information of M teeth; and inputting the image corresponding to the segmented region of interest into the segmentation model to obtain the tooth segmentation result.

[0010] Optionally, based on the root bone scan image, the boundary information of the tooth is determined, and based on the shape and boundary information of the tooth, the boundary distance information is determined. The boundary distance information is used to characterize the distance between the pixel point within the tooth range in the root bone scan image and the boundary point of the tooth, and / or the distance between the pixel point within the tooth range in the root bone scan image and the center of the tooth. Based on the boundary distance information, the image corresponding to the region of interest of the first tooth segmentation is determined in the root bone scan image.

[0011] Optionally, the first tooth includes a number of teeth that differ from N teeth to M teeth, where N is less than M.

[0012] Optionally, based on the first localization information, N regions of interest for tooth segmentation are determined; the images corresponding to the N regions of interest for tooth segmentation and the images corresponding to the regions of interest for differentially segmented teeth are input into the segmentation model to obtain the tooth segmentation result; or, the images corresponding to the N regions of interest for tooth segmentation are input into the segmentation model to obtain the segmentation result for the N teeth, and the images corresponding to the regions of interest for differentially segmented teeth are input into the segmentation model to obtain the segmentation result for the differentially segmented teeth; based on the segmentation results for the N teeth and the segmentation results for the differentially segmented teeth, the final tooth segmentation result is obtained.

[0013] Optionally, based on the first localization information, N regions of interest for tooth segmentation are determined; the images corresponding to the N regions of interest for tooth segmentation are input into the segmentation model to obtain the segmentation results of N teeth; three-dimensional reconstruction is performed based on the second localization information to obtain the center reconstruction results of M teeth; the segmentation results of N teeth and the center reconstruction results of M teeth are compared to determine the teeth with differences.

[0014] Optionally, the first positioning information includes information about the bounding boxes surrounding the N teeth; based on the bounding boxes of the N teeth, the root bone scan image is cropped to obtain images corresponding to the regions of interest segmented from the N teeth; the images corresponding to the regions of interest segmented from the N teeth are input into the segmentation model to obtain the segmentation results of the N teeth.

[0015] Optionally, the second positioning information includes a center mask image, which is used to characterize the center positions of the M teeth.

[0016] Optionally, based on the root bone scan image, the boundary information of the tooth is determined; based on the shape and boundary information of the tooth, a boundary distance image is determined, wherein the boundary distance information is used to characterize the distance between the pixel within the tooth range in the root bone scan image and the boundary point of the tooth, and / or the distance between the pixel within the tooth range in the root bone scan image and the center of the tooth; based on the boundary distance information of the tooth, a center mask image of the tooth is determined, wherein pixels with boundary distance information less than a preset threshold have a first gray value, and pixels with boundary distance information greater than a preset threshold have a second gray value.

[0017] Optionally, the calcaneal scan images may include cone-beam computed tomography (CBCT) images.

[0018] To achieve one of the objectives, one embodiment of this application provides a display method, comprising at least one of the following: presenting a tooth segmentation result in a graphical user interface; presenting a root bone scan image in a graphical user interface, the root bone scan image being superimposed with the tooth segmentation result; presenting a three-dimensional model of the segmented tooth in a graphical user interface, the three-dimensional model being generated by three-dimensional reconstruction based on the tooth segmentation result; the tooth segmentation result is obtained by inputting an image corresponding to a region of interest in the segmented tooth into a segmentation model, the region of interest in the segmented tooth being determined based on first positioning information of N teeth and second positioning information of M teeth, the first positioning information being obtained by inputting the root bone scan image into a positioning model, the second positioning information being obtained by identifying the center information of different teeth in the root bone scan image, the first positioning information being used to determine the position information of the teeth in the root bone scan image, and N and M being positive integers greater than 1.

[0019] Optionally, a bounding box surrounding each tooth is presented in the graphical user interface. The bounding box is either a three-dimensional bounding box or a two-dimensional bounding box, and the bounding box is determined by a neural network based on the location of different teeth according to the root bone scan image.

[0020] To achieve one of the objectives, one embodiment of this application provides a tooth segmentation device, comprising: a first module for obtaining a root bone scan image; a second module for processing the root bone scan image with a positioning model to obtain first positioning information for N teeth, wherein the first positioning information is used to determine the position information of the teeth in the root bone scan image, and N is a positive integer greater than 1; a third module for identifying the center information of different teeth in the root bone scan image to obtain second positioning information for M teeth, wherein M is a positive integer greater than 1; a fourth module for determining the region of interest for tooth segmentation based on the first positioning information of the N teeth and the second positioning information of the M teeth; and a fifth module for processing the image corresponding to the region of interest for tooth segmentation with a segmentation model to obtain a tooth segmentation result.

[0021] Optionally, the tooth segmentation device is connected to a first device, which is used to perform cone-beam annular radiography to obtain the root bone scan image.

[0022] To achieve one of the objectives, one embodiment of this application provides an electronic device, including a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory is used to store application programs; and the processor is used to implement the steps of any tooth segmentation method or the steps of any display method when executing the application programs stored in the memory.

[0023] To achieve one of the objectives, one embodiment of this application provides a storage medium storing an application program that, when executed, implements the steps of any tooth segmentation method or the steps of any display method.

[0024] Compared with existing technologies, the tooth segmentation method provided in this application determines the region of interest (ROI) for tooth segmentation based on two types of tooth positioning information, thereby improving the accuracy of tooth positioning and detection. When using the obtained ROI for tooth segmentation, it can achieve complete and accurate tooth segmentation, avoiding omissions of special cases such as impacted teeth and supernumerary teeth. One type of positioning information is used to determine the position of the tooth in the root bone scan image, enabling the tooth segmentation result to have the accuracy of single-tooth segmentation for conventional teeth; the other type of positioning information is obtained by identifying the center information of the root bone scan image, ensuring that the tooth segmentation result includes all teeth of the subject without omission. Attached Figure Description

[0025] Figure 1 This is a three-dimensional model of a tooth in one embodiment of this application.

[0026] Figure 2 This is a schematic diagram of the tooth-splitting device in one embodiment of this application.

[0027] Figure 3 This is a schematic diagram of the structure of an electronic device according to one embodiment of this application.

[0028] Figure 4 This is a schematic diagram of the steps of a tooth segmentation method in one embodiment of this application.

[0029] Figure 5 This is a schematic diagram illustrating the process of obtaining the segmentation result of N teeth in one embodiment of this application.

[0030] Figure 6 This is a schematic diagram of the process of obtaining the center reconstruction results of M teeth in one embodiment of this application.

[0031] Figure 7 This is a schematic diagram of a process for determining differentially shaped teeth according to an embodiment of this application.

[0032] Figure 8 This is a schematic diagram of a process for generating differential teeth according to one embodiment of this application.

[0033] Figure 9 This is a schematic diagram showing the steps of a method in one embodiment of this application. Detailed Implementation

[0034] The present application will now be described in detail with reference to the specific embodiments shown in the accompanying drawings. However, these embodiments do not limit the present application, and any structural, methodological, or functional modifications made by those skilled in the art based on these embodiments are included within the scope of protection of this application.

[0035] It should be noted that the term "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0036] The terms "first," "second," and "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. There is no necessary correlation between the terms "first," etc.; the inclusion of a feature related to "second" in one embodiment does not necessarily mean that the embodiment must include a feature related to "first."

[0037] The main idea of ​​this application is as follows: When segmenting and determining the morphology of each tooth, whether segmenting a 2D image or a 3D model, the teeth are usually located first based on the conventional tooth arrangement before their morphology is determined. However, when encountering unconventional situations such as supernumerary teeth or impacted teeth, it is impossible to locate these special teeth, resulting in segmentation results that cannot balance completeness and accuracy, and do not accurately reflect reality. The tooth segmentation method provided in this application, based on the positional information of the teeth in the root bone scan image, further references the center information of the teeth in the root bone scan image, which can improve the accuracy of tooth detection and segmentation while ensuring the completeness and accuracy of segmentation. The reference to the root bone scan image is not simply using both as input for calculation, but rather obtaining another set of localizations for the same dentition based on the center information of the teeth in the root bone scan image, and then using the two sets of localization information to cross-verify and reference each other, thereby identifying unconventional and missed teeth, and improving the accuracy of tooth detection.

[0038] The calcaneal scan image can be a CBCT (cone beam computed tomography) image.

[0039] CBCT images use a cone-beam X-ray to scan a target volume area, obtaining three-dimensional data through a single rotation. CBCT images can display the structure of teeth, bones, and soft tissues, offering high resolution and making them suitable for imaging small anatomical areas. CBCT images offer advantages such as high accuracy, ease of assessment of tooth roots and bone, convenient location of impacted teeth, and convenient temporomandibular joint assessment.

[0040] CBCT images can include multiple two-dimensional images corresponding to multiple slices. Slices can correspond to the coronal plane (from anterior to posterior), sagittal plane (from left to right), or cross-section plane (from top to bottom).

[0041] Tooth segmentation refers to the process of segmenting teeth into instances within an image, classifying each pixel into either a tooth category or the background. The segmentation results can assist operators in diagnosis and treatment planning. These results can be presented as a 3D model.

[0042] Segmenting teeth does not necessarily lead to physical distinctions between each tooth. The physical positional relationships of the segmented teeth can be adjusted, such as by spacing them apart; the segmented teeth can also be distinguished using labels, colors, border lines, arrow markings, etc. For example, Figure 1 The image shows the segmented teeth displayed using color.

[0043] Figure 1 Image (a) shows a three-dimensional model corresponding to a tooth. Figure 1 Figure (b) shows another three-dimensional model corresponding to the teeth. Comparing the two three-dimensional models, it can be seen that... Figure 1 The 3D model shown in (a) is missing a tooth selected by the dashed rectangular frame. This tooth could be a supernumerary tooth or an impacted tooth. By implementing the technical solution provided in this application, this tooth can be identified as the differential tooth described below, thereby enabling segmentation along with other teeth.

[0044] Dividing teeth into segments helps in setting different orthodontic forces for different teeth during orthodontic treatment. For example, smaller forces are needed for corresponding incisors to avoid root resorption, while larger forces are needed for corresponding molars to ensure their movement.

[0045] Distinguishing between impacted teeth, supernumerary teeth, and other special teeth helps in identifying the causes of adjacent root lesions or abnormal interdental spaces during orthodontic treatment, and assists the operator in deciding whether to add special tooth treatment steps (such as extracting special teeth) to the treatment plan.

[0046] One embodiment of this application provides a storage medium.

[0047] Storage media can be installed in a computer and store applications. Storage media can be any available medium that the computer can access data from, or it can be a storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media such as floppy disks, hard disks, and magnetic tapes; optical media such as DVDs (Digital Video Discs); or semiconductor media such as SSDs (Solid State Disks).

[0048] In one embodiment, when the application is executed, it implements the steps of a tooth segmentation method. In a specific embodiment, the tooth segmentation method includes at least one of the following steps:

[0049] Obtain radicular bone scan images;

[0050] The root bone scan image is input into the localization model to obtain the first localization information of N teeth. The first localization information is used to determine the position information of the teeth in the root bone scan image.

[0051] The center information of different teeth in the root bone scan image is identified to obtain the second localization information of M teeth; N and M are positive integers greater than 1.

[0052] Based on the first localization information of N teeth and the second localization information of M teeth, the region of interest for tooth segmentation is determined;

[0053] The image corresponding to the region of interest in the segmented teeth is input into the segmentation model to obtain the tooth segmentation result.

[0054] The stored content of the storage medium can also be configured based on the tooth segmentation method in any of the technical solutions provided below.

[0055] In one embodiment, when the application executes, it implements a display method. In a specific embodiment, the display method includes at least one of the following steps:

[0056] Present the tooth segmentation results in a graphical user interface;

[0057] The root bone scan image is presented in a graphical user interface, and the tooth segmentation result is superimposed on the root bone scan image.

[0058] A 3D model of the teeth after segmentation is presented in a graphical user interface. The 3D model is generated by 3D reconstruction based on the tooth segmentation results.

[0059] The tooth segmentation result is obtained by inputting the image corresponding to the segmented region of interest into the segmentation model. The region of interest of the segmented tooth is determined based on the first localization information of N teeth and the second localization information of M teeth. The first localization information is obtained by inputting the root bone scan image into the localization model, and the second localization information is obtained by identifying the center information of different teeth in the root bone scan image. The first localization information is used to determine the position information of the teeth in the root bone scan image, where N and M are positive integers greater than 1.

[0060] The content stored in the storage medium can also be configured based on the display method in any of the technical solutions provided below.

[0061] like Figure 2 As shown, one embodiment of this application provides a tooth-splitting device 100.

[0062] The tooth dividing device 100 includes at least one of the following components:

[0063] The first module 11 is used to obtain pedicle scan images;

[0064] The second module 12 is used to input the root bone scan image into the positioning model to obtain the first positioning information of N teeth;

[0065] The third module 13 is used to identify the center information of different teeth in the root bone scan image and obtain the second positioning information of M teeth, where M is a positive integer greater than 1;

[0066] The fourth module 14 is used to determine the region of interest for tooth segmentation based on the first localization information of N teeth and the second localization information of M teeth.

[0067] Module 5, 15, is used to process the image corresponding to the region of interest in tooth segmentation using a segmentation model to obtain the tooth segmentation result.

[0068] The first positioning information is used to determine the position of the tooth in the root bone scan image.

[0069] The localization model can be implemented using at least one neural network. The at least one neural network is pre-trained to determine the positional information of the tooth in the root bone scan image, and / or to determine first localization information. The second module 12 may carry the localization model.

[0070] The second positioning information may include the center information of different teeth in the root bone scan image. The center information may be the information of the two-dimensional center point, the three-dimensional center point, the long axis, or other anatomical features of the tooth. The information of the two-dimensional center point may be the information of the geometric center point, centroid point, or centroid point of the tooth in the two-dimensional image of each cross section, or it may be a set of multiple center points in multiple cross sections, or information of the three-dimensional structure fitted based on multiple center points. The third module 13 can use morphological features such as boundary distance to identify the center information of the tooth, specifically by using a boundary distance regression model or other neural network models to achieve identification. In other embodiments, the third module 13 may also use edge detection and contour extraction, minimum bounding rectangle and centroid calculation, ellipse fitting, template matching, etc.

[0071] Module 14 determines the region of interest (ROI) for tooth segmentation by comparing the second and first localization information. This ROI, used to guide tooth segmentation, can be all teeth in the root bone scan image or a subset of teeth. Specifically, a subset of teeth can be differentially expressed teeth included in one of the first and second localization information but excluded from the other, or overlapping teeth included in both localization information sets but with different data in the two sets. Differentially expressed teeth can be supernumerary teeth, impacted teeth, or omitted teeth.

[0072] The tooth segmentation result is determined based on the region of interest (ROI) of the segmented teeth, which can be all teeth or a portion of the teeth. The tooth segmentation result can be in the form of a two-dimensional image, a three-dimensional image, or a three-dimensional model. When the tooth segmentation result is a three-dimensional model, the fifth module 1 can also be used for three-dimensional reconstruction based on the ROI of the segmented teeth; similarly, the second module 11 can also be used for three-dimensional reconstruction based on the first localization information.

[0073] The first module 11 can obtain pedicle scan images acquired by external devices, or it can perform acquisition operations itself to obtain pedicle scan images.

[0074] In one embodiment, the tooth splitting device 100 may also be connected to the first device 101.

[0075] The first device 101 can be disposed outside the tooth segmentation device 100, serving as an external device of the tooth segmentation device 100. The first device 101 and the device 100 can also be integrated together.

[0076] The first device 101 is used to perform scanning to obtain radicular bone scanning information. The radicular bone scanning information can be the radicular bone scanning image or data information related to the radicular bone scanning image. The first device 101 can scan the jawbone area or perform an oral cavity scan.

[0077] The first device 101 can be used to implement a cone-beam ring hood to obtain calcaneal scan images.

[0078] The calcaneal scan image can be used to determine initial localization information. For example, the calcaneal scan image can be processed using one or more convolutional neural networks to determine the initial localization information.

[0079] Peripheral scan images can be used to determine secondary localization information. For example, one or more convolutional neural networks can be used to process peripheral scan images to determine secondary localization information.

[0080] Root bone scan images can be used to determine 3D models. For example, one or more convolutional neural networks can be used to process root bone scan images to obtain first localization information, and then tooth segmentation and 3D reconstruction can be performed based on the first localization information to generate a 3D model. Alternatively, tooth segmentation and 3D reconstruction can be performed based on both the first localization information and information about the first tooth to generate a 3D model. The information used to generate the 3D model can be in the form of a binary masked image.

[0081] The tooth segmentation device 100 can also be configured based on the tooth segmentation method in any of the technical solutions provided below. Specifically, depending on the relationship between the steps, related steps can be implemented in the same or different modules.

[0082] Electrical or communication connections can be established between the first module 11, the second module 12, the third module 13, the fourth module 14, and the fifth module 15 to enable data transmission.

[0083] One embodiment of this application may also provide a display device.

[0084] The display device includes at least one of the following:

[0085] The sixth module is used to present the tooth segmentation results in the graphical user interface;

[0086] The seventh module is used to present the root bone scan image in a graphical user interface, wherein the root bone scan image is superimposed with the tooth segmentation result;

[0087] The eighth module is used to present a three-dimensional model of the teeth after segmentation in a graphical user interface. The three-dimensional model is generated by three-dimensional reconstruction based on the teeth segmentation results.

[0088] The tooth segmentation result is obtained by inputting the image corresponding to the region of interest of the segmented tooth into the segmentation model.

[0089] The region of interest for tooth segmentation is determined based on the first localization information of N teeth and the second localization information of M teeth. N and M are positive integers greater than 1.

[0090] The first positioning information is obtained by inputting the root bone scan image into the positioning model. The first positioning information is used to determine the position of the tooth in the root bone scan image.

[0091] The second positioning information is obtained by identifying the center information of different teeth in the root bone scan image.

[0092] The display device can also be configured based on the display method of any of the technical solutions provided below. Specifically, depending on the relationship between the steps, related steps can be implemented in the same or different modules.

[0093] Electrical or communication connections can be established between the sixth, seventh, and eighth modules to enable data transmission.

[0094] The device, or its modules or units, may be implemented by a computer chip or physical entity, or by a product with corresponding functions. While the device is described in terms of multiple modules, in some embodiments, the functions of the modules may be implemented in one or more software or hardware components.

[0095] One embodiment of this application provides an electronic device 200, such as... Figure 3 As shown.

[0096] The electronic device 200 may be a computer, mobile phone, tablet computer, etc., and this application does not limit the specific type of the electronic device 200.

[0097] Electronic device 200 includes at least one processor 21, at least one memory 22 and communication bus 33.

[0098] At least one processor 21 and at least one memory 22 communicate with each other via a communication bus 23.

[0099] Memory 22 is used to store application programs.

[0100] In one embodiment, the processor 21 is configured to implement the steps of a tooth segmentation method when executing an application stored in the memory 22.

[0101] In one specific embodiment, the tooth segmentation method includes at least one of the following steps:

[0102] Obtain radicular bone scan images;

[0103] The root bone scan image is input into the localization model to obtain the first localization information of N teeth. The first localization information is used to determine the position information of the teeth in the root bone scan image.

[0104] The center information of different teeth in the root bone scan image is identified to obtain the second localization information of M teeth; N and M are positive integers greater than 1.

[0105] Based on the first localization information of N teeth and the second localization information of M teeth, the region of interest for tooth segmentation is determined;

[0106] The image corresponding to the region of interest in the segmented teeth is input into the segmentation model to obtain the tooth segmentation result.

[0107] The steps of the tooth segmentation method can also be configured based on any of the technical solutions provided below.

[0108] In one embodiment, the processor 21 is configured to implement a display method when executing an application stored in the memory 22.

[0109] In one specific embodiment, the display method includes at least one of the following steps:

[0110] Present the tooth segmentation results in a graphical user interface;

[0111] The root bone scan image is presented in a graphical user interface, and the tooth segmentation result is superimposed on the root bone scan image.

[0112] A 3D model of the teeth after segmentation is presented in a graphical user interface. The 3D model is generated by 3D reconstruction based on the tooth segmentation results.

[0113] The tooth segmentation result is obtained by inputting the image corresponding to the region of interest of the segmented tooth into the segmentation model.

[0114] The region of interest for tooth segmentation is determined based on the first localization information of N teeth and the second localization information of M teeth. N and M are positive integers greater than 1.

[0115] The first positioning information is obtained by inputting the root bone scan image into the positioning model.

[0116] The second positioning information is obtained by identifying the center information of different teeth in the root bone scan image.

[0117] The first positioning information is used to determine the position of the tooth in the root bone scan image.

[0118] The steps of the display method can also be configured based on any of the technical solutions provided below.

[0119] In one embodiment, the communication bus 23 may include any number of buses and bridge circuits. In some embodiments, in addition to connecting the processor and memory, the communication bus may also be used to connect peripheral devices or other peripheral circuits.

[0120] In one embodiment, the electronic device 200 may include a user interface 24 and at least one network interface 25. The user interface 23 may include a display, keyboard, mouse, trackball, click wheel, buttons, a touchpad, or a touch screen, etc.

[0121] like Figure 4 As shown, one embodiment of this application provides a tooth segmentation method.

[0122] The application corresponding to this method can be mounted in electronic devices, tooth segmentation devices and / or storage media, can be mounted in the carrier of the display method provided later, or can be mounted in a display device to achieve the corresponding technical effect.

[0123] A tooth segmentation method may include at least one of the following steps.

[0124] Step S11: Obtain a radicular bone scan image.

[0125] A root bone scan image can be an image used to characterize bones and / or tooth roots, which can be obtained by scanning the subject.

[0126] Periatomy scans can be used to determine jawbone features. Periatomy scans can be images used to characterize the jawbone. Periatomy scans can also be used to determine tooth root features. Periatomy scans can also be images used to characterize tooth roots.

[0127] A calcaneal bone scan image can be a tomographic image. A tomographic image can be a two-dimensional image or a combination of two-dimensional images. Specifically, CT (Computed Tomography) technology can be used to scan the subject layer by layer using X-rays, obtaining multiple tomographic images corresponding to different locations. The tomographic sections can correspond to the coronal plane (from front to back), sagittal plane (from left to right), or cross-section plane (from top to bottom).

[0128] Root bone scan images can be a sequence of two-dimensional tomographic images of the dental area.

[0129] When the slice is a transverse section, a calcaneal scan image can be obtained by scanning layer by layer along the vertical direction. In this case, the calcaneal scan image can show the tissue structure at a specific transverse section in the vertical direction. The pixel or grayscale values ​​of hard tissues such as bones and teeth in the calcaneal scan image can be higher than those of soft tissues.

[0130] The calcaneal scan image can be in the form of a two-dimensional image sequence or a two-dimensional image set. The image sequence or image set can include multiple images, each of which corresponds to a tomographic scan image at a different position in the scanning direction.

[0131] Root bone scan images can be a sequence of two-dimensional tomographic images of the dental area.

[0132] Periatomy images include cone-beam computed tomography (CBCT) images. Periatomy images can be composed of cone-beam computed tomography (CBCT) images. CBCT images are obtained through cone-beam X-ray scanning and are particularly suitable for examining hard tissue structures such as the jawbone and tooth roots.

[0133] CBCT images can provide three-dimensional image information of the teeth and soft tissues of the tested object. The three-dimensional model of the teeth provides important auxiliary information, which can help doctors diagnose and formulate treatment plans.

[0134] When the root bone scan image is a CBCT image, the tooth segmentation method provided in this application can also overcome the complex situation of uneven gray-level distribution, artifacts and low signal-to-noise ratio in CBCT images.

[0135] The calcaneal scan images in this application can also be normalized CBCT images.

[0136] S12, input the root bone scan image into the positioning model to obtain the first positioning information of N teeth.

[0137] The localization model can be pre-trained to determine the positional information of the tooth in the root bone scan image, and / or determine the first localization information.

[0138] N teeth represent the teeth that the localization model can locate based on the root bone scan image. The actual number of teeth included in the root bone scan image can be greater than or equal to N. N is a positive integer greater than 1.

[0139] The standard number and distribution of teeth can be set according to the characteristics of the subject. For example, when the subject is in the deciduous dentition stage (child), there are usually 20 teeth, 5 on each side of the upper and lower jaws. The teeth on each side, from front to back, are: central incisor, lateral incisor, canine, first molar, and second molar. For example, when the subject is in the permanent dentition stage (adult), there are usually 28 to 32 teeth, 16 on each side of the upper and lower jaws, symmetrically distributed. The teeth on each side, from front to back, are: central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar, and third molar (wisdom tooth).

[0140] The localization model may include at least one neural network. The neural network may be a deep neural network.

[0141] The first neural network localization model can be used to locate teeth one by one in a root bone scan image, determining the number and distribution of teeth. The localization information determined by the localization model can be used to determine the region of interest (ROI) of each tooth, and the image corresponding to the ROI can be cropped from the root bone scan image using a bounding box surrounding the tooth.

[0142] The localization model can be or includes a deep convolutional neural network. For example, it can be an object detection network, such as Faster-RCNN, YOLO, or RetinaNet (at least one of these). The second neural network can be a fully convolutional deep neural network, such as a medical image segmentation network, such as UNet, Atten-UNet, or UNet++ (at least one of these).

[0143] The first positioning information can be used to generate a three-dimensional model. In this case, the tooth segmentation method provided in this application may include the steps of: segmenting N teeth according to the first positioning information, performing surface reconstruction using the Marching Cubes algorithm, and obtaining a three-dimensional model corresponding to the N teeth.

[0144] The first location information and the second location information (described later) both point to at least the same or nearby test site on the same subject. This ensures that missed tooth information can be identified based on both.

[0145] The initial localization information can be used to determine the regions of interest (ROIs) for N teeth in the root bone scan image. Based on this initial localization information, images corresponding to the segmented ROIs of the N teeth can be cropped from the root bone scan image. These images can then be used for 3D reconstruction to obtain 3D models of the corresponding N teeth.

[0146] like Figure 5 The regions of interest (ROIs) of different teeth can be determined in the root bone scan image set0 based on tooth localization using a localization model. The root bone scan image set0 can be a three-dimensional image. The ROIs of N teeth can be determined based on first localization information, or the first localization information can characterize the ROIs of N teeth. From the root bone scan image set0, the corresponding image set111 can be determined based on the ROIs of the teeth. Each tooth has a set of images set111 corresponding to its ROIs. The images corresponding to the ROIs of the N teeth of the test subject, determined by the localization model, constitute the image set set11 corresponding to the ROIs. The first localization information loc11 can include information about the bounding boxes surrounding the N teeth, such as... Figure 5 As shown.

[0147] In one specific embodiment, the tooth segmentation method provided in this application may include the following steps: cropping the root bone scan image according to the bounding boxes of N teeth to obtain images corresponding to the regions of interest of N teeth segmentation.

[0148] This step can be included in the step: determining the region of interest for tooth segmentation based on the first localization information of N teeth and the second localization information of M teeth; when this step is implemented, the above steps can be specifically implemented.

[0149] In this specific embodiment, the localization model can be pre-trained to take a root bone scan image as input and the bounding box information surrounding each tooth as output. The bounding box information surrounding each tooth can be coordinate information; for example, it can be the coordinate information of the vertices of the bounding box, where the vertices can be diagonal vertices or all vertices of the bounding box; it can also be the coordinate information of the center point of the bounding box and the size information of the bounding box. The bounding box can be a two-dimensional bounding box or a three-dimensional bounding box.

[0150] The bounding box information can be either the first bounding box information for the corresponding tooth or the information of a second bounding box that has been expanded. The first bounding box can be the smallest rectangular bounding box surrounding a single tooth, and it can be three-dimensional or two-dimensional. The second bounding box can be a larger rectangular bounding box formed by expanding the smallest rectangular bounding box, and it can also be three-dimensional or two-dimensional.

[0151] For example, the minimum bounding box includes diagonal vertices: a first vertex v1 (which could be the top-left front vertex of the 3D rectangle) and a second vertex v2 (which could be the bottom-right back vertex of the 3D rectangle). The coordinates of the first vertex v1 are (x1, y1, z1), and the coordinates of the second vertex v2 are (x2, y2, z2). Based on this, the coordinates of the two vertices can be adjusted to expand the first bounding box into a second bounding box. The coordinates of the new first vertex v1' are (x1-margin_x, y1-margin_y, z1-margin_z), and the coordinates of the new second vertex v2' are (x2+margin_x, y2+margin_y, z2+margin_z).

[0152] The reason for adjusting the size of the bounding box to expand it is to include part of the area around a single tooth (background), which helps improve the recognition accuracy of the tooth (foreground).

[0153] The magnitude of the expansion (i.e., the aforementioned margin_x, margin_y, and margin_z) can be preset. In one embodiment, the preset expansion value is set as follows: the magnitude by which the first bounding box is expanded is a preset multiple of the corresponding side length within the first bounding box. Specifically, the preset value is 0.2.

[0154] For example, when adjusting the size of the x-coordinate direction, the expansion is equal to 0.2 times the side length of the first bounding box in the x-coordinate direction (e.g., |x2-x1|). The same applies to other coordinate directions.

[0155] For example, the default extended value can be set as follows:

[0156] margin_x = 0.2 × (x2 - x1);

[0157] margin_y = 0.2 × (y2 - y1);

[0158] margin_z = 0.2 × (z2 - z1).

[0159] The localization model can be pre-trained to output either the second bounding box or the first bounding box. For the latter, an additional step of expanding the first bounding box into the second bounding box can be performed.

[0160] Once the second bounding box is determined, it can characterize the regions of different teeth formed after the boundary expansion. Based on the second bounding box, N regions of interest can be segmented in the root bone scan image, and then the root bone scan image can be cropped to obtain the image corresponding to the region of interest.

[0161] The image corresponding to the region of interest can be a two-dimensional image sequence or a three-dimensional image.

[0162] If the region of interest (ROI) of N teeth is taken as at least one of the objects of tooth segmentation in this application, then the N teeth can be segmented after the ROI is determined. Specifically, a segmentation model can be used for segmentation.

[0163] The segmentation model may include at least one neural network. The neural network may be a deep neural network.

[0164] The segmentation model can segment the image based on the region of interest of the tooth to determine the morphological features of the tooth (including the crown and / or root).

[0165] The segmentation model can segment N teeth separately, and then combine the segmentation results of the N teeth to obtain a segmentation result of N teeth containing N tooth morphological features and positional distribution.

[0166] In one specific embodiment, the tooth segmentation method provided in this application may include the following steps: inputting the images corresponding to the regions of interest of N teeth into the segmentation model to obtain the segmentation results of N teeth.

[0167] If the segmentation results of N teeth are used as the final tooth segmentation result obtained by the tooth segmentation method, then this step can be: input the images corresponding to the regions of interest of the N teeth into the segmentation model to obtain the tooth segmentation results.

[0168] This step can be included in the step: inputting the image corresponding to the region of interest of the segmented teeth into the segmentation model to obtain the tooth segmentation result; when this step is implemented, the above steps can be specifically implemented.

[0169] The image set111 corresponding to the region of interest for a single tooth segmentation is input into the segmentation model for tooth segmentation, resulting in the corresponding tooth segmentation result info111. The tooth segmentation result can be a 3D image or a 2D image sequence. The tooth segmentation results info111 corresponding to N teeth of the test subject are combined to form the segmentation result info111 for N teeth.

[0170] The image corresponding to the region of interest input to the segmentation model can be a normalized image corresponding to the region of interest of each of the N teeth. For normalization, the pixel value range of the root bone scan image can be normalized to [-1,1] or [0,1] before cropping, or the pixel range of the cropped local region of interest image can be normalized to [-1,1] or [0,1] before being input to the segmentation model.

[0171] The output of the segmentation model can be a two-dimensional image, a two-dimensional image sequence, or a three-dimensional image, specifically a mask image. The mask image can be a binary image, where 1 represents the foreground and 0 represents the background.

[0172] The image corresponding to the region of interest can be a two-dimensional image. Based on this, each image in the image sequence corresponding to the region of interest can be segmented into teeth, resulting in multiple two-dimensional output images corresponding to that tooth. These multiple two-dimensional output images can then be combined in the same order as the image sequence to obtain a sequence of two-dimensional output images representing the segmentation result of that tooth. Combining N sets of two-dimensional output image sequences corresponding to N teeth yields the segmentation result for N teeth.

[0173] At this point, the segmentation model can include a two-dimensional deep fully convolutional neural network.

[0174] The image corresponding to the region of interest can be a 3D image. Based on this, tooth segmentation can be performed on the image corresponding to the region of interest to obtain a 3D output image corresponding to the segmentation result of that tooth. The N 3D output images corresponding to N teeth are combined to obtain the segmentation result of N teeth.

[0175] In this case, the segmentation model can be a three-dimensional deep fully convolutional neural network.

[0176] Step S13: Identify the center information of different teeth in the root bone scan image to obtain the second positioning information of M teeth.

[0177] M teeth represent the teeth whose corresponding positioning information can be obtained by identifying their center information. The actual number of teeth included in a root bone scan image can be equal to M. M is a positive integer greater than 1.

[0178] The second positioning information may include the center information of different teeth in the root bone scan image. The center information can be obtained by first determining the center of the tooth in each root bone scan image, and then fitting a three-dimensional structure based on multiple centers corresponding to the same tooth; or it can be obtained by directly using the center information of the corresponding tooth in the root bone scan image as the center information.

[0179] The center information of a tooth can be determined based on the geometric center or anatomical features of the tooth in the root bone scan image. It is used to indicate the core position of the tooth in the root bone scan image, which facilitates the localization, classification or comparison of teeth.

[0180] The center information can be information about the center point. The center point of a tooth can be its geometric center, centroid, or centroid, and can be determined by calculating the volume or two-dimensional boundary. The center point of a tooth can also be an anatomical feature point such as the cusp or the midpoint of the cervical line. The center point of a tooth can also be calculated and determined by annotating the tooth boundary through three-dimensional reconstruction.

[0181] Obtaining second positioning information determined by identifying the center information of teeth breaks away from the limitations of conventional tooth number or morphology, so that the obtained second positioning information can include information of all existing teeth and avoid omissions.

[0182] Center information can be determined by identifying the boundary distance information of the tooth. The reason is that boundary distance information can be used to characterize the distance between pixels within the tooth area and the tooth's boundary point, and / or the distance between pixels and the tooth's center in a root bone scan image. The closer a pixel is to the center point, the farther it is from its boundary; therefore, in these cases, center information can be indirectly characterized through boundary-related features.

[0183] The process of identifying the center information of different teeth to determine secondary positioning information can be achieved using a boundary distance regression model. Boundary distance regression models can include deep neural networks.

[0184] The boundary distance regression model can be used to output the distance from each pixel within the tooth's range to the nearest boundary point on that tooth. Based on the characteristic that "pixels closer to the tooth's center have larger distance values, and pixels closer to the tooth's boundary have smaller distance values," the center information of the tooth is determined. In some embodiments, step S13 can also be: inputting the root bone scan image into the boundary distance regression model to identify the center points of different teeth and obtain the second positioning information of M teeth.

[0185] The boundary distance regression model can include a deep convolutional neural network. For example, it can be a fully convolutional deep neural network. For example, it can be at least one of UNet, Atten-UNet, and UNet++.

[0186] The second location information can be a two-dimensional image, a two-dimensional image sequence, or a three-dimensional image. In an embodiment where the second location information is a three-dimensional image, the three-dimensional image can be determined based on an image that can characterize the boundary distances of pixels within the tooth area, using a region growing algorithm with the center point of the tooth as the seed point to determine the three-dimensional mask of the tooth, thereby determining the second location information in the form of a three-dimensional image.

[0187] In embodiments where the second positioning information is a two-dimensional image or a sequence of two-dimensional images, the second positioning information may be a boundary distance image that can characterize the boundary distance of points on the tooth, or a center mask image that can characterize the center point of the tooth.

[0188] In one embodiment, the tooth segmentation method provided in this application may include at least one of the following steps: determining the boundary information of the tooth based on the root bone scan image; and determining the boundary distance information of the tooth based on the tooth shape and boundary information.

[0189] This step can be included in step S13; when implementing step S13, this step can be implemented in detail.

[0190] Boundary distance information can be used to determine the second positioning information of the M teeth. The step of identifying center information in step S13 may include or include the step of determining boundary distance information.

[0191] Boundary distance information is used to characterize the distance between pixels within the tooth region and the tooth's boundary points in a root bone scan image, and / or the distance between pixels within the tooth region and the tooth's center. This boundary distance can be characterized by pixel values, grayscale values, colors, or numerical values. Based on the principle that "points closer to the tooth's center have larger distance values, and points closer to the tooth's boundary points have smaller distance values," center information can be determined based on boundary distance information, or boundary distance information can be determined based on the tooth's center information.

[0192] A boundary distance regression model can be pre-trained to take a root bone scan image as input and tooth boundary distance information as output. This model can be used to detect tooth morphological features, determine the tooth's extent, and then determine the boundary distances of pixels within that tooth's extent based on the boundary information. The root bone scan image can be a CBCT image, a two-dimensional image sequence, or a normalized image. The tooth boundary distance information can be in image form or a normalized image.

[0193] like Figure 6 The root bone scan image fig0 is included in the root bone scan image sequence set0. Based on the root bone scan image fig0, the boundary distance information of the tooth is determined, resulting in a boundary distance image fig21. In one embodiment, the boundary distance information of the tooth can be characterized by the boundary distance image fig21. In this case, the boundary distance information can be represented by the grayscale value of the pixels. For example, in the boundary distance image fig21, pixels closer to the center of the tooth within the tooth area are brighter, corresponding to larger grayscale values; pixels closer to the boundary of the tooth (away from the center) within the tooth area are darker, corresponding to smaller grayscale values.

[0194] In this embodiment, the tooth segmentation method provided by this application may include the step of: determining a boundary distance image of the teeth as boundary distance information based on a root bone scan image. In the boundary distance image, the boundary distance information is represented by the grayscale of pixels. The boundary distance information can be used to identify center information. This step may be included in the step "Identifying the center information of different teeth in the root bone scan image".

[0195] Boundary distance regression models can be used to determine the boundary distance images of teeth based on root bone scan images. The boundary distance regression model can be pre-trained to take a root bone scan image (fig0) as input and output the boundary distance image (fig21) corresponding to the tomographic location pointed to by fig0. The boundary distance regression model is used to determine boundary distances.

[0196] The root bone scan image can be a two-dimensional image. In this case, the boundary distance regression model can include a two-dimensional deep fully convolutional neural network. Boundary distance images corresponding to the same tooth are combined to represent the boundary distance information of that tooth, while boundary distance information corresponding to different teeth is used to determine the secondary localization information of the teeth.

[0197] The root bone scan image can be a three-dimensional image. In this case, the boundary distance regression model can include a three-dimensional deep fully convolutional neural network. The three-dimensional boundary distance image corresponding to the same tooth represents the boundary distance information of that tooth, while the boundary distance information corresponding to different teeth is used to determine the secondary localization information of the teeth.

[0198] The second localization information may include a central mask image. The central mask image is used to characterize the center positions of the M teeth. The central mask image can be used directly as the second localization information, or the second localization information can be determined based on the central mask image.

[0199] In one embodiment, the center mask image can be determined based on the boundary distance image to determine the center mask. For example... Figure 6 In the center mask image fig22, pixels with boundary distance information less than a preset threshold have a first grayscale value. For example, in the center mask image fig22, pixels on a tooth that are far from the center of the tooth and whose boundary distance information is less than the preset threshold are set to the first grayscale value; correspondingly, pixels on a tooth that are close to the center of the tooth and whose boundary distance is greater than the preset threshold can be set to a second grayscale value. The first grayscale value is 0, and the second grayscale value is 1. The center mask image fig22 can be determined based on the boundary distance image fig21 using a threshold segmentation method.

[0200] In this embodiment, the tooth segmentation method provided by this application may include the steps of: determining boundary distance information based on the boundary information of the teeth; determining the boundary distance information based on the shape and boundary information of the teeth; and determining the center mask image of the teeth based on the boundary distance information of the teeth.

[0201] Boundary distance information is used to characterize the distance between pixels within the tooth region in a root bone scan image and the boundary points of the tooth, and / or the distance between pixels within the tooth region in a root bone scan image and the center of the tooth.

[0202] In the central mask image, pixels with boundary distance information less than the preset threshold have the first gray value. Pixels with boundary distance information greater than the preset threshold have the second gray value. The central mask image is used to represent the center point of the tooth. This step can be included in the step of "identifying the center information of different teeth in the root bone scan image to obtain the second positioning information of M teeth".

[0203] A neural network can be used to determine the central mask image of the tooth based on the root bone scan image.

[0204] In one embodiment, the threshold segmentation method can be: set the preset threshold T = 0.8, and judge the numerical relationship between the gray value I(x, y) of the pixel in the boundary distance image (x, y) of the tooth and the preset threshold T. When I(x, y) < T, let I(x, y) = 0; when I(x, y) ≥ T, let I(x, y) = 1.

[0205] The boundary distance image of the tooth can be a grayscale image with gray values distributed between 0 and 255. The central mask image of the tooth can be a binary image.

[0206] The central mask image fig22 is used to determine the second positioning information. In one embodiment, the central mask image fig22 serves as the second positioning information.

[0207] The boundary distance information can also be used to determine the image of the region of interest of the first tooth. The first tooth can be all or part of the M teeth included in the boundary distance information. In one embodiment, the tooth segmentation method provided in this application can include the following steps: determine the image corresponding to the region of interest for the first tooth segmentation in the root bone scan image according to the boundary distance information.

[0208] This step can be included in determining the region of interest for tooth segmentation according to the first positioning information of N teeth and the second positioning information of M teeth; when this step is implemented, the above steps can be specifically implemented.

[0209] Step S14, determine the region of interest for tooth segmentation according to the first positioning information of N teeth and the second positioning information of M teeth.

[0210] Step S15, input the image corresponding to the region of interest for tooth segmentation into the segmentation model to obtain the tooth segmentation result.

[0211] N and M can be the same or different. If they are the same, it means that the number of teeth determined by the two positioning information is the same, and the tooth segmentation result can be obtained based on one of them. Preferably, determine the region of interest for tooth segmentation according to the first positioning information M, input the corresponding image into the segmentation model to obtain the tooth segmentation result.

[0212] In one embodiment, the tooth segmentation method provided in this application may include at least one of the following steps: determining N regions of interest for tooth segmentation based on first positioning information; inputting the images corresponding to the N regions of interest for tooth segmentation into a segmentation model to obtain tooth segmentation results.

[0213] At this point, the result of tooth segmentation is as follows: Figure 5 , Figure 7 , Figure 8 The segmentation results of N teeth are shown in info12.

[0214] If N and M are different, it indicates that one of the two localization information sets is missing, or that there is a discrepancy between them. In this case, they can be referenced and supplemented. For example, based on M teeth and N teeth, the differing teeth can be identified, and the region of interest for tooth segmentation can be determined based on the differing teeth. The corresponding image is then input into the segmentation model to obtain the tooth segmentation result.

[0215] The segmentation model may include at least one neural network. The neural network may be a deep neural network. The segmentation model may be configured as described above. The segmentation model for segmenting the image corresponding to the region of interest of N teeth, and the segmentation model for segmenting the image corresponding to the region of interest of the teeth segmented based on two localization information, may be one or two separate models.

[0216] In one embodiment, the region of interest for tooth segmentation includes regions of interest corresponding to differential teeth, which are determined based on M teeth and N teeth.

[0217] Based on this, in the step of determining the region of interest corresponding to the first tooth segmentation in the root bone scan image, the first tooth may include the difference between N teeth and M teeth.

[0218] A differential tooth can be a tooth included in the second positioning information but not included in the first positioning information. In this case, N can be less than M. For a tooth, if the second positioning information includes the center information of the tooth, but the first positioning information does not include the position information of the tooth, then the tooth can be identified as a differential tooth or a first tooth.

[0219] Difference teeth reflect the limitations of the first positioning information determined based on location. Difference teeth can be teeth that were missed by the first positioning information, or they can be supernumerary teeth, impacted teeth, etc.

[0220] The image corresponding to the region of interest (ROI) of the differentially segmented teeth can be input into the segmentation model to obtain the segmentation results of the differentially segmented teeth. The segmentation results of the differentially segmented teeth can be in the form of data information such as position coordinates, in the form of two-dimensional images, or in the form of three-dimensional models.

[0221] Determining the difference between N teeth and M teeth can be achieved by checking if a tooth included in the second positioning information exists in the first positioning information, and then assigning pixel values ​​to teeth that meet the criteria for a difference. Specifically, if a tooth included in the second positioning information exists in the first positioning information, its pixel value is assigned to 0; if a tooth included in the second positioning information does not exist in the first positioning information, its pixel value is assigned to the value found in the second positioning information.

[0222] Abstracting this mathematically, it can be understood as selecting teeth from the second localization information that are not present in the first localization information. For example, using the center mask image as the second localization information, assuming the center mask of M teeth is C, assuming the first localization information is S1, and assuming the pixel value of the corresponding difference tooth is Cm, then the pixel value Cm(x,y) at (x,y) of the pixel value Cm of the corresponding difference tooth satisfies:

[0223]

[0224] C(x,y) is the pixel value at (x,y) in the second positioning information, and S1(x,y) is the pixel value at (x,y) in the first positioning information. S1(x,y)>0 indicates that the pixel at (x,y) belongs to the foreground teeth; S1(x,y)=0 indicates that the pixel at (x,y) belongs to the background area.

[0225] Based on this, we can obtain the pixel value set Cm of all differential teeth in the second localization information. Cm can be the set of center masks of all differential teeth, and each connected component in Cm is the center mask of the differential tooth, such as... Figure 7 As shown.

[0226] In one embodiment, the tooth segmentation method provided in this application may include at least one of the following steps: determining N regions of interest for tooth segmentation based on first positioning information; determining regions of interest for differentially segmented teeth based on first positioning information and second positioning information. This step may be included in step S14; when implementing step S14, this step may be specifically implemented.

[0227] In one embodiment, N teeth and differentially segmented teeth can be segmented together. In this embodiment, the tooth segmentation method provided by this application may include at least one of the following steps: determining regions of interest (ROIs) for the N teeth segmentation based on first positioning information; inputting the images corresponding to the ROIs of the N teeth segmentation and the images corresponding to the ROIs of differentially segmented teeth into a segmentation model to obtain tooth segmentation results.

[0228] In one embodiment, N teeth and the differentially segmented teeth can be segmented separately, and then the two segmentation results can be combined to obtain the final tooth segmentation result. In this embodiment, the tooth segmentation method provided by this application may include at least one of the following steps: inputting the images corresponding to the regions of interest of the N teeth into a segmentation model to obtain the segmentation results of the N teeth; inputting the images corresponding to the regions of interest of the differentially segmented teeth into a segmentation model to obtain the segmentation results of the differentially segmented teeth; and obtaining the final tooth segmentation result based on the segmentation results of the N teeth and the segmentation results of the differentially segmented teeth. This step may be included in step S15; when implementing step S15, this step can be specifically implemented.

[0229] The region of interest for segmenting differential teeth can be the area where the differential teeth are located in the root bone scan image (after cropping, the image corresponding to the region of interest for segmenting differential teeth can be obtained).

[0230] In one specific embodiment, the tooth segmentation method provided in this application may include the steps of: determining second positioning information of the differential teeth in second positioning information; and determining the image corresponding to the region of interest of the differential teeth in the root bone scan image based on boundary distance information and the second positioning information of the differential teeth. The boundary distance information is used to characterize the distance between pixels within the tooth range in the root bone scan image and the boundary points of the teeth, and / or the distance between pixels within the tooth range in the root bone scan image and the center of the teeth.

[0231] This step can be included in step S14; when implementing step S14, this step can be implemented in detail.

[0232] To identify the region of interest for differentially expressed teeth in a root bone scan image, the region growing method can be specifically employed.

[0233] The second localization information used to determine the difference in teeth can be the second localization information itself, or the center reconstruction result of M teeth reconstructed based on the second localization information.

[0234] In one embodiment, the tooth segmentation method provided in this application may include at least one of the following steps: determining N regions of interest for tooth segmentation based on first positioning information; inputting the images corresponding to the N regions of interest for tooth segmentation into a segmentation model to obtain segmentation results for N teeth; performing three-dimensional reconstruction based on second positioning information to obtain center reconstruction results for M teeth; comparing the segmentation results for N teeth with the center reconstruction results for M teeth to determine the teeth with differences. This step may be included in step S14; when implementing step S14, this step may be specifically implemented.

[0235] For example, such as Figure 5After obtaining the first localization information loc11, the image set111 corresponding to the region of interest for N teeth segmentation in the root bone scan image can be determined based on the first localization information, and then input into the segmentation model to obtain the segmentation result info12 for the N teeth. For example... Figure 6 After obtaining the second localization information in the form of the center mask image fig22, three-dimensional reconstruction can be performed based on the second localization information to obtain the center reconstruction results info2 of M teeth. Figure 7 By comparing the segmentation results info12 of N teeth with the center reconstruction results info2 of M teeth, the center reconstruction results info3 of the differential teeth can be obtained as the second localization information of the differential teeth, thus identifying the differential teeth.

[0236] In other embodiments, the difference teeth can also be determined by comparing the first positioning information loc11 with the central mask image fig22, and the central mask image of the difference teeth can be used as the second positioning information of the difference teeth.

[0237] As mentioned above, one embodiment of this application further includes the steps of: determining the image corresponding to the region of interest of the first tooth segmentation in the root bone scan image based on the boundary distance information; and inputting the image corresponding to the region of interest of the tooth segmentation into the segmentation model to obtain the tooth segmentation result.

[0238] When the first tooth includes a differential tooth, the tooth segmentation result shall include at least the segmentation result of the differential tooth.

[0239] Continue as Figure 8 As shown, after determining the center reconstruction result info3 of the differential tooth as the second localization information of the differential tooth, the position of the differential tooth can be determined in the boundary distance image (as boundary distance information, contained in the boundary distance image sequence set21), and the image corresponding to the region of interest of the differential tooth segmentation can be determined in the root bone scan image based on its position in the boundary distance image.

[0240] When the boundary distance image is in the form of a two-dimensional image, the central mask image or central reconstruction result corresponding to the differential tooth can be determined first, and the boundary distance image corresponding to the differential tooth can be determined accordingly. In the boundary distance image corresponding to the differential tooth, the center point of the differential tooth can be used as the seed point to segment it using a two-dimensional region growing algorithm to obtain the two-dimensional distribution region of the differential tooth in the boundary distance image (which can be represented in the form of a two-dimensional mask image). Multiple boundary distance images corresponding to the differential tooth contain multiple two-dimensional distribution regions, which can be combined to obtain the three-dimensional distribution region of the corresponding differential tooth. Based on this three-dimensional distribution region, the local region of interest image can be determined in the root bone scan image.

[0241] When the boundary distance image has the form of a three-dimensional image, the central mask image or central reconstruction result corresponding to the differential teeth can be determined first. Then, the center point of the differential teeth can be used as the seed point to segment the region using a three-dimensional region growing algorithm to obtain the three-dimensional region distribution of the corresponding differential teeth, thereby determining the local region of interest image.

[0242] The images corresponding to the regions of interest (ROIs) of differentially segmented teeth are cropped from the root bone scan images based on the ROIs of differentially segmented teeth. The root bone scan images constitute the root bone scan image sequence set0, and the cropped ROI images constitute the image sequence set3 corresponding to the ROIs of differentially segmented teeth. The number of images corresponding to the ROIs in the image sequence set3 is less than or equal to the number of root bone scan images in the image sequence set0. Each image corresponding to the ROI of differentially segmented teeth has a corresponding root bone scan image, and each image has a corresponding tomographic region of the subject.

[0243] The region of interest corresponding to the segmentation of differential teeth can be represented by a bounding box surrounding the differential teeth. The bounding box can be a two-dimensional bounding box or a three-dimensional bounding box.

[0244] In one specific embodiment, the tooth segmentation method provided in this application may include the steps of: determining the region of interest for segmenting the differential teeth and obtaining a third bounding box for the corresponding differential teeth; adjusting the size of the third bounding box according to a preset expansion value to obtain a fourth bounding box; and determining the region of interest for segmenting the differential teeth in the root bone scan image according to the fourth bounding box.

[0245] The region of interest for the differential tooth segmentation can be either a two-dimensional or three-dimensional distribution region as described above. Adjusting the size of the third bounding box can involve expanding it to obtain a fourth bounding box with a larger volume or area. When the third bounding box is a two-dimensional rectangle, it can be determined based on the coordinates of its diagonal vertices. Adjusting the size of the third bounding box can be achieved by adjusting the coordinates of at least one of its two diagonal vertices. When the third bounding box is a three-dimensional rectangle, it can be determined based on the coordinates of its diagonal vertices. Adjusting the size of the third bounding box can be achieved by adjusting the coordinates of at least one of its two diagonal vertices.

[0246] For example, consider the third vertex v3 (which could be the top-left front vertex of the 3D rectangle enclosing the teeth with differences) and the fourth vertex v4 (which could be the bottom-right back vertex of the 3D rectangle enclosing the teeth with differences). The coordinates of the third vertex v3 are (x3, y3, z3), and the coordinates of the fourth vertex v4 are (x4, y4, z4). Based on this, the coordinates of the two vertices can be adjusted to expand the third bounding box into a fourth bounding box. The coordinates of the new third vertex v3' are (x3-margin_x, y3-margin_y, z3-margin_z), and the coordinates of the new fourth vertex v4' are (x4+margin_x, y4+margin_y, z4+margin_z).

[0247] The reason for adjusting the size of the bounding box to expand it is to include part of the area around the differential teeth (background), which helps improve the recognition accuracy of differential teeth (foreground).

[0248] The magnitude of the expansion (i.e., the aforementioned margin_x, margin_y, and margin_z) can be preset. In one embodiment, the preset expansion value is set as follows: the magnitude by which the third bounding box is expanded is a preset multiple of the corresponding side length within the third bounding box. Specifically, the preset value is 0.2.

[0249] For example, when adjusting the size of the x-coordinate direction, the expansion is equal to 0.2 times the side length of the third bounding box in the x-coordinate direction (e.g., |x4-x3|). The same applies to other coordinate directions.

[0250] For example, the default extended value can be set as follows:

[0251] margin_x = 0.2 × (x4 - x3);

[0252] margin_y = 0.2 × (y4 - y3);

[0253] margin_z = 0.2 × (z4 - z3).

[0254] Once the fourth bounding box is determined, it can characterize the region of the differentially defined teeth formed after the boundary expansion. Based on the fourth bounding box, the region of the differentially defined teeth can be identified in the root bone scan image. Then, based on the identified region of interest, the root bone scan image can be cropped to obtain the image corresponding to the segmented region of interest of the differentially defined teeth.

[0255] Based on the boundary distance information and the second localization information of the differential teeth, after determining the image corresponding to the region of interest for segmentation of the differential teeth in the root bone scan image, tooth segmentation can be performed based on the image (e.g., inputting the image into the segmentation model) to obtain the segmentation result 3d-3 of the differential teeth.

[0256] like Figure 8 The segmentation result 3d-3 of the differentially segmented teeth can be merged with the segmentation results info12 of N teeth to obtain the final tooth segmentation result 3d-0. The segmentation results 3d-3 of the differentially segmented teeth, info12 of the N teeth, and the final tooth segmentation result 3d-0 can be two-dimensional images, three-dimensional images, or three-dimensional models generated based on three-dimensional reconstruction of two-dimensional images.

[0257] In this application, a segmentation model can be used for tooth segmentation to obtain segmentation results of 3d-3 for teeth with differences. The segmentation model may include at least one neural network, and the segmentation model can be configured as described above. The segmentation model used to segment the image corresponding to the region of interest of N teeth and the segmentation model used to segment the image corresponding to the region of interest of the teeth with differences can be one or two separate models.

[0258] The segmentation model can be pre-trained to take as input an image the region of interest (ROI) corresponding to the segmented teeth with differences in morphological features, and as output a two-dimensional image. The ROI input to the segmentation model can be a normalized image corresponding to the ROI of the teeth with differences in morphology. If the output of the segmentation model is a two-dimensional image, this image can specifically be a mask image. The mask image can be a binary image, where 1 represents the foreground and 0 represents the background.

[0259] The region of interest (ROI) image can be a two-dimensional image. A sequence of ROI images for segmenting differentially segmented teeth is input into a segmentation model. Multiple processed two-dimensional images can be combined to obtain a two-dimensional output image sequence. In this case, the segmentation model can include a two-dimensional deep fully convolutional neural network. The two-dimensional output image sequence can be used to obtain the segmentation results for differentially segmented teeth. Alternatively, the ROI image can be a three-dimensional image. The ROI images for segmenting differentially segmented teeth are input into a segmentation model. After processing, the segmentation results for differentially segmented teeth are obtained. In this case, the segmentation model can include a three-dimensional deep fully convolutional neural network. The neural network can be at least one of UNet, Atten-UNet, or UNet++.

[0260] like Figure 9 One embodiment of this application provides a display method.

[0261] The application or instructions corresponding to this method can be mounted in an electronic device, a tooth segmentation device and / or a storage medium, or in a carrier that implements the display method, to achieve the corresponding technical effect.

[0262] The display method may specifically include at least one of the following steps.

[0263] Step S21: Present the final tooth segmentation result on the graphical user interface.

[0264] The tooth segmentation result is obtained by inputting the image corresponding to the segmented region of interest into the segmentation model. The final tooth segmentation result can be a two-dimensional image, a three-dimensional model, or a three-dimensional image; for example... Figure 8 In the end, the final tooth segmentation result was 3d-0.

[0265] The region of interest for tooth segmentation is determined based on the first localization information of N teeth and the second localization information of M teeth. N and M are positive integers greater than 1.

[0266] The first positioning information is obtained by inputting the root bone scan image into the positioning model. The first positioning information is used to determine the position information of the tooth in the root bone scan image.

[0267] The second positioning information is obtained by identifying the center information of different teeth in the root bone scan image.

[0268] The second localization information can be a center mask image. The center mask image can be determined based on the boundary distance information, which can be represented by a boundary distance image.

[0269] The first and second location information point to the same object being measured.

[0270] The first localization information includes the localization information of N teeth, and the second localization information includes the localization information of M teeth. The N teeth and M teeth may include teeth that differ from each other. The region of interest for tooth segmentation may include the regions of interest of the differing teeth.

[0271] When performing step S21, it is not necessarily required to display the corresponding first positioning information, second positioning information, segmentation results of N teeth, segmentation results of M teeth, or segmentation results of differential teeth.

[0272] In one embodiment, the display method provided in this application may include the step of: presenting a bounding box surrounding each tooth in a graphical user interface. This step may be included in step S21; when implementing step S21, this step may be specifically implemented.

[0273] The bounding box can be a two-dimensional or three-dimensional bounding box. The bounding box is determined by a neural network based on the location of different teeth in the root bone scan image.

[0274] Step S22: Present the radicular bone scan image in the graphical user interface.

[0275] The calcaneal scan image can be a CBCT image.

[0276] The root bone scan image is overlaid with the final tooth segmentation result.

[0277] The features related to the final tooth segmentation result can be implemented by referring to the description corresponding to step S21, or by referring to the tooth segmentation method in any of the preceding embodiments.

[0278] In one embodiment, the display method provided in this application may include the step of: presenting a bounding box surrounding each tooth in a graphical user interface. This step may be included in step S22; when implementing step S22, this step may be specifically implemented.

[0279] The bounding box can be a two-dimensional or three-dimensional bounding box. The bounding box is determined by a neural network based on the location of different teeth in the root bone scan image.

[0280] Step S23: Present the 3D model of the teeth after segmentation in the graphical user interface.

[0281] The three-dimensional model is generated by three-dimensional reconstruction based on the final tooth segmentation results.

[0282] The features related to the final tooth segmentation result can be implemented by referring to the description corresponding to step S21, or by referring to the tooth segmentation method in any of the preceding embodiments.

[0283] In one embodiment, the display method provided in this application may include the step of: presenting a bounding box surrounding each tooth in a graphical user interface. This step may be included in step S23; when implementing step S23, this step may be specifically implemented.

[0284] The bounding box can be a two-dimensional or three-dimensional bounding box. The bounding box is determined by a neural network based on the location of different teeth in the root bone scan image.

[0285] The parts of the display method provided in this application that are not fully described can be implemented as corresponding parts in other technical solutions of this application, such as the tooth segmentation method. Similarly, the various technical solutions, implementation methods, and embodiments provided in this application can be combined with each other or used to explain and illustrate each other.

[0286] In summary, the tooth segmentation method provided in this application determines the region of interest (ROI) for tooth segmentation based on two types of tooth localization information, thereby improving the accuracy of tooth localization and detection. When using the obtained ROI for tooth segmentation, it can achieve complete and accurate tooth segmentation, avoiding the omission of special cases such as impacted teeth and supernumerary teeth. One type of localization information is used to determine the position of the tooth in the root bone scan image, enabling the tooth segmentation result to have the accuracy of single-tooth segmentation for conventional teeth; the other type of localization information is obtained by identifying the center information of the root bone scan image, ensuring that the tooth segmentation result includes all teeth of the subject without omission.

[0287] It should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This way of describing the specification is only for clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

[0288] The detailed descriptions listed above are merely specific descriptions of feasible implementation methods of this application and are not intended to limit the scope of protection of this application. All equivalent implementation methods or modifications made without departing from the spirit of the art of this application should be included within the scope of protection of this application.

Claims

1. A method for tooth segmentation, characterized in that, include: Obtain radicular bone scan images; The root bone scan image is input into the localization model to obtain the first localization information of N teeth. The first localization information is used to determine the position information of the teeth in the root bone scan image. The center information of different teeth in the root bone scan image is identified to obtain the second positioning information of M teeth; N and M are positive integers greater than 1; Based on the first localization information of N teeth and the second localization information of M teeth, the region of interest for tooth segmentation is determined; The image corresponding to the region of interest in the segmented teeth is input into the segmentation model to obtain the tooth segmentation result.

2. The tooth segmentation method according to claim 1, characterized in that, include: Based on the root bone scan images, the boundary information of the teeth is determined. Based on the morphology and boundary information of the teeth, the boundary distance information is determined. The boundary distance information is used to characterize the distance between the pixels within the tooth range in the root bone scan image and the boundary point of the tooth, and / or the distance between the pixels within the tooth range in the root bone scan image and the center of the tooth. Based on the boundary distance information, the image corresponding to the region of interest segmented from the first tooth is determined in the root bone scan image.

3. The tooth segmentation method according to claim 2, characterized in that, The first tooth includes N teeth that differ from M teeth, where N is less than M.

4. The tooth segmentation method according to claim 3, characterized in that, include: Based on the initial localization information, determine N regions of interest for tooth segmentation; The images corresponding to the regions of interest (ROIs) of N segmented teeth, and the images corresponding to the ROIs of differentially segmented teeth, are input into the segmentation model to obtain the tooth segmentation results, or... The images corresponding to the regions of interest of N teeth are input into the segmentation model to obtain the segmentation results of N teeth. The image corresponding to the region of interest in the segmented teeth is input into the segmentation model to obtain the segmentation result of the teeth with differences. Based on the segmentation results of N teeth and the segmentation results of the differential teeth, the final tooth segmentation result is obtained.

5. The tooth segmentation method according to claim 3, characterized in that, include: Based on the initial localization information, determine N regions of interest for tooth segmentation; The images corresponding to the regions of interest of N teeth are input into the segmentation model to obtain the segmentation results of N teeth; Based on the second positioning information, a three-dimensional reconstruction is performed to obtain the center reconstruction results of M teeth; Compare the segmentation results of N teeth with the center reconstruction results of M teeth to identify the teeth with discrepancies.

6. The tooth segmentation method according to claim 1, characterized in that, The first positioning information includes information about the bounding box surrounding the N teeth; the method includes: Based on the bounding boxes of N teeth, the root bone scan image is cropped to obtain images corresponding to the regions of interest segmented from the N teeth; The images corresponding to the regions of interest of N teeth are input into the segmentation model to obtain the segmentation results of N teeth.

7. The tooth segmentation method according to claim 1, characterized in that, The second positioning information includes a central mask image, which is used to characterize the center position of the M teeth.

8. The tooth segmentation method according to claim 7, characterized in that, The method includes: Based on the root bone scan images, determine the boundary information of the teeth; Based on the morphology and boundary information of the teeth, the boundary distance information is determined. The boundary distance information is used to characterize the distance between the pixels within the tooth range in the root bone scan image and the boundary point of the tooth, and / or the distance between the pixels within the tooth range in the root bone scan image and the center of the tooth. Based on the boundary distance information of the teeth, a central mask image of the teeth is determined. Pixels with boundary distance information less than a preset threshold have a first gray value, and pixels with boundary distance information greater than the preset threshold have a second gray value.

9. The tooth segmentation method according to claim 1, characterized in that, The calcaneal scan images include cone-beam computed tomography (CBCT) images.

10. A display method, characterized in that, Includes at least one of the following: Present the tooth segmentation results in a graphical user interface; The root bone scan image is presented in a graphical user interface, and the tooth segmentation result is superimposed on the root bone scan image. A 3D model of the teeth after segmentation is presented in a graphical user interface. The 3D model is generated by 3D reconstruction based on the tooth segmentation results. The tooth segmentation result is obtained by inputting the image corresponding to the segmented region of interest into the segmentation model. The region of interest of the segmented tooth is determined based on the first localization information of N teeth and the second localization information of M teeth. The first localization information is obtained by inputting the root bone scan image into the localization model, and the second localization information is obtained by identifying the center information of different teeth in the root bone scan image. The first localization information is used to determine the position information of the teeth in the root bone scan image, where N and M are positive integers greater than 1.

11. The display method according to claim 10, characterized in that, Includes at least one of the following: The graphical user interface displays a bounding box surrounding each tooth. The bounding box can be a three-dimensional bounding box or a two-dimensional bounding box. The bounding box is determined by a neural network based on the location of different teeth according to the root bone scan image.

12. A tooth-splitting device, characterized in that, include: The first module is used to obtain pedicle scan images; The second module is used to process the root bone scan image with a positioning model to obtain the first positioning information of N teeth. The first positioning information is used to determine the position information of the teeth in the root bone scan image, where N is a positive integer greater than 1. The third module is used to identify the center information of different teeth in the root bone scan image and obtain the second positioning information of M teeth, where M is a positive integer greater than 1; The fourth module is used to determine the region of interest for tooth segmentation based on the first localization information of N teeth and the second localization information of M teeth. The fifth module is used to process the image corresponding to the region of interest in tooth segmentation using a segmentation model to obtain the tooth segmentation result.

13. The tooth-splitting device according to claim 12, characterized in that, The tooth segmentation device is connected to a first device, which is used to perform cone-beam circumferential radiography to obtain the root bone scan image.

14. An electronic device comprising a processor, a memory, and a communication bus, characterized in that, The processor and the memory communicate with each other via the communication bus; The memory is used to store application programs; The processor is configured to, when executing an application stored in the memory, implement the steps of the tooth segmentation method according to any one of claims 1-9, or implement the steps of the display method according to any one of claims 10-11.

15. A storage medium having an application program stored thereon, characterized in that, When the application is executed, it implements the steps of the tooth segmentation method according to any one of claims 1-9, or the steps of the display method according to any one of claims 10-11.

Images