Method for automatically segmenting teeth from three-dimensional volume data, and computer-readable recording medium having, recorded thereon, program for executing same on computer

Abstract

A method for automatically segmenting teeth from three-dimensional volume data comprises the steps of: determining feature points in three-dimensional volume data; generating a two-dimensional panoramic image from the three-dimensional volume data on the basis of the feature points; generating a two-dimensional bounding box corresponding to teeth from the two-dimensional panoramic image; generating a three-dimensional bounding box and three-dimensional sub-volume data from the three-dimensional volume data on the basis of the two-dimensional bounding box; and segmenting the teeth from the three-dimensional sub-volume data generated from the three-dimensional volume data on the basis of the three-dimensional bounding box.

Description

A method for automatically separating teeth from three-dimensional volume data and a computer-readable recording medium having recorded thereon a program for executing the method on a computer

[0001] The present invention relates to a method for automatically separating teeth from three-dimensional volume data and a computer-readable recording medium having recorded thereon a program for executing the same on a computer, and more particularly, to a method for automatically separating teeth from three-dimensional volume data, which is automatically performed through deep learning and can reduce the time and effort required for separating teeth from three-dimensional volume data, and a computer-readable recording medium having recorded thereon a program for executing the same on a computer.

[0002] 3D volume data represents data accumulated from hundreds of 2D slide images, such as CT (Computed Tomography), CBCT (Cone-Beam CT), and MRI (Magnetic Resonance Imaging). The 3D volume data can be acquired targeting the head and neck region in dentistry, oral and maxillofacial surgery, and plastic surgery, and can be utilized to diagnose and treat patients' maxillofacial and oral cavity.

[0003] In particular, in order to establish a diagnosis and treatment plan for the purpose of implant placement, maxillofacial correction, orthodontics, etc., or to compare before and after treatment, a process of three-dimensionally analyzing the shape of individual tooth areas and the direction of teeth is necessary.

[0004] However, since the 3D volume data only has intensity information of a single channel, a process of segmenting the region or reconstructing the segmented data into a 3D model is required to analyze a specific region 3Dly. However, since the 3D volume data is data accumulated from hundreds of 2D slide images, directly separating the teeth from the 3D volume data may require a relatively large amount of computational processing. Since the condition of the teeth varies depending on the patient, it may be difficult to accurately segment individual tooth regions. Maxillofacial surgery plates, metal prostheses for the teeth, orthodontic devices, etc. may cause metal artifacts in 3D volume data using X-rays (e.g., CT or CBCT), making it difficult to confirm the boundaries of the individual tooth regions. In addition, the root region of the teeth is included inside the maxilla and mandible, and the brightness values ​​of the bone tissue and the tooth tissue are similar, making it difficult to accurately segment the root region of the teeth.

[0005] The purpose of the present invention is to provide a method for automatically separating teeth from 3D volume data, which is performed automatically through deep learning, thereby reducing the time and effort required to separate teeth from 3D volume data and improving accuracy.

[0006] Another object of the present invention is to provide a computer-readable recording medium having recorded thereon a program for executing a method for automatically separating teeth from the three-dimensional volume data on a computer.

[0007] According to one embodiment of the present invention, a method for automatically separating teeth from three-dimensional volume data includes a step of determining feature points in three-dimensional volume data, a step of generating a two-dimensional panoramic image from the three-dimensional volume data based on the feature points, a step of generating a two-dimensional bounding box corresponding to a tooth in the two-dimensional panoramic image, a step of generating a three-dimensional bounding box and three-dimensional sub-volume data from the three-dimensional volume data based on the two-dimensional bounding box, and a step of separating the tooth from the three-dimensional sub-volume data.

[0008] In one embodiment of the present invention, the number of feature points may be 3 or more.

[0009] In one embodiment of the present invention, the three-dimensional volume data may be input into the first artificial intelligence neural network to output the feature points.

[0010] In one embodiment of the present invention, a spline curve can be generated based on the feature points, and the two-dimensional panoramic image can be generated based on the spline curve.

[0011] In one embodiment of the present invention, an extended spline curve is generated based on a first normal vector of the spline curve, a reduced spline curve is generated based on a second normal vector of the spline curve, and the two-dimensional panoramic image can be generated using voxels included in a reference area between the extended spline curve and the reduced spline curve.

[0012] In one embodiment of the present invention, the voxels of the reference area may include the teeth.

[0013] In one embodiment of the present invention, the teeth included in the two-dimensional panoramic image can be separated by a two-dimensional segmentation mask.

[0014] In one embodiment of the present invention, the number of the two-dimensional segmentation masks is 1, and each tooth included in the two-dimensional segmentation mask may have a different brightness value.

[0015] In one embodiment of the present invention, teeth having the same classification number among the teeth can be separated with the same two-dimensional segmentation mask, and teeth having different classification numbers among the teeth can be separated with different two-dimensional segmentation masks.

[0016] In one embodiment of the present invention, the two-dimensional bounding box can surround the tooth.

[0017] In one embodiment of the present invention, the parameter defining the two-dimensional bounding box is c ci , x ui , y ui , x li , y li (where i is a positive integer greater than or equal to 1 and less than or equal to N), and the above c ci represents the classification number of the above tooth, and the above x ui , y ui represents the coordinates of the upper corner of the first side of the two-dimensional bounding box, and x li , y li may represent the coordinates of the lower corner of the second side of the above two-dimensional bounding box.

[0018] In one embodiment of the present invention, the parameter defining the two-dimensional bounding box is c ci , x ci , y ci , h i , w i (where i is a positive integer greater than or equal to 1 and less than or equal to N), and the above c ci represents the classification number of the above tooth, and the above x ci , y ci represents the coordinates of the center point of the above two-dimensional bounding box, and h irepresents the upper and lower width of the above two-dimensional bounding box, and w i can represent the left and right width of the above two-dimensional bounding box.

[0019] In one embodiment of the present invention, the two-dimensional panoramic image may be input into a second artificial intelligence neural network, and a two-dimensional segmentation mask or a two-dimensional bounding box corresponding to the tooth may be output.

[0020] In one embodiment of the present invention, the operation of generating the 3D bounding box from the 3D volume data based on the 2D bounding box may utilize the inverse transformation of the operation of generating the 2D panoramic image from the 3D volume data.

[0021] In one embodiment of the present invention, the 3D sub-volume data may be input into a third artificial intelligence neural network, and a 3D volume segmentation mask image of a tooth separated from the 3D sub-volume data may be output.

[0022] In one embodiment of the present invention, the method for automatically separating teeth from the three-dimensional volume data may further include a step of rendering the separated teeth from the three-dimensional volume data.

[0023] In one embodiment of the present invention, the method for automatically separating teeth from the three-dimensional volume data may further include a step of converting the teeth separated from the three-dimensional volume data into mesh data.

[0024] In one embodiment of the present invention, a program for executing the automatic tooth separation method of the three-dimensional volume data on a computer can be recorded on a computer-readable recording medium.

[0025] According to a method for automatically separating teeth from three-dimensional volume data according to the present invention and a computer-readable recording medium having recorded thereon a program for executing the method on a computer, feature points can be determined from three-dimensional volume data, a two-dimensional panoramic image can be generated from the three-dimensional volume data based on the feature points, a two-dimensional bounding box corresponding to a tooth can be generated from the two-dimensional panoramic image, a three-dimensional bounding box can be generated from three-dimensional volume data based on the two-dimensional bounding box, three-dimensional sub-volume data can be generated from the three-dimensional volume data based on the three-dimensional bounding box, and the tooth can be separated from the three-dimensional sub-volume data. Accordingly, the operation processing operation can require less time and be accurate.

[0026] In addition, since at least one of the steps of determining the feature points from the three-dimensional volume data, the step of generating the two-dimensional bounding box corresponding to the tooth from the two-dimensional panoramic image, and the step of separating the tooth from the three-dimensional sub-volume data is performed using an artificial intelligence neural network, the computational processing operation can require less time and be accurate.

[0027] FIG. 1 is a flowchart illustrating a method for automatically separating teeth from three-dimensional volume data according to one embodiment of the present invention.

[0028] FIG. 2 is a diagram showing a step of creating a two-dimensional panoramic image from three-dimensional volume data based on the feature points of FIG. 1.

[0029] Figure 3 is a drawing showing teeth in the two-dimensional panoramic image of Figure 2.

[0030] Figures 4 to 6 are conceptual diagrams explaining the characteristic points of Figure 2.

[0031] Figures 7 to 9 are conceptual diagrams explaining a spline curve and a two-dimensional panoramic image generated based on feature points.

[0032] FIG. 10 and FIG. 11 are drawings showing a step of generating a two-dimensional bounding box corresponding to a tooth in the two-dimensional panoramic image of FIG. 1.

[0033] Figures 12 and 13 are drawings explaining the two-dimensional bounding box of Figure 11.

[0034] Figure 14 is a drawing in which the two-dimensional bounding box of Figure 11 is reflected in the two-dimensional panoramic image of Figure 1.

[0035] Figure 15 is a drawing showing a panoramic x-ray image.

[0036] FIG. 16 is a diagram showing a step of creating a 3D bounding box from 3D volume data based on the 2D bounding box of FIG. 1 and using this to create 3D sub-volume data from the 3D volume data.

[0037] FIG. 17 is a diagram showing a step of separating teeth from 3D sub-volume data generated from 3D volume data based on the 3D bounding box of FIG. 1.

[0038] Fig. 18 is a drawing showing a tooth separated from the 3D sub-volume data of Fig. 1.

[0039] Fig. 19 is a drawing showing a step of rendering teeth separated from all 3D sub-volume data of Fig. 1 into 3D volume data.

[0040] With respect to the embodiments of the present invention disclosed in the text, specific structural and functional descriptions are merely illustrative for the purpose of explaining the embodiments of the present invention, and the embodiments of the present invention may be implemented in various forms and should not be construed as being limited to the embodiments described in the text.

[0041] The present invention is susceptible to various modifications and takes various forms. Specific embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the present invention to specific disclosed forms, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0042] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms may be used to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0043] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0044] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0045] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be construed in an idealized or overly formal sense unless explicitly defined herein.

[0046] Meanwhile, if a particular embodiment can be implemented differently, the functions or operations specified within a particular block may occur in a different order than specified in the flowchart. For example, two consecutive blocks may actually be executed substantially simultaneously, or, depending on the related functions or operations, the blocks may be executed in reverse order.

[0047] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Identical components in the drawings are designated by the same reference numerals, and redundant descriptions of identical components are omitted.

[0048] FIG. 1 is a flowchart illustrating a method for automatically separating teeth from three-dimensional volume data according to one embodiment of the present invention.

[0049] Referring to FIG. 1, a method for automatically separating teeth from 3D volume data according to the present embodiment may include a step of determining feature points in 3D volume data (step S100), a step of generating a 2D panoramic image from the 3D volume data based on the feature points (step S200), a step of generating a 2D bounding box corresponding to a tooth in the 2D panoramic image (step S300), a step of generating a 3D bounding box and 3D sub-volume data from the 3D volume data based on the 2D bounding box (step S400), and a step of separating the tooth from the 3D sub-volume data (step S500). The method for automatically separating teeth from 3D volume data may further include a step of rendering the separated tooth from the 3D volume data (step S600). The method for automatically separating teeth from 3D volume data may further include a step of converting the separated tooth from the 3D volume data into mesh data (step S700).

[0050] The method for automatically separating teeth from the above three-dimensional volume data can be performed by a computing device.

[0051] Here, the 3D volume data refers to data obtained by photographing the head and neck region, including the teeth and oral cavity. For example, the 3D volume data may be data accumulated from hundreds of 2D slide images, such as CT (Computed Tomography), CBCT (Cone-Beam CT), and MRI (Magnetic Resonance Imaging). The 3D volume data may be expressed as an intensity value.

[0052] According to the method for automatically segmenting teeth from 3D volume data according to the present embodiment, individual teeth can be segmented completely automatically from 3D volume data using deep learning. Fig. 1 illustrates the overall flowchart of the method for automatically segmenting teeth from 3D volume data. Each step is described in detail in Figs. 2 to 19.

[0053] FIG. 2 is a diagram showing a step (step S200) of generating a two-dimensional panoramic image (2PI) from three-dimensional volume data (3VD) based on the feature points of FIG. 1. FIG. 3 is a diagram showing a tooth (TT_2PI) of the two-dimensional panoramic image (2PI) of FIG. 2. FIGS. 4 to 6 are conceptual diagrams explaining the feature points of FIG. 2. FIGS. 7 to 9 are conceptual diagrams explaining a spline curve (SL) and a two-dimensional panoramic image (2PI) generated based on the feature points.

[0054] Referring to FIGS. 1 to 9, when directly segmenting the tooth (TT_2PI) from the three-dimensional volume data (3VD), a relatively large number of computational processing operations may be required. Therefore, the computational processing operations may require a lot of time and computer resources, and may be inaccurate. The method for automatically segmenting teeth from the three-dimensional volume data (3VD) may perform the computational processing operations by converting the three-dimensional volume data (3VD) into the two-dimensional panoramic image (2PI).

[0055] The method for automatically separating teeth from the above 3D volume data (3VD) may include the step (step S100) of determining the feature points from the 3D volume data (3VD) and the step (step S200) of generating the 2D panoramic image (2PI) from the 3D volume data (3VD) based on the feature points.

[0056] The step (step S100) of determining the feature points from the three-dimensional volume data (3VD) may be manually processed by a user or may be automatically processed by deep learning. For example, the step (step S100) of determining the feature points from the three-dimensional volume data (3VD) may be performed using a first artificial intelligence neural network. That is, the three-dimensional volume data (3VD) may be input into the first artificial intelligence neural network, and the three-dimensional coordinates of the feature points may be output. The first artificial intelligence neural network is disclosed in Republic of Korea Registration No. 10-2373500, Republic of Korea Registration No. 10-2334480, etc., and the contents disclosed in the above patent registration documents may be incorporated by reference as if fully disclosed in the present specification.

[0057] Figures 4 to 6 illustrate examples of the above-described feature points. For example, the feature points may be maxillofacial feature points existing on the skin surface. Figure 4 illustrates maxillofacial feature points existing on the skin surface. The feature points of Figure 4 may include Soft Tissue Glabella (Soft_Glabella), Soft Tissue Nasion (Soft_N), Pronasale (Pn), Soft Tissue A-point (Sls), Right Alar Base (Ala_R), Left Alar Base (Ala_L), Stomion Superior (Sts), Stomion Inferius (Sti), Mentolabial Sulcus (Si), and Soft Tissue Pogonion (Soft_Pog). For example, the feature points may be feature points associated with the maxillofacial bone. Figures 5a to 5d illustrate feature points associated with the maxillofacial bone. The feature points of FIGS. 5A to 5D can be expressed according to the frontal portion (A), the cross-section of the side portion (B), the side portion (C), and the bottom portion (D).The characteristic points in Figures 5a to 5d are Sella (S), Nasion (N), Anterior Nasal Spine (ANS), Point-A (A), Posterior Nasal Spine (PNS), Point-B (B), Pogonion (Pg), Gnathion (Gn), Right / Left of Orbitale Superius (OrSR / OrSL), Right / Left of Orbitale Inferius(OriR / OriL), Right / Left of Sutura Zygomaticofrontale(ZyFrR / ZyFrL), Right / Left of Foramen Mentale(FoMR / FoML), Basion(Ba), Right Porion(PoR), Right / Left of Condylus Medialis(CmR / CmL), Right / Left of Condylus Lateralis(ClR / ClL), Right / Left of Areus Zygomatieus (ArZyR / ArZyL), Right / Left of Inferior Gonion (IGoR / IGoL), Right / Left of Posterior Gonion (PGoR / PGoL), Right of Processus Coronoideus (PrCor). For example, the above-described feature points may be feature points associated with a tooth (TT_2PI). Fig. 6 shows feature points associated with a tooth (TT_2PI). Feature points associated with the root of the tooth (TT_2PI) may exist inside the jaw. The feature points of Fig. 6 may include Central Incisor Root, Mid Point of Central Incisors, First Molar Distal Root, Canine Root, Distal Point of First Molar crown, Cusp Tip, Distal Point of Canine Crown, Canine Root. However, the feature points of the present invention are not limited to the feature points of Figs. 4 to 6.

[0058] The step (step S200) of generating the two-dimensional panoramic image (2PI) from the three-dimensional volume data (3VD) based on the above-described feature points may generate a spline curve (SL) based on the feature points. For example, the spline curve (SL) may be a curve connecting the feature points (or points generated based on the feature points). Therefore, the number of the feature points may be 3 or more, and the spline curve (SL) may vary depending on the feature points. Fig. 7 shows a spline curve and a two-dimensional panoramic image (2PI) generated based on the Right Porion (PoR), the Anterior Nasal Spine (ANS), and the Left Porion. Fig. 8 shows a spline curve and a two-dimensional panoramic image (2PI) generated based on the Right Mandibular Foramen, the Stomion Superior (Sts), and the Left Mandibular Foramen. The first spline curve (SL1) of the first image, the second image, and the third image of FIG. 9 can be generated based on the Right Mandibular Foramen, the Right Maxillary Central Incisor Crown Point, the Left Maxillary Central Incisor Crown Point, and the Left Mandibular Foramen. The second spline curve (SL2) of the third image of FIG. 9 can be generated by moving the Right Maxillary Central Incisor Crown Point and the Left Maxillary Central Incisor Crown Point to the same height as the Anterior NasalSpine. The fourth image of FIG. 9 represents a two-dimensional panoramic image (2PI) generated based on the second spline curve (SL2) of the third image of FIG. 9.Here, the first spline curve (SL1) and the second spline curve (SL2) can be included in the spline curve (SL).

[0059] As illustrated in FIG. 2, the step (step S200) of generating the two-dimensional panoramic image (2PI) from the three-dimensional volume data (3VD) based on the feature points may generate an extended spline curve (SL_EXP) and a reduced spline curve (SL_RED) based on the spline curve (SL). The extended spline curve (SL_EXP) may be a spline curve located at the anterior part of the body. The extended spline curve (SL_EXP) may be generated based on a first normal vector (n_side1) defined as a normal vector orthogonal to a tangent line of the spline curve (SL). The reduced spline curve (SL_RED) may be a spline curve located at the posterior part of the body. The above-described reduced spline curve (SL_RED) can be generated based on a second normal vector (n_side2) defined as a normal vector orthogonal to a tangent line of the above-described spline curve (SL). For example, the first normal vector (n_side1) and the second normal vector (n_side2) can point in opposite directions.

[0060] The area between the extended spline curve (SL_EXP) and the reduced spline curve (SL_RED) can be defined as a reference area. The extended spline curve (SL_EXP) and the reduced spline curve (SL_RED) defining the reference area can be generated so that the tooth (TT_2PI) is included in the reference area. Therefore, the two-dimensional panoramic image (2PI) can be generated using voxels included in the reference area. The voxels of the reference area can include the tooth (TT_2PI). For example, the two-dimensional panoramic image (2PI) can be generated using a DRR (Digitally Reconstructed Radiograph) generation method. The DDR generation method may be a method of generating a two-dimensional radiographic image generated by digitally reconstructing a three-dimensional image.

[0061] FIGS. 10 and 11 are diagrams showing a step (step S300) of generating a two-dimensional bounding box (2BB) corresponding to a tooth (TT_2PI) in the two-dimensional panoramic image (2PI) of FIG. 1. FIGS. 12 and 13 are diagrams explaining the two-dimensional bounding box (2BB) of FIG. 11. FIG. 14 is a diagram showing the two-dimensional bounding box (2BB) of FIG. 11 reflected in the two-dimensional panoramic image (2PI) of FIG. 1. FIG. 15 is a diagram showing a panoramic x-ray image (2PI_X-RAY).

[0062] Referring to FIGS. 1 to 15, the method for automatically separating teeth from the three-dimensional volume data (3VD) may include the step (step S300) of generating the two-dimensional bounding box (2BB) corresponding to the tooth (TT_2PI) in the two-dimensional panoramic image (2PI).

[0063] The step (step S300) of generating the two-dimensional bounding box (2BB) corresponding to the tooth (TT_2PI) in the two-dimensional panoramic image (2PI) can be automatically processed by the deep learning. For example, the step (step S300) of generating the two-dimensional bounding box (2BB) corresponding to the tooth (TT_2PI) in the two-dimensional panoramic image (2PI) can be performed using a second artificial intelligence neural network. For example, the two-dimensional panoramic image (2PI) can be input to the second artificial intelligence neural network, and a two-dimensional segmentation mask (SM) corresponding to the tooth (TT_2PI) or the two-dimensional bounding box (2BB) can be output. When the two-dimensional segmentation mask (SM) is output by the second artificial intelligence neural network, the two-dimensional bounding box (2BB) can be obtained by post-processing the output two-dimensional segmentation mask (SM). For example, the second artificial intelligence neural network may be a convolutional neural network (CNN).

[0064] The teeth (TT_2PI) included in the above two-dimensional panoramic image (2PI) can be separated by a two-dimensional segmentation mask (SM). The two-dimensional segmentation mask (SM) can be used to distinguish the teeth (TT_2PI) included in the two-dimensional panoramic image (2PI) from each other. The number of the two-dimensional segmentation masks (SM) can be a preset value. Generally, the number of teeth of a patient including all four wisdom teeth is at most 32. Therefore, the number of the two-dimensional segmentation masks (SM) can be set to at most 32.

[0065] Let us assume that the number of the teeth (TT_2PI) is 32 and the number of the two-dimensional segmentation masks (SM) is 32. In one embodiment, each segmentation mask (SM) may include one tooth (TT_2PI). In this case, each tooth (TT_2PI) may have a different classification number. That is, the classification number of the tooth (TT_2PI) may be from 1 to 32.

[0066] However, in this case, the above-described computational processing operation may require a lot of time and a lot of computer resources. To solve this problem, the number of classification numbers of the teeth (TT_2PI) can be reduced. Specifically, the patient teeth may include an upper tooth group located in the maxilla and a lower tooth group located in the mandible, and the patient teeth may be pairwise symmetrical on the left and right. (For example, the patient teeth may be arranged in the following order: central incisors, lateral incisors, canines, first premolars, second premolars, first molars, second molars, and third molars with the center as the standard for each of the upper tooth group and the lower tooth group.) Therefore, by having the same pairs have the same classification number, the number of classification numbers of the teeth (TT_2PI) can be reduced. That is, the classification number of the tooth (TT_2PI) may have 8 numbers for the upper tooth group and 8 numbers for the lower tooth group. Teeth (TT_2PI) having the same classification number can be separated by the same two-dimensional segmentation mask (SM). Even if the two-dimensional segmentation mask (SM) includes teeth (TT_2PI) having the same classification number, the left and right pairs can be easily separated by post-processing using an image processing technique such as connected component labeling.

[0067] However, even in this case, the front teeth (i.e., the central incisors) are attached to each other, so separation through post-processing may be difficult. In such cases, the problem can be solved by assigning different classification numbers to the teeth included in the central incisors. Accordingly, the classification number of the tooth (TT_2PI) may have 9 numbers for the upper tooth group and 9 numbers for the lower tooth group.

[0068] In this way, the two-dimensional segmentation mask (SM) may include one or more teeth (TT_2PI) having the same classification number. In addition, when the number of the two-dimensional segmentation masks (SM) is greater than the number of teeth present in the patient, there may be a two-dimensional segmentation mask (SM) that does not include the tooth (TT_2PI). In addition, teeth (TT_2PI) having the same classification number may be separated by the same two-dimensional segmentation mask (SM) and may have the same brightness value. On the other hand, teeth (TT_2PI) having different classification numbers may be separated by different two-dimensional segmentation masks (SM) and may have different brightness values.

[0069] Meanwhile, when the number of the two-dimensional segmentation masks (SM) is 1, each tooth (TT_2PI) included in the two-dimensional segmentation masks (SM) may have a different brightness value.

[0070] In the above description, it was assumed that the number of teeth (TT_2PI) was 32. The number of teeth (TT_2PI) of the patient is not limited to this. The number of teeth (TT_2PI) of the patient may be less than 32. Alternatively, the number of teeth (TT_2PI) of the patient may be more than 32.

[0071] The above two-dimensional segmentation mask (SM) can be post-processed to generate the above two-dimensional bounding box (2BB). The number of the above two-dimensional bounding boxes (2BB) can correspond to the number of the above two-dimensional segmentation mask (SM).

[0072] The above two-dimensional bounding box (2BB) can surround the tooth (TT_2PI). For example, the ROI (Region Of Interest) of the two-dimensional bounding box (2BB) can be set to a size that fits the tooth (TT_2PI). Therefore, the size of the two-dimensional bounding box (2BB) can vary depending on the tooth (TT_2PI).

[0073] The above two-dimensional bounding box (2BB) can be defined by parameters. For example, the parameters defining the above two-dimensional bounding box (2BB) are c ci , x ui , y ui , x li , y li can be. Here, i is a positive integer greater than or equal to 1 and less than or equal to N. The above c ci represents the classification number of the above tooth (TT_2PI), and the above c ci is c c1 , c c2 , ..., c cN-1 , c cN may include the above x ui , y ui represents the coordinates of the upper corner of the first side (e.g., left) of the two-dimensional bounding box (2BB), and x ui is x u1 , x u2 , ..., x uN-1 , x uN may include, and the above y ui is y u1 , y u2 , ..., y uN-1 , y uN may include the above x li , yli represents the coordinates of the lower corner of the second side (e.g., right) of the two-dimensional bounding box (2BB), and x li is x l1 , x l2 , ..., x lN-1 , x lN may include, and the above y li is y l1 , y l2 , ..., y lN-1 , y lN may include. For example, the parameters defining the two-dimensional bounding box (2BB) are c ci , x ci , y ci , h i , w i is a parameter. Here, i is a positive integer greater than or equal to 1 and less than or equal to N. The above c ci represents the classification number of the above tooth (TT_2PI), and the above c ci is c c1 , c c2 , ..., c cN-1 , c cN may include the above x ci , y ci represents the coordinates of the center point of the above two-dimensional bounding box (2BB), and x ci is x c1 , x c2 , ..., x cN-1 , x cN may include, and the above y ci is y c1 , y c2 , ..., y cN-1 , y cN may include the above h i represents the upper and lower width of the two-dimensional bounding box (2BB), and w i represents the left and right width of the above two-dimensional bounding box (2BB), and h i are h1, h2, ..., h N-1 , h Nmay include, and the above w i are w1, w2, ..., w N-1 , w N may include.

[0074] Meanwhile, the two-dimensional panoramic image (2PI) may be an image indirectly generated from the three-dimensional volume data (3VD), or the two-dimensional panoramic image (2PI) may be a panoramic x-ray image (2PI_X-RAY) directly acquired by a panoramic x-ray device. Since the panoramic x-ray image (2PI_X-RAY) directly acquired by the panoramic x-ray device is similar to the image indirectly generated from the three-dimensional volume data (3VD), the second artificial intelligence neural network can be trained based on not only the image indirectly generated from the three-dimensional volume data (3VD) but also the panoramic x-ray image (2PI_X-RAY) directly acquired by the panoramic x-ray device. Alternatively, the second artificial intelligence neural network may be trained based on the panoramic x-ray image (2PI_X-RAY) directly acquired by the panoramic x-ray device and then trained based on the image indirectly generated from the three-dimensional volume data (3VD), thereby enabling transfer learning of the second artificial intelligence neural network. Alternatively, the second artificial intelligence neural network may be trained based on the image indirectly generated from the three-dimensional volume data (3VD) and then trained based on the panoramic x-ray image (2PI_X-RAY) directly acquired by the panoramic x-ray device, thereby enabling transfer learning of the second artificial intelligence neural network.

[0075] FIG. 16 is a diagram showing a step (step S400) of generating a 3D bounding box (3BB) from 3D volume data (3VD) based on the 2D bounding box (2BB) of FIG. 1 and using the same to generate 3D sub-volume data (3VD_SUB).

[0076] Referring to FIGS. 1 to 16, the method for automatically separating teeth from the 3D volume data (3VD) may include the step (step S400) of generating the 3D bounding box (3BB) and the 3D sub-volume data (3VD_SUB) from the 3D volume data (3VD) based on the 2D bounding box (2BB).

[0077] The operation of generating the 3D bounding box (3BB) from the 3D volume data (3VD) based on the 2D bounding box (2BB) may utilize the inverse transformation of the operation of generating the 2D panoramic image (2PI) from the 3D volume data (3VD). Here, the 3D sub-volume data (3VD_SUB) may be generated from the 3D volume data (3VD) based on the 3D bounding box (3BB), and thus, the 3D sub-volume data (3VD_SUB) may be a part of the 3D volume data (3VD). The 3D volume data (3VD) may include the 3D bounding box (3BB).

[0078] The above 3D sub-volume data (3VD_SUB) may include the tooth (TT_3VD_SUB). Each 3D bounding box (3BB) may include one tooth (TT_3VD_SUB), but in some cases, each 3D bounding box (3BB) may not include one tooth (TT_3VD_SUB). For example, the number of teeth (TT_3VD_SUB) included in the 3D bounding box (3BB) may be 2 or more.

[0079] Fig. 17 is a diagram showing a step (step S500) of separating a tooth (TT_3VD_SUB) from 3D sub-volume data generated from 3D volume data (3VD) based on a 3D bounding box (3BB) of Fig. 1. Fig. 18 is a diagram showing a tooth (TT_3VD_SUB') separated from the 3D sub-volume data (3VD_SUB) of Fig. 1. Fig. 19 is a diagram showing a step (step S600) of rendering teeth (TT_3VD_SIB') separated from all 3D sub-volume data (3VD_SUB) of Fig. 1 with 3D volume data (3VD).

[0080] Referring to FIGS. 1 to 19, the method for automatically separating teeth from the 3D volume data (3VD) may include the step (step S500) of separating the teeth (TT_3VD_SUB) from the 3D sub-volume data (3VD_SUB) generated from the 3D volume data (3VD) based on the 3D bounding box (3BB), and the step (step S600) of rendering the teeth (TT_3VD_SUB') separated from the 3D volume data (3VD).

[0081] The step (step S500) of separating the tooth (TT_3VD_SUB) from the 3D sub-volume data (3VD_SUB) based on the 3D bounding box (3BB) can be automatically processed by the deep learning. For example, the step (step S500) of separating the tooth (TT_3VD_SUB) from the 3D sub-volume data (3VD_SUB) generated from the 3D volume data (3VD) based on the 3D bounding box (3BB) can be performed using a third artificial intelligence neural network. That is, the 3D sub-volume data (3VD_SUB) can be input to the third artificial intelligence neural network, and a 3D volume segmentation mask image of the tooth (TT_3VD_SUB') separated from the 3D sub-volume data (3VD_SUB) can be output. Here, the inside and outside of the separated tooth (TT_3VD_SUB') can be distinguished based on the three-dimensional volume segmentation mask image of the separated tooth (TT_3VD_SUB'). For example, the third artificial intelligence neural network may be the convolutional neural network, and the convolutional neural network may be a two-dimensional convolutional neural network or a three-dimensional convolutional neural network.

[0082] In order to improve the result accuracy of the separated tooth (TT_3VD_SUB'), the separated tooth (TT_3VD_SUB') can be comprehensively learned based on additional data. The additional data may be data of the tooth (TT_3VD_SUB') that can be obtained without any additional work. For example, the additional data may be the tooth axis of the tooth (TT_3VD_SUB'), the classification number of the tooth (TT_3VD_SUB'), distance map data that obtains a 3D distance from the periphery to the center of the separated tooth (TT_3VD_SUB') from the 3D sub-volume data (3VD_SUB), a background area remaining from the 3D sub-volume data (3VD_SUB) excluding the separated tooth (TT_3VD_SUB'), etc. The separated tooth (TT_3VD_SUB') from the 3D sub-volume data (3VD_SUB) may undergo a rendering process. Fig. 18 shows the result of rendering the separated tooth (TT_3VD_SUB') from the 3D sub-volume data (3VD_SUB). Fig. 19 shows the result of superimposing the 3D volume segmentation mask image onto the 3D volume data (3VD) of Fig. 2.

[0083] The method for automatically segmenting teeth from the above-described 3D volume data (3VD) may include the step (step S700) of converting the separated teeth (TT_3VD_SUB') into the mesh data. For example, the 3D volume segmentation mask image may be converted into the mesh data using a Marching cube algorithm. The mesh data may include 3D points (Vertex) and a triangular surface (Triangle) or a rectangular surface (Rectangle) created by connecting the points.

[0084] According to the present embodiment, the feature points can be determined from the 3D volume data (3VD), the 2D panoramic image (2PI) is generated from the 3D volume data (3VD) based on the feature points, the 2D bounding box (2BB) corresponding to the tooth (TT_2PI) can be generated from the 2D panoramic image (2PI), the 3D bounding box (3BB) is generated from the 3D volume data (3VD) based on the 2D bounding box (2BB), the 3D sub-volume data (3VD_SUB) is generated from the 3D volume data (3VD) based on the 3D bounding box (3BB), and the tooth (TT_3VD_SUB) can be separated from the sub-volume data (3VD_SUB). Accordingly, the operation processing operation can require less time and be accurate.

[0085] In addition, at least one of the step (step S100) of determining the feature points from the 3D volume data (3VD), the step (step S300) of generating the 2D bounding box (2BB) corresponding to the tooth (TT_2PI) from the 2D panoramic image (2PI), and the step (step S500) of separating the tooth (TT_3VD_SUB) from the 3D sub-volume data (3VD_SUB) generated based on the 3D bounding box (3BB) from the 3D volume data (3VD) is performed using an artificial intelligence neural network, so the computational processing operation can require less time and be accurate.

[0086] According to one embodiment of the present invention, a computer-readable recording medium having recorded thereon a program for executing a method for automatically separating teeth from three-dimensional volume data (3VD) according to the above embodiments on a computer may be provided. The above-described method can be written as a program that can be executed on a computer, and can be implemented on a general-purpose digital computer that executes the program using the computer-readable medium. In addition, the structure of data used in the above-described method can be recorded on a computer-readable medium through various means. The computer-readable medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention, or may be known and usable by those skilled in the art in the field of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language codes, such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention.

[0087] Additionally, the above-described method for automatically separating teeth from three-dimensional volume data (3VD) can also be implemented in the form of a computer program or application executed by a computer and stored in a recording medium.

[0088] The present invention relates to a method for automatically separating teeth from three-dimensional volume data and a computer-readable recording medium having recorded thereon a program for executing the method on a computer, which can reduce the effort and time for computational processing operations and improve accuracy and productivity.

[0089] Although the present invention has been described above with reference to preferred embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the claims below.

Claims

1. A step for determining feature points in 3D volume data; A step of generating a two-dimensional panoramic image from the three-dimensional volume data based on the above feature points; A step of generating a two-dimensional bounding box corresponding to a tooth in the two-dimensional panoramic image; A step of generating a 3D bounding box and 3D sub-volume data from the 3D volume data based on the 2D bounding box; and A method for automatically separating teeth from 3D volume data, comprising a step of separating the teeth from the 3D sub-volume data generated based on a 3D bounding box from the 3D volume data.

2. A method for automatically separating teeth from 3D volume data, characterized in that the number of feature points in the first paragraph is 3 or more.

3. A method for automatically separating teeth using 3D volume data, characterized in that in the first paragraph, the 3D volume data is input into the first artificial intelligence neural network and the feature points are output.

4. A method for automatically separating teeth from 3D volume data, characterized in that in the first paragraph, a spline curve is generated based on the feature points, and the 2D panoramic image is generated based on the spline curve.

5. In paragraph 4, An extended spline curve is generated based on the first normal vector of the spline curve, and a reduced spline curve is generated based on the second normal vector of the spline curve. A method for automatically separating teeth from three-dimensional volume data, characterized in that the two-dimensional panoramic image is generated using voxels included in a reference area between the expanded spline curve and the reduced spline curve.

6. A method for automatically separating teeth from three-dimensional volume data, characterized in that the voxels of the reference area in the fifth paragraph include the teeth.

7. A method for automatically separating teeth from three-dimensional volume data, characterized in that the teeth included in the two-dimensional panoramic image in the first paragraph are separated using a two-dimensional segmentation mask.

8. A method for automatically separating teeth from 3D volume data, characterized in that in the 7th paragraph, the number of the 2D segmentation masks is 1, and each tooth included in the 2D segmentation mask has a different brightness value.

9. A method for automatically segmenting teeth of 3D volume data, characterized in that, in the 7th paragraph, teeth having the same classification number among the teeth are segmented with the same 2D segmentation mask, and teeth having different classification numbers among the teeth are segmented with different 2D segmentation masks.

10. A method for automatically separating teeth from three-dimensional volume data, characterized in that, in the 7th paragraph, the teeth having the same classification number among the teeth are symmetrical in pairs on the left and right.

11. A method for automatically separating teeth from three-dimensional volume data, characterized in that in the first paragraph, the two-dimensional bounding box surrounds the teeth.

12. In the first paragraph, the parameter defining the two-dimensional bounding box is c ci , x ui , y ui , x li , y li (where i is a positive integer greater than or equal to 1 and less than or equal to N), c above ci represents the classification number of the above tooth, and the above x ui , y ui represents the coordinates of the upper corner of the first side of the two-dimensional bounding box, and x li , y li A method for automatically separating teeth from three-dimensional volume data, characterized in that it represents the coordinates of the lower edge of the second side of the two-dimensional bounding box.

13. In the first paragraph, the parameter defining the two-dimensional bounding box is c ci , x ci , y ci , h i , w i (where i is a positive integer greater than or equal to 1 and less than or equal to N), c above ci represents the classification number of the above tooth, and the above x ci , y ci represents the coordinates of the center point of the above two-dimensional bounding box, and h i represents the upper and lower width of the above two-dimensional bounding box, and w i A method for automatically separating teeth from three-dimensional volume data, characterized in that it represents the left and right width of the two-dimensional bounding box.

14. A method for automatically segmenting teeth from 3D volume data, characterized in that in the first paragraph, the 2D panoramic image is input into the second artificial intelligence neural network and a 2D segmentation mask or the 2D bounding box corresponding to the teeth is output.

15. A method for automatically separating teeth from 3D volume data, characterized in that, in the first paragraph, the operation of generating the 3D bounding box from the 3D volume data based on the 2D bounding box uses an inverse transformation of the operation of generating the 2D panoramic image from the 3D volume data.

16. A method for determining a target tooth, characterized in that, in the first paragraph, 3D sub-volume data generated based on the 3D bounding box from the 3D volume data is input to the third artificial intelligence neural network, and a 3D volume segmentation mask image of a tooth separated from the 3D sub-volume data is output.

17. A method for automatically separating teeth from 3D volume data, characterized in that the method further comprises a step of rendering the separated teeth from the 3D volume data in the first paragraph.

18. A method for automatically separating teeth from 3D volume data, characterized in that the method further comprises a step of converting the teeth separated from the 3D volume data into mesh data in the first paragraph.

19. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 18 on a computer.

Images