Method for automatically classifying prosthesis type from three-dimensional oral data, and computer-readable recording medium having program recorded thereon for executing same method on computer.

Abstract

A method for automatically classifying a prosthesis type from three-dimensional oral data comprises the steps of: aligning three-dimensional oral data including teeth; extracting, from the aligned three-dimensional oral data, features for determining a prosthesis type to be used for the teeth; combining the aligned three-dimensional oral data with feature data including the features; and classifying the prosthesis type to be used for the teeth on the basis of the aligned three-dimensional oral data and the feature data.

Description

A method for automatically classifying prosthesis types from three-dimensional oral data and a computer-readable recording medium having recorded thereon a program for executing the method on a computer

[0001] The present invention relates to a method for automatically classifying prosthesis types from three-dimensional oral data and a computer-readable recording medium having recorded thereon a program for executing the method on a computer. More specifically, the present invention relates to a method for automatically classifying prosthesis types from three-dimensional oral data using deep learning and a computer-readable recording medium having recorded thereon a program for executing the method on a computer.

[0002] With the advancement of artificial intelligence (AI) technology, research on explainable artificial intelligence (XAI) is actively underway to explain the operating principles of AI neural networks. Among XAI research, research that explains the reasons behind AI model predictions utilizes the layer-wise relevance propagation (LRP) technique that constitutes AI models.

[0003] The inter-layer contribution propagation technique is based on the principle that 'the sum of the contributions of neurons belonging to each layer constituting the AI ​​model is the same for each layer', and by redistributing the predicted values ​​of the AI ​​model based on the weights set for each layer constituting the model, the contribution (relevance score) of each neuron belonging to each layer can be calculated. In other words, the inter-layer contribution propagation technique is a backpropagation method that goes back from the output layer to the input layer, and accordingly, it can relatively identify which part of the input data contributed more to derive the final predicted value of the AI ​​model.

[0004] Meanwhile, 3D oral data refers to data scanned using a 3D scanner, modeling or reconstructing objects such as teeth and the oral cavity. 3D oral data can be used for patient prosthetic or dental treatment. Recently, research is underway to automatically generate and learn data from patient 3D oral data to be used or assisted in prosthetic or dental treatment, utilizing AI models that utilize various techniques, such as interlayer contribution propagation.

[0005] The purpose of the present invention is to provide a method for automatically classifying a prosthesis type from three-dimensional oral data.

[0006] However, the problem to be solved by the present invention is not limited to the problem mentioned above, and may be expanded in various ways without departing from the spirit and scope of the present invention.

[0007] In order to achieve the above-described object of the present invention, a method for automatically classifying a prosthesis type from three-dimensional oral data according to an embodiment of the present invention may include a step of aligning three-dimensional oral data including teeth, a step of extracting a feature for determining a prosthesis type to be used for the teeth from the aligned three-dimensional oral data, a step of combining the aligned three-dimensional oral data and feature data including the feature, and a step of classifying the prosthesis type to be used for the teeth based on the aligned three-dimensional oral data and the feature data.

[0008] In one embodiment, the step of extracting the feature is performed using a first artificial intelligence neural network, and the first artificial intelligence neural network receives the sorted three-dimensional oral data as input and calculates the contribution of each of a plurality of layers included in the first artificial intelligence neural network to extract the feature.

[0009] In one embodiment, the output of the first artificial intelligence neural network may be the feature data in the form of a heat map that highlights a portion corresponding to the feature within the aligned three-dimensional oral data.

[0010] In one embodiment, when the prosthesis type is a screw implant, the portion corresponding to the feature may be a boundary portion of a hole penetrating the tooth in the axial direction.

[0011] In one embodiment, when the prosthesis type is a bridge, the portion corresponding to the feature may be a connecting portion located between the tooth and adjacent surrounding teeth.

[0012] In one embodiment, when the prosthesis type is an inlay, the portion corresponding to the feature may be an occlusal cavity formed in the tooth.

[0013] In one embodiment, the step of classifying the prosthesis type may be performed using a second artificial intelligence neural network, and the input of the second artificial intelligence neural network may be data in which the aligned three-dimensional oral data and the feature data are combined with each other, and the output of the second artificial intelligence neural network may be the prosthesis type.

[0014] In one embodiment, while the step of classifying the prosthesis type is performed, the first artificial intelligence neural network can learn a process of extracting the feature from the aligned three-dimensional oral data, and the second artificial intelligence neural network can learn a process of classifying the prosthesis type based on the aligned three-dimensional oral data and the feature data.

[0015] In one embodiment, the step of combining the aligned three-dimensional oral data and the feature data may include combining tensors included in the aligned three-dimensional oral data and tensors included in the feature data in the channel direction.

[0016] In one embodiment, the step of aligning the three-dimensional oral data may include the step of extracting a region of interest from the three-dimensional oral data and the step of aligning the three-dimensional oral data within the region of interest to generate alignment data.

[0017] In one embodiment, the step of generating the alignment data may include the steps of extracting point information from the three-dimensional oral data within the region of interest, generating a multidimensional tree based on the point information, generating a three-dimensional voxel based on the point information, searching for points adjacent to the three-dimensional voxel using the multidimensional tree, and determining the number of adjacent points as a value of the three-dimensional voxel.

[0018] In one embodiment, in the step of generating a 3D voxel based on the point information, the size of the 3D voxel and the center of the 3D voxel can be determined using the maximum value of the vector coordinates included in the point information, the minimum value of the vector coordinates, and the size of the 3D oral data within the region of interest.

[0019] In one embodiment, in the step of determining the value of the 3D voxel, the distance from the center of the 3D voxel to the adjacent point may be less than 1 / 2 of the size of the 3D voxel.

[0020] In one embodiment, the step of generating the alignment data may include the steps of extracting boundary information and point information from the three-dimensional oral data within the region of interest, generating an implicit mesh model based on the point information, using the implicit mesh model to generate a three-dimensional binary array, and aligning the three-dimensional binary array.

[0021] In one embodiment, the step of generating the three-dimensional binary array may include the step of setting a dimension based on the boundary information and the point information using the implicit mesh model, the step of dividing the three-dimensional oral data within the region of interest into voxels based on the set dimension, the step of storing the shortest distance between each mesh of the three-dimensional oral data within the region of interest included in each of the voxels, the step of generating an image array based on the shortest distance and the dimension, and the step of converting the image array into the three-dimensional binary array through binarization.

[0022] In one embodiment, in the step of aligning the three-dimensional binary array, the implicit mesh model may be used to arrange the three-dimensional binary array at the center of the tensors forming the input size of the implicit mesh model.

[0023] In one embodiment, a program for executing a method for automatically classifying a prosthesis type from three-dimensional oral data on a computer can be recorded on a computer-readable recording medium.

[0024] In a method for automatically classifying a prosthesis type from three-dimensional oral data according to embodiments of the present invention, three-dimensional oral data is input, features of teeth included in the three-dimensional oral data are automatically extracted, and the type of prosthesis to be used for the teeth can be automatically classified. Accordingly, the process of analyzing the condition of individual teeth, which would otherwise be time-consuming if prosthesis classification were performed manually, can be simplified. In addition, objective prosthesis classification is possible using an artificial intelligence neural network, thereby reducing the risk of misclassification of the prosthesis. Therefore, the method for automatically classifying prosthesis types from three-dimensional oral data can efficiently proceed with the prosthesis manufacturing process for patients and provide prompt treatment to patients. In addition, by generating feature data to be used in the dental medical and research fields, the dental medical and research fields can be further developed.

[0025] However, the effects of the present invention are not limited to the effects described above, and may be expanded in various ways without departing from the spirit and scope of the present invention.

[0026] FIG. 1 is a flowchart illustrating a method for automatically classifying a prosthesis type from three-dimensional oral data according to one embodiment of the present invention.

[0027] FIG. 2 is a drawing for explaining a method for automatically classifying a prosthesis type from the three-dimensional oral data of FIG. 1.

[0028] FIG. 3 is a flowchart illustrating an example of a step of generating alignment data by aligning three-dimensional oral data within the region of interest of FIG. 1.

[0029] FIG. 4 is a drawing for explaining a step of generating alignment data by aligning three-dimensional oral data within the region of interest of FIG. 3.

[0030] FIG. 5 is a drawing for explaining an example of a process in which a step of generating feature data by extracting features from the sorting data of FIG. 1 according to the input of the sorting data of FIG. 4 is performed.

[0031] FIG. 6 is a drawing for explaining another example of a process in which a step of generating feature data by extracting features from the alignment data of FIG. 1 according to the input of the alignment data of FIG. 4 is performed.

[0032] FIG. 7 is a drawing for explaining another example of a process in which a step of generating feature data by extracting features from the sorting data of FIG. 1 according to the input of the sorting data of FIG. 4 is performed.

[0033] FIG. 8 is a drawing for explaining a step of combining the alignment data and features of FIG. 1 and a step of classifying the type of prosthesis to be used for a tooth.

[0034] FIG. 9 is a flowchart illustrating another example of a step of generating alignment data by aligning three-dimensional oral data within the region of interest of FIG. 1.

[0035] Fig. 10 is a flowchart showing the steps of forming a three-dimensional binary array using the implicit mesh model of Fig. 9.

[0036] FIG. 11 is a drawing for explaining a step of generating alignment data by aligning three-dimensional oral data within the region of interest of FIG. 9.

[0037] FIG. 12 is a drawing for explaining an example of a process in which a step of generating feature data by extracting features from the alignment data of FIG. 1 according to the input of the alignment data of FIG. 11 is performed.

[0038] FIG. 13 is a drawing for explaining another example of a process in which a step of generating feature data by extracting features from the alignment data of FIG. 1 according to the input of the alignment data of FIG. 11 is performed.

[0039] FIG. 14 is a diagram for explaining another example of a process in which a step of generating feature data by extracting features from the alignment data of FIG. 1 according to the input of the alignment data of FIG. 11 is performed.

[0040] With respect to the embodiments of the present invention disclosed in the text, specific structural and functional descriptions are merely illustrative for the purpose of explaining the embodiments of the present invention, and the embodiments of the present invention may be implemented in various forms and should not be construed as being limited to the embodiments described in the text.

[0041] The present invention is susceptible to various modifications and takes various forms. Specific embodiments are illustrated in the drawings and described in detail herein. However, this is not intended to limit the present invention to specific disclosed forms, but rather to encompass all modifications, equivalents, and alternatives falling within the spirit and technical scope of the present invention.

[0042] While terms like "first" and "second" may be used to describe various components, these components should not be limited by these terms. These terms may be used to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0043] When a component is referred to as being "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. Conversely, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between. Other expressions that describe the relationship between components, such as "between" and "directly between" or "adjacent to" and "directly adjacent to", should be interpreted similarly.

[0044] The terminology used in this application is only used to describe specific embodiments and is not intended to limit the present invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, it should be understood that the terms "comprise" or "have" indicate the presence of a described feature, number, step, operation, component, part, or combination thereof, but do not preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0045] Unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which this invention pertains. Terms defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and shall not be construed in an idealized or overly formal sense unless explicitly defined herein.

[0046] Meanwhile, if a particular embodiment can be implemented differently, the functions or operations specified within a particular block may occur in a different order than specified in the flowchart. For example, two consecutive blocks may actually be executed substantially simultaneously, or, depending on the related functions or operations, the blocks may be executed in reverse order.

[0047] Hereinafter, preferred embodiments of the present invention will be described in more detail with reference to the attached drawings. Identical components in the drawings are designated by the same reference numerals, and redundant descriptions of identical components are omitted.

[0048] FIG. 1 is a flowchart illustrating a method for automatically classifying a prosthesis type from three-dimensional oral data according to one embodiment of the present invention. FIG. 2 is a diagram for explaining a method for automatically classifying a prosthesis type from the three-dimensional oral data of FIG. 1. FIG. 3 is a flowchart illustrating an example of a step of aligning three-dimensional oral data within a region of interest of FIG. 1 to generate alignment data. FIG. 4 is a diagram for explaining a step of aligning three-dimensional oral data within a region of interest of FIG. 3 to generate alignment data. FIG. 5 is a diagram for explaining an example of a process in which a step of extracting features from the alignment data of FIG. 1 and generating feature data is performed according to input of the alignment data of FIG. 4. FIG. 6 is a diagram for explaining another example of a process in which a step of extracting features from the alignment data of FIG. 1 and generating feature data is performed according to input of the alignment data of FIG. 4. FIG. 7 is a diagram for explaining another example of a process in which a step of extracting features from the alignment data of FIG. 1 and generating feature data is performed according to input of the alignment data of FIG. 4. FIG. 8 is a drawing for explaining a step of combining the alignment data and feature data of FIG. 1 and a step of classifying the type of prosthesis to be used for a tooth.

[0049] Referring to FIGS. 1 to 8, a method for automatically classifying a prosthesis type from three-dimensional oral data according to an embodiment of the present invention may include a step of extracting a region of interest from three-dimensional oral data (DAT1) (S100), a step of aligning three-dimensional oral data (hereinafter, interest region data (DAT2)) within a region of interest to generate alignment data (DAT3) (S200), a step of extracting features from the alignment data (DAT3) to generate feature data (DAT4) (S300), a step of combining the alignment data (DAT3) and the feature data (DAT4) (S400), and a step of classifying the type of prosthesis to be used on a tooth (S500).

[0050] A method for automatically classifying a prosthesis type from the above three-dimensional oral data can be performed by a computing device.

[0051] In one embodiment, the three-dimensional oral data (DAT1) may be data obtained by scanning the patient's oral cavity using a three-dimensional scanner. In one embodiment, the three-dimensional oral data (DAT1) may include a dental arch including the maxilla and the mandible, and at least one tooth arranged according to the shape of the dental arch. For example, the shape of the dental arch may have an arch shape, and the teeth may include natural teeth and teeth prepared for treatment. In addition, the three-dimensional oral data (DAT1) may include an abutment artificially transplanted for an implant. However, the type and dimension of the three-dimensional oral data (DAT1) according to embodiments of the present invention are not limited thereto, and the three-dimensional oral data (DAT1) may also include various types of medical image data for treatment, such as CT and MRI.

[0052] In one embodiment, the 3D oral cavity data (DAT1) may be 3D mesh data. For example, the 3D oral cavity data (DAT1) refers to data obtained by scanning teeth and the oral cavity, or an object modeled or reconstructed thereof, using a 3D scanner. For example, the 3D oral cavity data (DAT1) may be mesh data including 3D points (Vertex) and triangular or rectangular faces (Rectangle) created by connecting the points. The 3D oral cavity data (DAT1) may be image data captured using a 3D scanner. There is no limitation on the file extension of the 3D oral cavity scan data, and may be, for example, one of ply, obj, and stl. However, the form of the 3D oral cavity data (DAT1) according to embodiments of the present invention is not limited thereto, and may be collected data having various forms such as a tensor or an array.

[0053] In the step (S100) of extracting a region of interest from the three-dimensional oral data (DAT1), the region of interest may be a region from which a portion of the three-dimensional oral data (DAT1) is extracted. For example, the region of interest may be a region including a tooth included in the three-dimensional oral data (DAT1). In one embodiment, one region of interest may include one tooth included in the three-dimensional oral data (DAT1). In one embodiment, one region of interest may include a plurality of teeth included in the three-dimensional oral data (DAT1). In one embodiment, the region of interest may further include the tooth or a portion of the dental arch connected to the teeth. However, the region of interest according to embodiments of the present invention is not limited thereto, and the region of interest may be an area from which an area necessary for performing subsequent steps (a step of aligning region of interest data (DAT2) to generate alignment data (DAT3) (S200), a step of extracting features from the alignment data (DAT3) to generate feature data (DAT4) (S300), a step of combining the alignment data (DAT3) and the feature data (DAT4) (S400), and a step of classifying the type of prosthesis to be used for the tooth (S500)) is extracted among the areas in the three-dimensional oral data (DAT1).

[0054] In one embodiment, the step (S100) of extracting a region of interest from the three-dimensional oral data (DAT1) may be automatically performed by an artificial intelligence neural network. In one embodiment, the step (S100) of extracting a region of interest from the three-dimensional oral data (DAT1) may be artificially performed by a user. In one embodiment, the step (S100) of extracting a region of interest from the three-dimensional oral data (DAT1) may be performed multiple times, such that the process of extracting a region within the three-dimensional oral data (DAT1) such that at least one tooth is included in the region of interest. After the region of interest is extracted from the three-dimensional oral data (DAT1), region of interest data (DAT2) may be generated.

[0055] In the step (S200) of aligning region of interest data (DAT2) to generate alignment data (DAT3), the region of interest data (DAT2) may be converted into a data format for performing the step (S300) of extracting features from the alignment data (DAT3) to generate feature data (DAT4). For example, in the step (S200) of aligning region of interest data (DAT2) to generate alignment data (DAT3), a data alignment process such as axis transformation may be performed on the region of interest data (DAT2) to generate alignment data (DAT3).

[0056] The step (S200) of generating sorted data (DAT3) by aligning region of interest data (DAT2) may include the steps of extracting point information from region of interest data (DAT2) (S210), generating a multidimensional tree based on the point information (S220), generating a three-dimensional voxel based on the point information (S230), searching for points adjacent to the three-dimensional voxel using the multidimensional tree (S240), and determining the number of adjacent points as a value of the three-dimensional voxel (S250).

[0057] In the step (S210) of extracting point information from the region of interest data (DAT2), point information of each object included in the region of interest data (DAT2) can be extracted. For example, point information of each of the 3D meshes included in the region of interest data (DAT2) can be extracted. In the step (S220) of generating a multidimensional tree based on the point information, the extracted point information can be used to map a space for multidimensional search. In one embodiment, the step (S220) of generating a multidimensional tree based on the point information can be a step of constructing a KD tree using the point information.

[0058] In the step S230 of generating the three-dimensional voxel based on the point information, the three-dimensional voxel may be a single cell corresponding to the minimum unit for dividing the region of interest data (DAT2) into a three-dimensional grid form. In one embodiment, in the step S230 of generating the three-dimensional voxel based on the point information, the arrangement and size of the three-dimensional voxel may be determined using the point information. For example, in the step S230 of generating the three-dimensional voxel based on the point information, the size of the three-dimensional voxel and the center of the three-dimensional voxel may be determined using the maximum value of the vector coordinates included in the point information, the minimum value of the vector coordinates, and the size of the region of interest data (DAT2). Specifically, the maximum value of the vector coordinates may include the maximum value of the coordinates corresponding to each of three axes (e.g., the x-axis, the y-axis, and the z-axis), and the minimum value of the vector coordinates may include the minimum value of the coordinates corresponding to each of the three axes. After the size of the above 3D voxel and the center of the above 3D voxel are determined, a plurality of 3D voxels that partition the entire region of interest data (DAT2) can be generated.

[0059] In one embodiment, the step of generating a multidimensional tree based on the point information (S220) and the step of generating a three-dimensional voxel based on the point information (S230) may be performed simultaneously. However, the order in which the step of generating a multidimensional tree based on the point information (S220) and the step of generating a three-dimensional voxel based on the point information (S230) according to embodiments of the present invention are performed is not limited thereto, and the multidimensional tree may be generated before or after the generation of the three-dimensional voxel.

[0060] In the step (S240) of searching for points adjacent to the 3D voxel using the multidimensional tree, the points (e.g., the adjacent points) that satisfy a criterion set based on one 3D voxel may be searched for using a specific algorithm. In one embodiment, in the step (S240) of searching for points adjacent to the 3D voxel using the multidimensional tree, the distance from the center of the 3D voxel to the adjacent point may be less than half the size of the 3D voxel. Specifically, the points that satisfy the set criterion may be located at a distance from the center of the 3D voxel that is less than half the size of the 3D voxel. Thereafter, in the step (S250) of determining the number of adjacent points as a value of the 3D voxel, the number of the searched points may be designated as a value of the one 3D voxel that serves as a reference.

[0061] In the step (S240) of searching for points adjacent to the 3D voxel using the multidimensional tree and the step (S250) of determining the number of adjacent points as the value of the 3D voxel, the process of searching for the number of adjacent points and specifying the value of the 3D voxel may be repeated until a value is assigned to each of all 3D voxels. The alignment data (DAT3) may be finally generated using the value assigned to the 3D voxel. In other words, the alignment data (DAT3) generated through the step (S200) of generating the alignment data (DAT3) by aligning the region of interest data (DAT2) of FIG. 3 may be aligned data based on a grid (e.g., the 3D voxel). However, the type of the alignment data (DAT3) according to embodiments of the present invention is not limited thereto and may have various types of data.

[0062] The step (S300) of extracting features from the alignment data (DAT3) to generate feature data (DAT4) may be performed using a first artificial intelligence neural network (100). The input of the first artificial intelligence neural network (100) may be the alignment data (DAT3). The output of the first artificial intelligence neural network (100) may be feature data (DAT4). The alignment data (DAT3) may be aligned three-dimensional oral data (DAT1), and the feature data (DAT4) may be data including the features extracted from the alignment data (DAT3).

[0063] In one embodiment, the first artificial intelligence neural network (100) may be an artificial intelligence neural network based on deep learning that utilizes the layer-wise relevance propagation (LRP) technique. For example, the first artificial intelligence neural network (100) may receive alignment data (DAT3) as input and extract the features by calculating the contributions of each of the multiple layers included in the first artificial intelligence neural network (100).

[0064] In one embodiment, the feature data (DAT4), which is the output of the first artificial intelligence neural network (100), may be in the form of a heat map. For example, the feature data (DAT4) may highlight a portion corresponding to the feature within the alignment data (DAT3). However, the feature data (DAT4) according to embodiments of the present invention may have various forms other than a heat map. The feature may include data or values ​​for determining the type of prosthesis to be used for the tooth included in the three-dimensional oral data (DAT1). In addition, the portion corresponding to the feature may be a portion that is emphasized differently depending on the type of prosthesis.

[0065] In one embodiment, when the type of prosthesis to be used on the tooth is a screw implant as illustrated in FIG. 5, the portion corresponding to the feature may be a boundary portion of a hole penetrating the tooth in the axial direction. In one embodiment, when the type of prosthesis to be used on the tooth is a bridge as illustrated in FIG. 6, the portion corresponding to the feature may be a connecting portion located between the tooth and adjacent neighboring teeth. In one embodiment, when the type of prosthesis is an inlay as illustrated in FIG. 7, the portion corresponding to the feature may be an occlusal cavity formed on the tooth. However, the type of the prosthesis according to embodiments of the present invention is not limited thereto, and may include various types including onlays, etc. In addition, the portion corresponding to the feature is not limited thereto, and as the type of the prosthesis has various types, the portion corresponding to the feature may also be various parts of the tooth.

[0066] In the step S400 of combining the alignment data (DAT3) and the feature data (DAT4), the alignment data (DAT3) and the feature data (DAT4) may be combined. In one embodiment, in the step S400 of combining the alignment data (DAT3) and the feature data (DAT4), each of the tensors included in the alignment data (DAT3) and the tensors included in the feature data (DAT4) may be combined in the channel direction. For example, the alignment data (DAT3) may include a three-dimensional tensor including values ​​for each of depth (D), height (H), and width (W). In addition, the feature data (DAT4) may include a three-dimensional tensor including values ​​for each of depth (D), height (H), and width (W). In the step (S400) of combining the sorted data (DAT3) and the feature data (DAT4), each of the sorted data (DAT3) and the feature data (DAT4) may be generated as combined data having a four-dimensional tensor in which the values ​​for depth (D), height (H), and width (W) are overlapped with each other. However, the number of dimensions of the combined data according to embodiments of the present invention is not limited thereto.

[0067] The step (S500) of classifying the type of prosthesis to be used for the tooth may be performed using a second artificial intelligence neural network (200). The input of the second artificial intelligence neural network (200) may be alignment data (DAT3) and the feature, and the output of the second artificial intelligence neural network (200) may be the type of prosthesis. For example, the input of the second artificial intelligence neural network (200) may be the combined data in which alignment data (DAT3) and feature data (DAT4) are combined.

[0068] In one embodiment, the second artificial intelligence neural network (200) may include a classification model. For example, the classification model may include ResNet, Vit, etc. However, the type of the second artificial intelligence neural network (200) according to embodiments of the present invention is not limited thereto.

[0069] The second artificial intelligence neural network (200) can classify the prosthesis type corresponding to the input combined data through an operation process such as convolution using multiple layers. In one embodiment, the prosthesis type corresponding to the output of the second artificial intelligence neural network (200) may include an inlay, an onlay, an implant, a bridge, etc. The screw implant described with reference to FIG. 5 may be classified as an implant. However, the prosthesis type according to embodiments of the present invention is not limited thereto.

[0070] In one embodiment, while the step (S500) of classifying the type of prosthesis to be used for the tooth is performed, the first artificial intelligence neural network (100) can learn the process of extracting the feature from the alignment data (DAT3). In addition, while the step (S500) of classifying the type of prosthesis to be used for the tooth is performed, the second artificial intelligence neural network (200) can learn the process of classifying the type of prosthesis based on the alignment data (DAT3) and the feature. That is, while the step (S500) of classifying the type of prosthesis to be used for the tooth is performed, the second artificial intelligence neural network (200) can learn the process of classifying the type of prosthesis using the combination data.

[0071] As described above, in the method for automatically classifying the type of prosthesis from the three-dimensional oral data, the three-dimensional oral data (DAT1) is input, the features of the teeth included in the three-dimensional oral data (DAT1) are automatically extracted, and the type of prosthesis to be used for the teeth can be automatically classified. Accordingly, the process of analyzing the condition of individual teeth, which would otherwise be time-consuming if the prosthesis classification were performed manually, can be simplified. In addition, objective prosthesis classification is possible using an artificial intelligence neural network, thereby reducing the risk of misclassification of the prosthesis. Therefore, the method for automatically classifying the type of prosthesis from the three-dimensional oral data can efficiently proceed with the prosthesis manufacturing process for patients and provide prompt treatment to patients. In addition, by generating feature data (DAT4) to be used in the dental medical and research fields, the dental medical and research fields can be further developed.

[0072] FIG. 9 is a flowchart illustrating another example of a step of generating alignment data by aligning three-dimensional oral data within a region of interest of FIG. 1. FIG. 10 is a flowchart illustrating a step of forming a three-dimensional binary array using the implicit mesh model of FIG. 9. FIG. 11 is a diagram for explaining a step of generating alignment data by aligning three-dimensional oral data within a region of interest of FIG. 9. FIG. 12 is a diagram for explaining an example of a process in which a step of generating feature data by extracting from the alignment data of FIG. 1 is performed according to input of the alignment data of FIG. 11. FIG. 13 is a diagram for explaining another example of a process in which a step of generating feature data by extracting from the alignment data of FIG. 1 is performed according to input of the alignment data of FIG. 11. FIG. 14 is a diagram for explaining another example of a process in which a step of generating feature data by extracting features from the alignment data of FIG. 1 is performed according to input of the alignment data of FIG. 11.

[0073] The step (S300A) of generating feature data (DAT4a) by extracting features from the alignment data (DAT3a) described with reference to FIGS. 12 to 14 is substantially the same as or similar to the step (S300) of generating feature data (DAT4) by extracting features from the alignment data (DAT3) described with reference to FIGS. 5 to 7, except that the alignment data (DAT3a) is input and the feature data (DAT4a) is output. Hereinafter, any content overlapping with that described with reference to FIGS. 12 to 14 is omitted or simplified.

[0074] Referring to FIGS. 9 to 14, a method for automatically classifying a prosthesis type from the three-dimensional oral data may include a step (S200A) of aligning three-dimensional oral data within a region of interest (hereinafter, region of interest data (DAT2)) to generate alignment data (DAT3a). The step (S200A) of aligning the region of interest data (DAT2) to generate alignment data (DAT3a) may include a step (210A) of extracting point information and boundary information from the region of interest data (DAT2), a step (S220A) of generating an implicit mesh model based on the point information, a step (S230A) of generating a three-dimensional binary array using the implicit mesh model, and a step (S240A) of aligning the three-dimensional binary array.

[0075] The step (S230A) of generating a three-dimensional binary array using the implicit mesh model may include a step (S2310) of setting a dimension based on the boundary information and the point information using the implicit mesh model, a step (S2320) of dividing the region of interest data (DAT2) into voxels based on the set dimension, a step (S2330) of storing the shortest distance between each mesh of the region of interest data (DAT2) included in each of the voxels, a step (S2340) of generating an image array based on the shortest distance and the dimension, and a step (S2350) of converting the image array into the three-dimensional binary array through binarization.

[0076] In one embodiment, the implicit mesh model may be a model that generates a cross-sectional image by cutting the region of interest data (DAT2) of a 3D mesh in one axial direction. In the step (S220A) of generating the implicit mesh model based on the point information, parameters for generating the implicit mesh model may be set. For example, the parameters may be parameters for shell size, padding, etc.

[0077] In the step (S2310) of setting a dimension based on the boundary information and the point information using the implicit mesh model, the dimension can be set by multiplying the input size of the implicit mesh model by the ratio of each axis (e.g., x-axis, y-axis, z-axis). Specifically, the ratio of each axis is set based on the axis with the longest length based on the boundary information of the region of interest data (DAT2), and the dimension can be set by multiplying the ratio of each axis by the input size of the model. In one embodiment, the input size of the implicit mesh model can have a fixed size. For example, the input size can be 64*64*64. However, the process of setting the input size and the dimension is exemplary, and the implicit mesh model according to embodiments of the present invention is not limited thereto.

[0078] In the step (S2320) of dividing the region of interest data (DAT2) into voxels based on the set dimension, the voxels, which are three-dimensional grids for the set dimension based on the boundary information, can be generated. The voxels can divide the region of interest data (DAT2) into multiple regions.

[0079] In the step (S2330) of storing the shortest distance between each mesh of the region of interest data (DAT2) included in each of the voxels, the shortest distance between the boundaries of each mesh forming the region of interest data (DAT2) included in each of the voxels is calculated. If the shortest distance between the boundaries of each mesh is greater than a set maximum value, the set maximum value can be stored in the voxel. For example, the maximum value may be the size of the voxel. However, the process of storing the shortest distance according to embodiments of the present invention is not limited thereto.

[0080] In the step (S2340) of generating the image array based on the shortest distance and the dimension, the image array having a size corresponding to the dimension can be generated. In the step (S2350) of converting the image array into the three-dimensional binary array through binarization, a binarization operation can be performed on the image array to remove values ​​arbitrarily stored for empty voxels among the voxels and express the shape of the mesh.

[0081] In the step (S240A) of aligning the above three-dimensional binary array, the implicit mesh model can be used to arrange the three-dimensional binary array at the center of the tensors forming the input size. Accordingly, the three-dimensional binary array having a variable size can be transformed into the form of a three-dimensional tensor having a fixed size.

[0082] After the step (S240A) of sorting the above three-dimensional binary array is performed, the sorted three-dimensional binary array can be used to generate sorted data (DAT3a). Unlike the process of generating the sorted data (DAT3) of FIG. 4 using a multidimensional tree that generates the sorted data (DAT3a) using the above three-dimensional binary array, since no missing values ​​are generated in specific cells, data can be evenly distributed across cells.

[0083] In the step (S300A) of generating feature data (DAT4a) by extracting features from the alignment data (DAT3a), the feature data (DAT4a) can be generated from the alignment data (DAT3a) based on the three-dimensional binary array. The feature data (DAT4a) can be substantially the same as the feature data (DAT4) of FIGS. 5 to 7. For example, the feature data (DAT4a) can be in the form of a heat map that extracts features from the alignment data (DAT3a) based on the three-dimensional binary array and emphasizes the features.

[0084] According to one embodiment of the present invention, a computer-readable recording medium having recorded thereon a program for executing a method for automatically classifying a prosthesis type from three-dimensional oral data according to the above embodiments on a computer may be provided. The above-described method can be written as a program executable by a computer, and can be implemented in a general-purpose digital computer that executes the program using the computer-readable medium. In addition, the structure of data used in the above-described method can be recorded on a computer-readable medium through various means. The computer-readable medium may include program commands, data files, data structures, etc., either singly or in combination. The program commands recorded on the medium may be those specially designed and configured for the present invention, or may be known and usable by those skilled in the art in the field of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specially configured to store and execute program commands such as ROMs, RAMs, and flash memories. Examples of program instructions include not only machine language codes, such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the present invention.

[0085] Additionally, the method for automatically classifying the prosthesis type from the aforementioned three-dimensional oral data can also be implemented in the form of a computer program or application executed by a computer and stored in a recording medium.

[0086] The present invention relates to a method for automatically classifying a prosthesis type from three-dimensional oral data and a computer-readable recording medium having recorded thereon a program for executing the method on a computer, which can reduce the effort and time required for manufacturing a prosthesis and improve the accuracy and productivity of the prosthesis.

[0087] Although the present invention has been described above with reference to exemplary embodiments thereof, it will be understood by those skilled in the art that various modifications and changes may be made to the present invention without departing from the spirit and scope of the present invention as set forth in the following claims.

Claims

1. A step of aligning 3D oral data including teeth; A step of extracting features for determining the type of prosthesis to be used for the tooth from the aligned three-dimensional oral data; A step of combining the aligned three-dimensional oral data and the feature data including the feature; and A method for automatically classifying a prosthesis type from three-dimensional oral data, comprising a step of classifying the prosthesis type to be used for the tooth based on the aligned three-dimensional oral data and the feature data.

2. In the first paragraph, the step of extracting the feature is performed using a first artificial intelligence neural network, A method for automatically classifying a prosthesis type from 3D oral data, characterized in that the first artificial intelligence neural network receives the sorted 3D oral data as input and extracts the features by calculating the contribution of each of the plurality of layers included in the first artificial intelligence neural network.

3. In the second paragraph, the output of the first artificial intelligence neural network is A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the feature data has a form of a heat map that highlights a part corresponding to the feature within the sorted three-dimensional oral data.

4. In the third paragraph, if the prosthesis type is a screw implant, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the portion corresponding to the above feature is a boundary portion of a hole penetrating the tooth in the axial direction.

5. In the third paragraph, if the prosthesis type is a bridge, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the portion corresponding to the above feature is a connecting portion located between the tooth and adjacent surrounding teeth.

6. In the third paragraph, if the prosthesis type is an inlay, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the part corresponding to the above feature is an occlusal cavity formed in the tooth.

7. In the second paragraph, the step of classifying the prosthesis type is performed using a second artificial intelligence neural network, The input of the second artificial intelligence neural network is data in which the sorted three-dimensional oral data and the feature data are combined with each other. A method for automatically classifying a prosthesis type from three-dimensional oral data, wherein the output of the second artificial intelligence neural network is the prosthesis type.

8. In the 7th paragraph, while the step of classifying the prosthesis type is performed, The above first artificial intelligence neural network learns the process of extracting the features from the aligned three-dimensional oral data, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the second artificial intelligence neural network learns a process of classifying the prosthesis type based on the sorted three-dimensional oral data and the feature data.

9. In the first paragraph, the step of combining the aligned three-dimensional oral data and the feature data is as follows: A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that each of the tensors included in the sorted three-dimensional oral data and the tensors included in the feature data are combined in a channel direction.

10. In the first paragraph, the step of aligning the three-dimensional oral data comprises: A step of extracting a region of interest from the above 3D oral data; and A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that it comprises a step of generating alignment data by aligning the three-dimensional oral data within the region of interest.

11. In the 10th paragraph, the step of generating the sorting data is: A step of extracting point information from the three-dimensional oral data within the region of interest; A step of generating a multidimensional tree based on the above point information; A step of generating a 3D voxel based on the above point information; A step of searching for points adjacent to the 3D voxel using the multidimensional tree; and A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that it comprises a step of determining the number of adjacent points as a value of the three-dimensional voxel.

12. In the step of generating a 3D voxel based on the point information in paragraph 11, A method for automatically classifying a prosthesis type from 3D oral data, characterized in that the size of the 3D voxel and the center of the 3D voxel are determined using the maximum value of the vector coordinates included in the point information, the minimum value of the vector coordinates, and the size of the 3D oral data within the region of interest.

13. In the step of determining the value of the 3D voxel in the 12th paragraph, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the distance from the center of the three-dimensional voxel to the adjacent point is less than half the size of the three-dimensional voxel.

14. In the 10th paragraph, the step of generating the sorting data is: A step of extracting boundary information and point information from the three-dimensional oral data within the region of interest; A step of generating an implicit mesh model based on the above point information; A step of generating a three-dimensional binary array using the above implicit mesh model; and A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that it comprises a step of aligning the three-dimensional binary array.

15. In the 14th paragraph, the step of generating the three-dimensional binary array comprises: A step of setting a dimension based on the boundary information and the point information using the implicit mesh model; A step of dividing the three-dimensional oral data within the region of interest into voxels based on the set dimension; A step of storing the shortest distance between each mesh of the three-dimensional oral data within the region of interest included in each of the voxels; generating an image array based on the shortest distance and the dimension; and A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that it comprises a step of converting the image array into the three-dimensional binary array through binarization.

16. In the step of setting the dimension in paragraph 15, Using the above implicit mesh model, the ratio of each axis is set based on the boundary information, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the above dimension is set by multiplying the input size of the implicit mesh model and the above ratio of each axis.

17. In the step of aligning the three-dimensional binary array in the 16th paragraph, A method for automatically classifying a prosthesis type from three-dimensional oral data, characterized in that the three-dimensional binary array is arranged at the center of tensors forming the input size of the implicit mesh model using the implicit mesh model.

18. A computer-readable recording medium having recorded thereon a program for executing the method of any one of claims 1 to 17 on a computer.

Images