Method, apparatus, and system for recommending customized artificial tooth

Abstract

The present invention relates to an artificial tooth recommendation method implemented by a computing apparatus, and the artificial tooth recommendation method comprises the steps of: receiving image data as an input; generating corresponding mesh data from the input image data; generating feature data from the generated mesh data by using a machine learning-based feature data generation unit; generating type data or size data from the generated feature data by using a machine learning-based type and size data generation unit; and generating recommended artificial tooth appearance data corresponding to the type data or the size data by using a machine learning-based recommended artificial tooth appearance data generation unit.

Description

Methods, devices, and systems for recommending customized artificial teeth

[0001] The present invention relates to a method, device and system for recommending an appropriate artificial tooth for an individual patient using image data obtained by scanning or photographing the face, dental contour and oral arch of a patient in need of selection of an artificial tooth for dental treatment.

[0002] Prosthodontic treatment using dentures can be performed on patients with complete or partial tooth loss. Dentures, artificial teeth, come in a variety of shapes and sizes. The choice of denture shape and size for a patient is determined by the dentist performing the treatment. Therefore, the quality of prosthodontic treatment is heavily dependent on the dentist's experience. To overcome this issue, conventional techniques exist for selecting the appropriate denture type and size using transparent rulers. However, these techniques also suffer from the limitation that the type and size of the denture are ultimately determined by the dentist's subjective judgment. Currently, no objective criteria exist for selecting the appropriate denture type and size for a patient. Therefore, there is a need in the art for a method that provides criteria for selecting the appropriate denture type and size for a patient.

[0003] The present invention aims to provide a technique for objectively selecting an artificial tooth optimized for a patient by using image data obtained by photographing or scanning the patient's face, oral cavity, and teeth.

[0004] According to one aspect of the disclosed invention, there is provided a method for recommending an artificial tooth, which is executed by one or more processors and implemented by a computing device, the method comprising: receiving image data, wherein the image data includes at least one of facial image data, oral image data, and dental image data; generating corresponding mesh data from the input image data, wherein the mesh data includes at least one of facial mesh data, oral mesh data, and dental mesh data; generating feature data from the generated mesh data using a machine learning-based feature data generating unit; generating type data or size data from the generated feature data using a machine learning-based type and size data generating unit, wherein the type data or the size data is data representing the type or size of any one of a face, an oral cavity, and a tooth; and generating recommended artificial tooth appearance data corresponding to the type data or the size data using a machine learning-based recommended artificial tooth appearance data generating unit.

[0005] In addition, the feature data generation unit, the type and size data generation unit, and the recommended artificial tooth appearance data generation unit may be characterized in that they operate based on any one of the facial image data, the oral image data, and the tooth image data.

[0006] Additionally, the above feature data may be characterized by including a plurality of feature points.

[0007] In addition, the step of generating the type data or the size data may include a step of generating feature point attribute data of the plurality of feature points, wherein the feature point attribute data includes ratio, distance, and depth information of the plurality of facial feature points; and a step of generating the type data or the size data based on the feature point attribute data using the type and size data generating unit.

[0008] Additionally, the type data may be characterized by including at least one of facial type information, tooth type information, or arch type information.

[0009] In addition, the feature data generation unit, the type and size data generation unit, and the recommended artificial tooth appearance data generation unit may be configured based on a plurality of machine learning models.

[0010] In addition, the plurality of machine learning models may be characterized by including a first machine learning model that operates based on facial image data; a second machine learning model that operates based on oral image data; and a third machine learning model that operates based on dental image data.

[0011] In addition, the feature data generation unit, the type and size data generation unit, and the recommended artificial tooth appearance data generation unit may be characterized as being an ensemble model of the plurality of machine learning models.

[0012] In addition, the first machine learning model, the second machine learning model, and the third machine learning model may be characterized in that they are configured using different types of machine learning models.

[0013] In addition, the feature data generation unit, the type and size data generation unit, and the recommended artificial tooth appearance data generation unit may be characterized as being different types of machine learning models.

[0014] According to one aspect of the disclosed invention, an objective method for selecting an artificial tooth optimized for a patient can be provided.

[0015] According to one aspect of the disclosed invention, the overall quality of prosthetic treatment can be improved by reducing the dependence on the experience of individual dentists in prosthetic treatment.

[0016] FIG. 1 is a block diagram showing the configuration of an artificial tooth recommendation device according to one aspect of the disclosed invention.

[0017] FIG. 2 is a conceptual diagram illustrating the generation of facial mesh data from facial image data according to one aspect of the disclosed invention.

[0018] FIG. 3 is a conceptual diagram illustrating the generation of oral mesh data from oral image data according to one aspect of the disclosed invention.

[0019] FIG. 4 is a conceptual diagram illustrating an example of a prosthetic treatment method and an artificial tooth used in the prosthetic treatment that can be selected according to one aspect of the disclosed invention.

[0020] FIG. 5 is a conceptual diagram showing the correspondence between selectable arch shapes and artificial tooth shapes according to one aspect of the disclosed invention.

[0021] FIG. 6 is a conceptual diagram showing the correspondence between facial appearance, arch appearance, and artificial teeth according to one aspect of the disclosed invention.

[0022] FIG. 7 is a conceptual diagram illustrating an example of a facial appearance according to one aspect of the disclosed invention.

[0023] FIG. 8 is a conceptual diagram illustrating providing facial type data and artificial tooth recommendation information from facial appearance data according to one aspect of the disclosed invention.

[0024] FIG. 9 is a block diagram showing the internal configuration of a computing device according to one embodiment of the present disclosure.

[0025] Like reference numerals refer to like elements throughout the specification. This specification does not describe all elements of the embodiments, and general information within the technical field to which the disclosed invention pertains or information that overlaps between the embodiments is omitted.

[0026] Additionally, when a part is said to "include" a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0027] As used herein, the term "~unit" refers to a unit that processes at least one function or operation, and may refer to, for example, software, an FPGA, or a hardware component. The function provided by the "~unit" may be performed separately by multiple components, or may be integrated with other additional components. The "~unit" of this specification is not necessarily limited to software or hardware, and may be configured to be located in an addressable storage medium, or may be configured to play one or more processors. According to embodiments, multiple "~units" may be implemented as a single component, or a single "~unit" may include multiple components.

[0028] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0029] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0030] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0031] In the present disclosure, machine learning models or artificial intelligence models may include relatively simple machine learning models, such as expert systems, support vector machines, and decision trees, as well as artificial intelligence models based on artificial neural networks. However, the terms "machine learning" and "artificial intelligence" may be used for the purpose of explaining general concepts.

[0032] The artificial intelligence-related functions according to the present disclosure are operated via a processor and memory. The processor may be comprised of one or more processors. In this case, one or more processors may be a general-purpose processor such as a CPU, an AP, a Digital Signal Processor (DSP), a graphics-only processor such as a GPU or a Vision Processing Unit (VPU), or an artificial intelligence-only processor such as an NPU. One or more processors control the processing of input data according to predefined operating rules or artificial intelligence models stored in memory. Alternatively, if one or more processors are artificial intelligence-only processors, the artificial intelligence-only processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model.

[0033] According to some embodiments of the present disclosure, a machine learning or artificial intelligence model is characterized by being created through learning. Here, "created through learning" means that a basic artificial intelligence model is trained using a learning algorithm using a plurality of learning data, thereby creating a predefined operation rule or artificial intelligence model set to perform a desired characteristic (or purpose). This learning may be performed on the device itself on which the artificial intelligence model according to the present disclosure is executed, or may be performed through a separate server and / or system. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0034] An artificial intelligence model may be composed of multiple neural network layers. Each of the multiple neural network layers has multiple weight values, and performs neural network operations through operations between the operation results of the previous layer and the multiple weights. The multiple weights of the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, the multiple weights may be updated so that the loss value or cost value obtained from the artificial intelligence model is reduced or minimized during the learning process. The artificial neural network may include a deep neural network (DNN), and examples thereof include, but are not limited to, a convolutional neural network (CNN), a deep neural network (DNN), a recurrent neural network (RNN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), or deep Q-networks.

[0035] According to an exemplary embodiment of the present disclosure, a processor can execute an artificial intelligence model. Artificial intelligence refers to a machine learning technology based on an artificial neural network that imitates human neurons (biological neurons) to enable machines to learn. Artificial intelligence methodologies can be categorized into supervised learning, in which input data and output data are provided together as training data depending on the learning method, so that the solution (output data) to the problem (input data) is determined; unsupervised learning, in which only input data is provided without output data, so that the solution (output data) to the problem (input data) is not determined; and reinforcement learning, in which a reward is provided from an external environment whenever an action is taken in the current state (State), and learning is performed in a direction that maximizes this reward. In addition, artificial intelligence methodologies can be categorized according to the architecture of the learning model. The architectures of widely used deep learning technologies can be categorized into convolutional neural networks (CNNs), recurrent neural networks (RNNs), transformers, and generative adversarial networks (GANs).

[0036] The machine learning model and / or artificial intelligence model according to the present disclosure may be a single artificial intelligence model or may be implemented as multiple artificial intelligence models. The machine learning and / or artificial intelligence model may be composed of a neural network (or artificial neural network) and may include statistical learning algorithms that mimic biological neurons, as studied in machine learning and cognitive science. A neural network may refer to a general model that has problem-solving capabilities by changing the binding strength of synapses through learning, formed by artificial neurons (nodes) that form a network by combining synapses. The neurons of a neural network may include a combination of weights or biases. The neural network may include one or more layers composed of one or more neurons or nodes. For example, a device may include an input layer, a hidden layer, and an output layer. The neural network constituting the device can infer a desired result (output) from any input (input) by changing the weights of neurons through learning.

[0037] The processor can create a neural network, train (or learn) a neural network, perform operations based on received input data, generate an information signal based on the results of the operations, or retrain the neural network. The models of the neural network can include various types of models such as CNN (Convolution Neural Network) such as GoogleNet, AlexNet, VGG Network, R-CNN (Region with Convolution Neural Network), RPN (Region Proposal Network), RNN (Recurrent Neural Network), S-DNN (Stacking-based deep Neural Network), S-SDNN (State-Space Dynamic Neural Network), Deconvolution Network, DBN (Deep Belief Network), RBM (Restrcted Boltzman Machine), Fully Convolutional Network, LSTM (Long Short-Term Memory) Network, Classification Network, etc., but are not limited thereto. The processor can include one or more processors for performing operations according to the models of the neural network. For example, the neural network can be a deep neural network. It may include a deep neural network.

[0038] The operating principle and embodiments of the disclosed invention are described below with reference to the attached drawings.

[0039]

[0040] FIG. 1 is a block diagram showing the configuration of an artificial tooth recommendation device according to one aspect of the disclosed invention.

[0041] Referring to FIG. 1, the artificial tooth recommendation system according to the present disclosure may include an image data input unit (100), a feature and appearance determination unit (200), and an artificial tooth recommendation unit (300).

[0042] An image data input unit (100) according to some embodiments of the present disclosure may be a module configured through software, hardware, and / or a combination thereof for receiving image data that serves as basic data for recommending an artificial tooth.

[0043] In one embodiment, the image data input unit (100) can input image data required for recommending artificial teeth. While FIG. 1 illustrates that facial image data, oral image data, and / or dental image data can be input into the image data input unit (100), the types of image data that can be input through the image data input unit (100) are not limited thereto. For example, image data obtained by photographing and / or scanning the temporomandibular joint area, image data obtained by photographing and / or scanning the philtrum or chin area, etc. can also be input through the image data input unit (100).

[0044] Image data input through the image data input unit (100) according to some embodiments of the present disclosure may be image data generated through shooting and / or scanning, such as camera, X-ray, CT and / or MRI images.

[0045] In some embodiments of the present disclosure, the facial image data input unit (110) may be a module implemented using software, hardware, and / or a combination thereof for receiving data generated by photographing and / or scanning a face. In one embodiment, the facial image data may be an image that represents the front, side, and / or both the front and side of the face. In another embodiment, the facial image data may be image data that represents only a portion of the face. For example, the facial image data may be image data that represents only the cheekbones, mouth, and temporomandibular joints among the facial parts.

[0046] In some embodiments of the present disclosure, the oral image data input unit (120) may be a module implemented using software, hardware, and / or a combination thereof for receiving data generated by photographing and / or scanning the inside of the oral cavity. In some embodiments, the oral image data may be image data representing the oral cavity as a whole. In one embodiment, the oral image data may be image data generated by photographing and / or scanning the oral cavity from the front and / or side. In another embodiment, the oral image data may be image data representing the upper and / or lower parts of the oral cavity as a whole.

[0047] In some embodiments of the present disclosure, the tooth image data input unit (130) may be a module implemented by software, hardware, and / or a combination thereof for receiving data generated by photographing and / or scanning one or more teeth. In some embodiments, the tooth image data may be image data representing individual teeth or representing a specific group of teeth. In one embodiment, the tooth image data may be generated for each of wisdom teeth, molars, incisors, and / or canines. In another embodiment, the tooth image data may be image data generated by photographing and / or scanning a group of teeth including a plurality of molars located within a specific portion of the oral cavity.

[0048] Referring back to FIG. 1, an artificial tooth recommendation system according to some embodiments of the present disclosure may include a feature and appearance determination unit (200).

[0049] In some embodiments, the feature and appearance determination unit (200) may be a module implemented with software, hardware, and / or a combination thereof for generating mesh data and feature data based on input image data, and generating data on the type and size of the face, oral cavity, and / or teeth using the generated mesh data and feature data.

[0050] In some embodiments, the feature and appearance determination unit (200) may include one or more machine learning modules for generating feature data, type data, and / or size data from input image data. In one embodiment, the feature data, type data, and size data may all be generated by a single machine learning module. In another embodiment, multiple machine learning modules may separately generate the feature data, type data, and / or size data. Furthermore, each of the machine learning modules for generating the feature data, type data, and / or size data may again be constructed using multiple machine learning modules. For example, the machine learning model for generating the feature data, type data, and / or size data may be constructed using an ensemble technique, etc.

[0051] The mesh generation unit (210) according to some embodiments of the present disclosure may be a module implemented with software, hardware, and / or a combination thereof for generating mesh data corresponding to image data input from the image data input unit (100).

[0052] In some embodiments, mesh data may be data representing a three-dimensional shape of a face, oral cavity, and / or teeth using vertices, edges, and faces. In one embodiment, the mesh generation unit (210) may represent a face, oral cavity, or teeth represented in image data as mesh data using a combination of vertices, edges, and faces.

[0053] In one embodiment, the types of image data and mesh data may correspond. For example, if the image data is a facial image, the mesh data may be mesh data representing the face. In another embodiment, if the image data is an oral image, the mesh data may be mesh data representing the oral cavity. Similarly, if the image data is a dental image, the mesh data may be dental mesh data.

[0054] In one embodiment, the mesh generation unit (210) may generate mesh data using at least one of a facial image, an oral image, and / or a dental image. Preferably, the mesh generation unit (210) may generate mesh data using only one of the facial image, the oral image, and / or the dental image.

[0055] In some embodiments, the mesh generation unit (210) may generate mesh data from input image data using a machine learning model. In one embodiment, the mesh generation unit (210) may generate mesh data using one or more machine learning models. Preferably, the mesh generation unit (210) may be implemented as an ensemble of multiple machine learning models. In addition, the machine learning model used in the mesh generation unit (210) may have as its output a graph including vertices and edges or a data structure including vertices, edges, and surfaces. In this case, the machine learning model used in the mesh generation unit (210) may be specifically trained to output an appropriate data structure. For example, the machine learning model used in the mesh generation unit (210) may use a graph neural network, etc.

[0056] The feature data generation unit (220) according to some embodiments of the present disclosure may be a module implemented using software, hardware, and / or a combination thereof for generating feature data based on mesh data generated from the mesh generation unit (210).

[0057] In one embodiment, the feature data may be feature data of a type corresponding to the image data input from the image data input unit (100). For example, if the image data is a facial image, the generated feature data may be facial feature data. As another example, if the image data is an oral image, the generated feature data may be oral image data. As yet another example, if the image data is a dental image, the generated feature data may be dental image data.

[0058] The feature data generated by the feature data generation unit (220) according to some embodiments of the present disclosure may include one or more feature points and attribute data of the feature points generated from mesh data.

[0059] In some embodiments, feature points may refer to particularly important or meaningful portions among the vertices included in the mesh data, and may refer to selected vertices that can well represent an object represented by the mesh data. In one embodiment, the attribute data of the feature points may be data expressing information about the ratio, distance, and / or depth of the feature points. In one embodiment, the ratio of the feature points may refer to the ratio of the feature points to the total vertices included in the mesh data. In another embodiment, the distance between the feature points may refer to the distance between a plurality of feature points. In another embodiment, the depth of the feature point may be information expressing the position of the feature point in space, and may be information indicating the distance from the shooting point of the image data.

[0060] In one embodiment, the feature data generation unit (220) may generate feature data using at least one of facial image, oral image, and / or dental image data. Preferably, the feature data generation unit (220) may generate feature data using any one of facial image, oral image, and dental image data.

[0061] In one embodiment, the feature data generation unit (220) may be constructed using one or more machine learning modules. In one embodiment, the feature data generation unit (220) may be constructed using an ensemble of multiple machine learning modules. In this case, the multiple machine learning modules used in the feature data generation unit (220) may differ from each other in their structure and specific implementation. In one embodiment, the data output by the feature data generation unit (220) may include feature point attribute data including identification data of the feature point, location, ratio, distance, and depth information of the feature point.

[0062] In some embodiments, the feature data generation unit (220) may utilize existing mesh data and feature data as ground truth. In another embodiment, the feature data generation unit (220) may be trained by performing unsupervised learning, which extracts feature data from mesh data, generates mesh data again from the extracted feature data, and uses the difference between the generated mesh data and the original mesh data as a reconstruction loss.

[0063] The feature and appearance determination unit (200) according to some embodiments of the present disclosure may further include a type and size data generation unit (230).

[0064] In some embodiments, the type and size data generation unit (230) may be a module implemented by software, hardware, and / or a combination thereof for generating type data and / or size data of at least one of a face, a jaw, or a tooth from image data, mesh data, and / or feature data.

[0065] In some embodiments, the type data may be data for representing information regarding the type of the face, the type of the jaw, and / or the type of the teeth. In some embodiments, the type of the face may be represented by one or more face type categories pre-stored in the memory. In some other embodiments, the type of the jaw may be represented by one or more jaw type categories pre-stored in the memory. In yet other embodiments, the type of the teeth may be represented by one or more tooth type categories pre-stored in the memory. In addition, preferably, the type data may additionally include information regarding whether the face, the jaw, and / or the teeth are normal.

[0066] In some embodiments, the size data may be data for representing information regarding the size of the face, the size of the jaw, and / or the size of the teeth. In some embodiments, the size data may be represented as one or more size categories (e.g., large / medium / small or 1 / 2 / 3 / 4 / 5) stored in the memory. In another embodiment, such size categories may correspond to size information included in artificial tooth product information previously stored in the memory. In another embodiment, the size data may be real number data and may not be represented as a specific category.

[0067] In some embodiments, the type and size data generation unit (230) may generate the type and size data based on at least one of a facial image, an oral image, and / or a dental image. Preferably, the type and size data generation unit (230) may generate the type and size data using any one of the facial image, the oral image, and / or the dental image. In addition, the type data and size data generated by the type and size data generation unit (230) may correspond to the type of the image data.

[0068] In some embodiments, the type and size data generation unit (230) may be implemented using one or more machine learning modules. In one embodiment, the type data and the size data may each be generated by different machine learning modules. When the type and size data generation unit (230) utilizes multiple machine learning modules, the type and size data generation unit (230) may be implemented by ensembling these multiple machine learning modules.

[0069] In some embodiments, the training of the type and size data generation unit (230) may be implemented in a supervised learning manner based on image data, mesh data, and / or feature data and the type data and size data corresponding thereto. For example, during training of the type and size data generation unit (230), feature data may be provided, and type data and size data corresponding to the provided feature data may be inferred. In one embodiment, when the type data and size data are expressed as categories, the training of the type and size data generation unit (230) may be performed using a cross-entropy function as a loss function. In another embodiment, when the size data is expressed as real numbers, the training of the machine learning module for generating the size data may be performed using an index such as MSE and / or MAE as a loss function. That is, when the type data and the size data are respectively generated by different machine learning modules, their training may be performed independently.

[0070] Referring back to FIG. 1, the artificial tooth recommendation unit (300) according to some embodiments of the present disclosure may be a module implemented with software, hardware, and / or a combination thereof for providing a user with appropriate recommended artificial tooth information based on generated mesh data, feature data, type data, and / or size data.

[0071] An artificial tooth recommendation unit (300) according to some embodiments of the present disclosure may include a recommended artificial tooth appearance data generation unit (310). In some embodiments, the recommended artificial tooth appearance data generation unit (310) may be a module for recommending an appearance of an artificial tooth based on generated mesh data, feature data, type data, and / or size data. Preferably, the recommended artificial tooth appearance data generation unit (310) may determine appearance data of an artificial tooth suitable for input image data, mesh data, feature data, type data, and / or size data. In one embodiment, the appearance data of the artificial tooth may be expressed as one or more categories pre-stored in a memory. In one embodiment, the appearance data of the artificial tooth may include the type and size of the artificial tooth. Preferably, the type of the artificial tooth may be expressed as Square, Tapering, Ovoid, Short square, and / or Combination depending on the arch type, and the size of the artificial tooth may be expressed as Small, Medium.

[0072] In one embodiment, the recommended artificial tooth appearance data generation unit (310) may be constructed using one or more machine learning models. Preferably, the recommended artificial tooth appearance data generation unit may receive at least one of mesh data, feature data, type data, and / or image data to generate the recommended artificial tooth appearance data. At this time, the machine learning model may be trained using the previously stored mesh data, feature data, type data, and / or image data, the recommended artificial tooth appearance data, and the inferred recommended artificial tooth appearance data. Preferably, one or more machine learning models constructing the recommended artificial tooth appearance data generation unit (310) may be trained using a cross entropy function as a loss function.

[0073] The recommended artificial tooth product determination unit (320) according to some embodiments of the present disclosure may be a module implemented with software, hardware, and / or a combination thereof that determines a recommended artificial tooth product based on recommended artificial tooth appearance data generated from the recommended artificial tooth appearance data generation unit (310).

[0074] In some embodiments, the recommended artificial tooth product determination unit (320) may determine an artificial tooth product corresponding to the generated recommended artificial tooth appearance data as the recommended artificial tooth product. In one embodiment, information regarding the recommended artificial tooth product may include information regarding the brand, size, and arch type of the recommended artificial tooth product.

[0075] In addition, the recommended artificial tooth product determination unit (320) can determine multiple artificial tooth products as recommended artificial tooth products. Preferably, the recommended artificial tooth product determination unit (320) determines multiple artificial tooth products as recommended artificial tooth products, and can determine the recommendation ranking of the multiple artificial tooth products. At this time, various attribute information of the artificial tooth products can be used to determine the recommendation ranking. For example, the recommendation ranking of the multiple artificial tooth products can be determined based on the external appearance information of the artificial tooth products.

[0076] In some embodiments, the recommended artificial tooth product information determined by the recommended artificial tooth product determination unit (320) may be provided to the user by being output by an output unit (not shown) including a display unit.

[0077] The components illustrated in FIG. 1 are exemplary components for implementing the technical features of the present invention. Therefore, the configurations for implementing the technical features of the present invention are not limited to the details illustrated in FIG. 1. That is, additional components other than those illustrated in FIG. 1 may be added to implement the technical features of the present invention, or multiple components illustrated in FIG. 1 may be combined to implement a single component. Furthermore, the components for implementing the technical features of the present invention should not be interpreted as being limited to the details illustrated in FIG. 1.

[0078] FIG. 2 is a conceptual diagram illustrating the generation of facial mesh data from facial image data according to one aspect of the disclosed invention.

[0079] Referring to FIG. 2, mesh data and feature data (420) can be generated from image data (410) according to some embodiments of the present disclosure.

[0080] In some embodiments, the image data (410) may be an image of the face of the target for artificial tooth recommendation. For convenience of explanation, the image data (410) is expressed as a photograph taken by a camera, but the image data may be composed of various data for representing the face. For example, the image data (410) may include medical image data such as X-ray, CT, and / or MRI, and image data scanned from the same.

[0081] The artificial tooth recommendation method according to some embodiments of the present disclosure can generate mesh data and feature data (420) from image data (410).

[0082] In some embodiments, the mesh data may be a combination or set of vertices, edges, and / or surfaces for representing a facial area corresponding to the image data (410), or structured data (e.g., a graph, etc.) representing the same. In one embodiment, feature data may be generated from the mesh data. In one embodiment, the feature data may include a plurality of feature points corresponding to a subset of vertices included in the mesh data for appropriately representing facial features, and attribute data of the feature points. In one embodiment, the attribute data of the feature points may include ratio, distance, and depth information of the feature points. Preferably, the extraction of the feature points may be implemented using one or more pre-trained machine learning models.

[0083]

[0084] FIG. 3 is a conceptual diagram illustrating the generation of oral mesh data from oral image data according to one aspect of the disclosed invention.

[0085] Referring to FIG. 3, mesh data and feature data (520) can be generated from image data (510) according to some embodiments of the present disclosure.

[0086] In some embodiments, the image data (510) may be an image of the oral cavity of the target for artificial tooth recommendation. For convenience of explanation, the image data (410) is expressed as a photograph taken by a camera, but the image data may be composed of various data for representing the facial area. For example, the image data (410) may include medical image data such as X-ray, CT, and / or MRI, and image data scanned from such data.

[0087] The artificial tooth recommendation method according to some embodiments of the present disclosure can generate mesh data and feature data (420) from image data (410).

[0088] In some embodiments, the mesh data may be a combination or set of vertices, edges, and / or surfaces for representing an oral cavity corresponding to the image data (410), or structured data (e.g., a graph, etc.) representing the same. In one embodiment, feature data may be generated from the mesh data. In one embodiment, the feature data may include a plurality of feature points corresponding to a subset of vertices included in the mesh data for appropriately representing facial features, and attribute data of the feature points. In one embodiment, the attribute data of the feature points may include ratio, distance, and depth information of the feature points. Preferably, the extraction of the feature points may be implemented using one or more pre-trained machine learning models.

[0089]

[0090] FIG. 4 is a conceptual diagram illustrating an example of a prosthetic treatment method and an artificial tooth used in the prosthetic treatment that can be selected according to one aspect of the disclosed invention.

[0091] Referring to Figure 4, prosthetic treatment can be divided into full denture treatment corresponding to all missing teeth in the oral cavity or partial denture treatment corresponding to some missing teeth in the oral cavity.

[0092] In some embodiments, the artificial tooth recommendation system according to the present disclosure may recommend an appropriate artificial tooth for each tooth included in a complete denture or an individual denture. In one embodiment, the artificial tooth recommendation system may determine an appropriate denture artificial tooth product for each individual tooth included in a denture based on generated image data, mesh data, feature data, and / or type and size data.

[0093]

[0094] FIG. 5 is a conceptual diagram showing the correspondence between selectable arch shapes and artificial tooth shapes according to one aspect of the disclosed invention.

[0095] Referring to FIG. 5, there is corresponding data to assist in selecting the shape and size of an artificial tooth based on the external appearance of the jaw of a patient receiving conventional prosthetic treatment. In some embodiments of the present disclosure, a feature and shape determination unit (or, preferably, a type and size data generation unit) for determining a recommended artificial tooth and / or an artificial tooth recommendation unit (or, preferably, a recommended artificial tooth shape data generation unit) may be learned based on corresponding data as disclosed in FIG. 5.

[0096] For example, as illustrated in FIG. 5, categories of artificial tooth products based on arch type and categories of artificial tooth products based on arch and tooth size may be stored in structured data based on previously stored corresponding information. In this case, the feature and appearance determination unit and / or the artificial tooth recommendation unit may utilize this as ground truth to train one or more machine learning models to generate recommended artificial tooth appearance information.

[0097]

[0098] FIG. 6 is a conceptual diagram showing the correspondence between facial appearance, arch appearance, and artificial teeth according to one aspect of the disclosed invention.

[0099] Referring to FIG. 6, the artificial tooth recommendation system according to some embodiments of the present disclosure may include correspondence data indicating a correspondence relationship between a facial appearance, a jaw appearance, and an artificial tooth. In one embodiment, the facial appearance may be included in the correspondence data as image information, and the jaw appearance may be included in the correspondence data as a data format for expressing category information.

[0100]

[0101] FIG. 7 is a conceptual diagram illustrating an example of a facial appearance according to one aspect of the disclosed invention.

[0102] Referring to FIG. 7, the artificial tooth recommendation system according to some embodiments of the present disclosure may represent facial appearance as one or more categories. For example, as illustrated in FIG. 7, the categories of facial appearance may include a first type (610), a second type (620), and a third type (630). In one embodiment, the pre-stored facial appearance categories may be determined in association with facial image data and may be pre-stored in one or more storage devices included in the artificial tooth recommendation system.

[0103]

[0104] FIG. 8 is a conceptual diagram illustrating providing facial type data and artificial tooth recommendation information from facial appearance data according to one aspect of the disclosed invention.

[0105] Referring to FIG. 8, an artificial tooth recommendation system according to some embodiments of the present disclosure may include an interface for receiving image data from a user and providing information extracted from the provided image data to the user.

[0106] The artificial tooth recommendation system according to some embodiments of the present disclosure may include an image input interface (710). In one embodiment, the image input interface (710) allows a user to input a facial image, an oral image, and / or a dental image. For example, the image input interface (710) may be linked to an image input device (e.g., a camera) of the artificial tooth recommendation system, and output an image input to the camera, thereby allowing the user to appropriately input a facial, oral, or dental image into the artificial tooth recommendation system.

[0107] The artificial tooth recommendation system according to some embodiments of the present disclosure can provide a user with mesh data and feature data generated from input image data through a mesh data and feature data output interface (720).

[0108] Referring back to FIG. 8, the artificial tooth recommendation system according to some embodiments of the present disclosure may provide one or more generated type data and / or size data via a type and size output interface (730). In this case, preferably, the artificial tooth recommendation system may provide a plurality of type data to the user, determine a ranking of the type data and / or size data based on similarity, and then provide the ranking via the output interface (730).

[0109] In some embodiments, the artificial tooth recommendation system may provide recommended artificial tooth product data via the artificial tooth product information interface (740). Preferably, the artificial tooth recommendation system may generate multiple recommended artificial tooth product data. More preferably, the artificial tooth recommendation system may determine a ranking of the multiple recommended artificial tooth products and provide the determined ranking via the artificial tooth product information interface (740).

[0110]

[0111] FIG. 9 is a block diagram showing the internal configuration of a computing device (800) according to one embodiment of the present disclosure.

[0112] The computing device (800) may include a memory (810), a processor (820), a communication module (830), and an input / output interface (840), and as illustrated in FIG. 8, the computing device (800) may be configured to communicate information and / or data via a network using the communication module (830).

[0113] The memory (810) may include any non-transitory computer-readable recording medium. In one embodiment, the memory (810) may include a non-volatile mass storage device such as a random access memory (RAM), a read only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. As another example, a non-volatile mass storage device such as a ROM, an SSD, a flash memory, a disk drive, etc. may be included in the computing device (800) as a separate permanent storage device distinct from the memory. In addition, the memory (810) may store an operating system and at least one program code.

[0114] These software components may be loaded from a computer-readable recording medium separate from the memory (810). This separate computer-readable recording medium may include a recording medium directly connectable to the computing device (800), for example, a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, etc. As another example, the software components may be loaded into the memory (810) via a communication module (830) other than a computer-readable recording medium. For example, at least one program may be loaded into the memory (810) based on a computer program that is installed by files provided by developers or a file distribution system that distributes installation files of applications via the communication module (830).

[0115] Here, the program may include program commands, data files, and data structures, either singly or in combination. The program may be designed and produced using machine language code or high-level language code. The program may be specifically designed to implement the aforementioned malicious application detection system, or may be implemented using various functions or definitions that are known and available to those skilled in the art of computer software. The program for implementing the aforementioned malicious application detection system may be recorded on a processor-readable recording medium.

[0116] The memory can store a program that performs the operations described above and the operations described below, and the processor can execute the program stored in the memory. In the case where there are multiple processors and memories, they can be integrated into a single chip or provided in physically separate locations. The memory can include volatile memory such as Static Random Access Memory (S-RAM) and Dynamic Random Access Memory (D-RAM) for temporarily storing data. In addition, the memory can include non-volatile memory such as Read Only Memory (ROM), Erasable Programmable Read Only Memory (EPROM), and Electrically Erasable Programmable Read Only Memory (EEPROM) for long-term storage of control programs and control data.

[0117]

[0118] The processor (820) may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to another user terminal (not shown) or another external system via the memory (810) or the communication module (830).

[0119] A processor may include various logic circuits and arithmetic circuits, process data according to a program provided from memory, and generate control signals based on the processing results. The memory and processor may be implemented as separate chips. Alternatively, the memory and processor may be implemented as a single chip.

[0120] The processor (820) may include one or more processors. In this case, the one or more processors may be homogeneous or heterogeneous processors. For example, the processor (820) may include heterogeneous processors, such as a central processing unit (CPU), a graphics processing unit (GPU), a tensor processing unit (TPU), and / or a neural processing unit (NPU). For convenience of explanation, one or more homogeneous or heterogeneous processors may be referred to as a "processor" in this specification.

[0121] The communication module (830) may provide a configuration or function for a user terminal (not shown) and a computing device (800) to communicate with each other via a network, and may provide a configuration or function for the computing device (800) to communicate with an external system (e.g., a separate cloud system, etc.). For example, control signals, commands, data, etc. provided under the control of the processor (820) of the computing device (800) may be transmitted to the user terminal and / or the external system via the communication module (830) and the network via the communication module of the user terminal and / or the external system.

[0122] The wired communication module may include various wired communication modules such as a Local Area Network (LAN) module, a Wide Area Network (WAN) module, or a Value Added Network (VAN) module, as well as various cable communication modules such as a Universal Serial Bus (USB), a High Definition Multimedia Interface (HDMI), a Digital Visual Interface (DVI), RS-232 (recommended standard 232), power line communication, or plain old telephone service (POTS).

[0123] The wireless communication module may include a wireless communication module that supports various wireless communication methods such as GSM (global System for Mobile Communication), CDMA (Code Division Multiple Access), WCDMA (Wideband Code Division Multiple Access), UMTS (universal mobile telecommunications system), TDMA (Time Division Multiple Access), LTE (Long Term Evolution), 4G, 5G, and 6G, in addition to a WiFi module and a Wireless Broadband module.

[0124] The wireless communication module may include a wireless communication interface including an antenna and a transmitter for transmitting a Wi-Fi signal. Furthermore, the wireless communication module may further include a Wi-Fi signal conversion module that modulates a digital control signal output from the control unit through the wireless communication interface into an analog wireless signal under the control of the control unit.

[0125] The wireless communication module may include a wireless communication interface including an antenna and a receiver for receiving Wi-Fi signals. Furthermore, the wireless communication module may further include a Wi-Fi signal conversion module for demodulating an analog wireless signal received through the wireless communication interface into a digital control signal.

[0126] The short-range communication module is for short-range communication, and can support short-range communication using at least one of Bluetooth, RFID (Radio Frequency Identification), Infrared Data Association (IrDA), UWB (Ultra Wideband), ZigBee, NFC (Near Field Communication), Wi-Fi (Wireless-Fidelity), Wi-Fi Direct, and Wireless USB (Wireless Universal Serial Bus) technologies.

[0127] Additionally, the input / output interface (840) of the computing device (800) may be a means for interfacing with a device (not shown) for input or output that may be connected to the computing device (800) or that the computing device (800) may include.

[0128] The input unit is configured to receive information and data including audio information (or signals), text, etc. from a network or a user, and may include at least one microphone and at least one user input unit. Data collected from the input unit may be analyzed and processed into a user control command.

[0129] The user input unit is for receiving information from the user, and when information is input through the user input unit, the control unit can control the operation of the device to correspond to the input information. The user input unit may include a hardware physical key (e.g., a button located on at least one of the front, rear, and side of the device, a dome switch, a jog wheel, a jog switch, etc.) and a software touch key. As an example, the touch key may be a virtual key, a soft key, or a visual key displayed on a touch screen type display unit through software processing, or may be a touch key disposed on a part other than the touch screen. Meanwhile, the virtual key or visual key may have various forms and be displayed on the touch screen, and may be, for example, formed of a graphic, text, an icon, a video, or a combination thereof.

[0130] Although the input / output interface (840) is illustrated as a separate element from the processor (820) in FIG. 8, this is not limiting, and the input / output interface (840) may be configured to be included in the processor (820). The computing device (800) may include more components than those illustrated in FIG. 8. However, there is no need to explicitly illustrate most of the conventional components.

[0131] The processor (820) of the computing device (800) may be configured to manage, process and / or store information and / or data received from multiple user terminals and / or multiple external systems.

[0132] The above-described methods and / or various embodiments may be realized by digital electronic circuits, computer hardware, firmware, software, and / or a combination thereof. Various embodiments of the present disclosure may be implemented as a computer program that is executed by a data processing device, for example, one or more programmable processors and / or one or more computing devices, or stored on a computer-readable recording medium and / or a computer-readable recording medium. The above-described computer program may be written in any form of programming language, including a compiled language or an interpreted language, and may be distributed in any form, such as a standalone program, a module, a subroutine, etc. The computer program may be distributed through a single computing device, multiple computing devices connected through the same network, and / or multiple computing devices distributed to be connected through multiple different networks.

[0133] The methods and / or various embodiments described above may be performed by one or more processors configured to execute one or more computer programs that process, store, and / or manage any function, function, etc. by operating on the basis of input data or generating output data. For example, the methods and / or various embodiments of the present disclosure may be performed by special purpose logic circuits such as Field Programmable Gate Arrays (FPGAs) or Application Specific Integrated Circuits (ASICs), and an apparatus and / or system for performing the methods and / or embodiments of the present disclosure may be implemented as special purpose logic circuits such as FPGAs or ASICs.

[0134] The one or more processors executing the computer program may include a general-purpose or special-purpose microprocessor and / or one or more processors of any type of digital computing device. The processor may receive instructions and / or data from each of read-only memory and random-access memory, or may receive instructions and / or data from the read-only memory and the random-access memory. In the present invention, components of a computing device performing the methods and / or embodiments may include one or more processors for executing instructions, and one or more memory devices for storing instructions and / or data.

[0135] According to one embodiment, the computing device can transmit and receive data to and from one or more mass storage devices for storing data. For example, the computing device can receive and / or transfer data from a magnetic disc or an optical disc. A computer-readable storage medium suitable for storing instructions and / or data associated with a computer program may include, but is not limited to, any form of non-volatile memory, including semiconductor memory devices such as Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory devices, and the like. For example, the computer-readable storage medium may include a magnetic disk such as an internal hard disk or a removable disk, a magneto-optical disk, a CD-ROM, and a DVD-ROM disk.

[0136] To provide interaction with a user, a computing device may include, but is not limited to, a display device (e.g., a cathode ray tube (CRT), a liquid crystal display (LCD), etc.) for providing or displaying information to the user, and a pointing device (e.g., a keyboard, a mouse, a trackball, etc.) for allowing the user to provide input and / or commands to the computing device. That is, the computing device may further include any other types of devices for providing interaction with the user. For example, the computing device may provide any form of sensory feedback to the user, including visual feedback, auditory feedback, and / or tactile feedback, for interaction with the user. In this regard, the user may provide input to the computing device through various gestures, such as visual, vocal, or motion.

[0137] In the present invention, various embodiments may be implemented in a computing system that includes backend components (e.g., a data server), middleware components (e.g., an application server), and / or front-end components. In this case, the components may be interconnected via any form or medium of digital data communication, such as a communications network. For example, the communications network may include a Local Area Network (LAN), a Wide Area Network (WAN), etc.

[0138] A computing device based on the exemplary embodiments described herein may be implemented using hardware and / or software configured to interact with a user, including a user device, a user interface (UI) device, a user terminal, or a client device. For example, the computing device may include a portable computing device, such as a laptop computer. Additionally or alternatively, the computing device may include, but is not limited to, Personal Digital Assistants (PDAs), tablet PCs, game consoles, wearable devices, Internet of Things (IoT) devices, virtual reality (VR) devices, augmented reality (AR) devices, and the like. The computing device may further include other types of devices configured to interact with a user. Furthermore, the computing device may include a portable communication device (e.g., a mobile phone, a smart phone, a cordless cellular phone, etc.) suitable for wireless communication over a network, such as a mobile communication network. The computing device may be configured to communicate wirelessly with a network server using wireless communication technologies and / or protocols, such as Radio Frequency (RF), Microwave Frequency (MWF), and / or Infrared Ray Frequency (IRF).

[0139] The various embodiments of the present invention, including specific structural and functional details, are exemplary. Therefore, the embodiments of the present disclosure are not limited to those described above and may be implemented in various other forms. Furthermore, the terminology used herein is intended to describe certain embodiments and is not intended to limit the embodiments. For example, singular terms and the above may be interpreted to include plural forms, unless the context clearly dictates otherwise.

[0140] In the present invention, unless otherwise defined, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by those of ordinary skill in the art to which they pertain. Furthermore, commonly used terms, such as terms defined in dictionaries, should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology.

[0141] While the present invention has been described in connection with certain embodiments herein, various modifications and variations can be made without departing from the scope of the present disclosure, which would be apparent to those skilled in the art. Furthermore, such modifications and variations are intended to fall within the scope of the claims appended to this specification.

[0142]

[0143] The disclosed embodiments have been described with reference to the attached drawings as described above. Those skilled in the art will understand that the present invention can be implemented in forms other than the disclosed embodiments without altering the technical aspects or essential features of the present invention. The disclosed embodiments are illustrative and should not be construed as limiting.

Claims

1. As a method of recommending artificial teeth, the above artificial teeth recommendation method is: A step of receiving image data, wherein the image data includes at least one of facial image data, oral image data, or dental image data; A step of generating corresponding mesh data from the input image data, wherein the mesh data includes at least one of facial mesh data, oral mesh data, or dental mesh data; A step of generating feature data from the generated mesh data using a machine learning-based feature data generation unit; A step of generating type data or size data from the generated feature data using a machine learning-based type and size data generation unit, wherein the type data or the size data is data representing the type or size of any one of the face, oral cavity, or teeth; and A step of generating recommended artificial tooth appearance data corresponding to the type data or the size data using a machine learning-based recommended artificial tooth appearance data generation unit; including; How to recommend artificial teeth.

2. In paragraph 1, The above feature data generation unit, the above type and size data generation unit, and the above recommended artificial tooth appearance data generation unit, Characterized in that it operates based on any one of the facial image data, the oral image data, and the dental image data. How to recommend artificial teeth.

3. In paragraph 1, The above feature data is, characterized by including multiple feature points, How to recommend artificial teeth.

4. In paragraph 3, The step of generating the above type data or the above size data is, A step of generating feature point attribute data of the plurality of feature points, wherein the feature point attribute data includes ratio, distance, and depth information of the plurality of feature points; and A step of generating the type data or the size data based on the feature point attribute data using the type and size data generating unit; including; How to recommend artificial teeth.

5. In paragraph 1, The above type of data is, characterized in that it includes at least one of facial type information, tooth type information, or arch type information, How to recommend artificial teeth.

6. In paragraph 1, The above feature data generation unit, the above type and size data generation unit, and the above recommended artificial tooth appearance data generation unit, characterized by being constructed based on multiple machine learning models, How to recommend artificial teeth.

7. In paragraph 6, The above multiple machine learning models are, A first machine learning model operating based on facial image data; A second machine learning model that operates based on oral image data; and A third machine learning model that operates based on dental image data; characterized in that it includes; How to recommend artificial teeth.

8. In paragraph 6, The above feature data generation unit, the above type and size data generation unit, and the above recommended artificial tooth appearance data generation unit, Characterized in that it is an ensemble model of the above multiple machine learning models, How to recommend artificial teeth.

9. In paragraph 7, The first machine learning model, the second machine learning model, and the third machine learning model are, characterized by being constructed using different types of machine learning models, How to recommend artificial teeth.

10. In paragraph 1, The above feature data generation unit, the type and size data generation unit, and the recommended artificial tooth appearance data generation unit are characterized in that they are different types of machine learning models. How to recommend artificial teeth.

11. A device for recommending a malignant artificial tooth for performing the method for recommending an artificial tooth according to Article 1.

12. A computer-readable storage medium storing a computer program for performing the artificial tooth recommendation method according to Article 1.

Images