Method, apparatus, and recording medium recording instructions for aligning 3D data of oral cavity with occlusal plane

Abstract

Disclosed is a technique for aligning three-dimensional data of an oral cavity to an occlusal plane. A method performed by an electronic device according to an aspect of the present disclosure may include the steps of: obtaining first data which is three-dimensional data for an oral cavity including a plurality of teeth; determining three-dimensional center coordinates of each of the plurality of teeth indicated by the first data; determining one or more target teeth among the plurality of teeth; determining one or more reference center points respectively corresponding to the one or more target teeth on an occlusal plane for aligning the plurality of teeth; and generating a transformation matrix for aligning the first data to the occlusal plane on the basis of three-dimensional center coordinates of each of the one or more target teeth and the one or more reference center points.

Description

Method, device and recording medium for aligning three-dimensional data of the oral cavity to the occlusal plane and recording command thereon

[0001] The present disclosure relates to a technique for aligning three-dimensional data of an oral cavity to an occlusal plane.

[0002] The fabrication of dental prosthetics and oral appliances requires accurate data on the patient's oral structure. In the past, physical impressions were primarily used to create models based on the patient's oral condition. However, technological advancements have led to the use of 3D oral scanners to capture 3D images of the patient's oral structure. These scanners project lasers or light beams from various angles and measure the degree to which these beams are reflected, converting the precise shape of the internal oral structures, including teeth and gingiva, into 3D data.

[0003] Using 3D data, CAD / CAM (Computer-Aided Design and Computer-Aided Manufacturing) systems can be used to design and manufacture prosthetics, crowns, bridges, implants, and orthodontic appliances tailored to each patient's oral condition. When performing various processes based on 3D data to design oral appliances, it is necessary to precisely align the 3D data with the occlusal plane to ensure that each process proceeds as intended.

[0004] A technical problem to be solved by at least one embodiment of the present disclosure is to provide a technology capable of precisely aligning three-dimensional data of the oral cavity to an occlusal plane.

[0005] A technical problem to be solved by at least one embodiment of the present disclosure is to provide a technique capable of generating a transformation matrix for aligning three-dimensional data to an occlusal plane.

[0006] A technical problem to be solved by at least one embodiment of the present disclosure is to provide a technology capable of precisely aligning three-dimensional data to an occlusal plane even if there are no corresponding tooth pairs in the three-dimensional data.

[0007] The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art of the present disclosure from the description below.

[0008] A method performed by an electronic device according to one aspect of the present disclosure may include: acquiring first data, which is three-dimensional data for an oral cavity including a plurality of teeth; determining three-dimensional center coordinates of each of the plurality of teeth represented by the first data; determining one or more target teeth among the plurality of teeth; determining one or more reference center points, each corresponding to one or more target teeth, in an occlusal plane for aligning the plurality of teeth; and generating a transformation matrix for aligning the first data to the occlusal plane based on the three-dimensional center coordinates of each of the one or more target teeth and the one or more reference center points.

[0009] In one embodiment, the step of determining at least one target tooth among the plurality of teeth may include the steps of: classifying each of the plurality of teeth into one of a first tooth group and a second tooth group; determining whether a first tooth pair that is located at a corresponding position relative to a midline of the oral cavity and belongs to the first tooth group exists; and determining whether a second tooth pair that is located at a corresponding position relative to the midline and belongs to the second tooth group exists.

[0010] In one embodiment, the method further comprises the step of determining an identification number of each of the plurality of teeth, and the step of determining whether the first tooth pair exists and the step of determining whether the second tooth pair exists may be based on the identification number of each of the plurality of teeth.

[0011] In one embodiment, the step of determining one or more target teeth among the plurality of teeth may include, in response to a determination that both the first tooth pair and the second tooth pair are present, determining four teeth included in the first tooth pair and the second tooth pair as the one or more target teeth.

[0012] In one embodiment, the step of determining at least one target tooth among the plurality of teeth may further include, when there are a plurality of tooth pairs that are located at corresponding positions with respect to the midline among the one or more teeth included in the first tooth group, the step of determining, as the first tooth pair, a tooth pair that includes a tooth that is closest in distance from the midline among the plurality of tooth pairs that are located at corresponding positions with respect to the midline.

[0013] In one embodiment, the step of determining at least one target tooth among the plurality of teeth may further include, when there are a plurality of tooth pairs that are located at corresponding positions with respect to the midline among the one or more teeth included in the second tooth group, the step of determining, as the second tooth pair, a tooth pair that includes a tooth that is closest in distance from the midline among the plurality of tooth pairs that are located at corresponding positions with respect to the midline.

[0014] In one embodiment, the step of generating the transformation matrix may include, in response to a determination that both the first tooth pair and the second tooth pair are present, generating the transformation matrix so as to align two or more straight lines derived based on the three-dimensional center coordinates of the four teeth with corresponding straight lines on the occlusal plane.

[0015] In one embodiment, the step of determining one or more target teeth among the plurality of teeth may include, in response to a determination that at least one of the first tooth pair or the second tooth pair is not present, determining all of the plurality of teeth as the one or more target teeth.

[0016] In one embodiment, the step of generating the transformation matrix may include, in response to a determination that at least one of the first tooth pair or the second tooth pair is not present, generating the transformation matrix so as to minimize an amount of movement of the oral cavity represented by the first data while moving the three-dimensional center coordinates of each of the one or more target teeth as close as possible to a corresponding reference center point of the occlusal plane.

[0017] In one embodiment, the step of determining the three-dimensional center coordinates of each of the plurality of teeth indicated by the first data may include the steps of: generating second data, which is two-dimensional data about the oral cavity, based on the first data; identifying each of the plurality of teeth in the second data; determining the two-dimensional center coordinates of each of the plurality of teeth; and determining the three-dimensional center coordinates of each of the plurality of teeth based on the two-dimensional center coordinates.

[0018] In one embodiment, the step of generating second data, which is two-dimensional data about the oral cavity, may include the steps of: identifying a tooth region indicated by the first data, wherein the tooth region includes the plurality of teeth; adding a color indicating curvature to each point of the tooth region; and generating the second data based on data about the tooth region to which the color is added among the first data.

[0019] In one embodiment, the step of determining the two-dimensional center coordinates of each of the plurality of teeth may include the steps of: identifying each of the plurality of teeth represented by the second data; generating a plurality of bounding boxes corresponding to each of the plurality of teeth, wherein the plurality of bounding boxes are in the shape of the smallest rectangles that include an area in which each of the plurality of teeth is located in the second data; and determining the center of each of the plurality of bounding boxes as the two-dimensional center coordinates of the corresponding tooth.

[0020] In one embodiment, the step of determining the two-dimensional center coordinates of each of the plurality of teeth includes the step of applying the second data to a learning model to obtain output data, wherein the output data can identify each of the plurality of teeth and represent the two-dimensional center coordinates of each of the plurality of teeth.

[0021] In one embodiment, the step of determining the three-dimensional center coordinates of each of the plurality of teeth may include: determining a plurality of first points corresponding to the two-dimensional center coordinates of each of the plurality of teeth and on a plane identical to or parallel to the occlusal plane; determining a plurality of second points on surfaces of the plurality of teeth corresponding to the plurality of first points, each of the plurality of second points being an intersection of a straight line passing through the corresponding first point and a normal line of the occlusal plane; and determining the plurality of second points as three-dimensional center coordinates of each of the plurality of teeth.

[0022] In one embodiment, the step of determining the three-dimensional center coordinates of each of the plurality of teeth may include the step of determining a plurality of third points, which are cusp points of each of the plurality of teeth, based on the plurality of second points; and the step of determining the plurality of third points as the three-dimensional center coordinates of each of the plurality of teeth.

[0023] In one embodiment, the step of determining the third point of each of the plurality of teeth may include the step of determining a point with a maximum curvature among points obtained by moving the plurality of second points in the buccal and lingual directions along the surfaces of each of the plurality of teeth as the third point of each of the plurality of teeth.

[0024] In one embodiment, the method may further include a step of aligning the first data to the occlusion plane based on the transformation matrix.

[0025] In one embodiment, the method may further include the steps of: dividing a surface of a plurality of teeth represented by the first data into one or more polygons; determining one or more first normal vectors for each of the one or more polygons; determining a second normal vector corresponding to the first data based on the one or more first normal vectors; determining a third normal vector that is a normal vector of the occlusal plane; and generating a rotation matrix for rotating the first data such that the second normal vector coincides with the third normal vector.

[0026] An electronic device according to one embodiment of the present disclosure comprises one or more processors, one or more memories storing instructions to be executed by the one or more processors, and when the instructions are executed by the one or more processors, the one or more processors may be configured to execute a method according to the present disclosure.

[0027] In one embodiment of the present disclosure, a non-transitory computer-readable recording medium having recorded thereon instructions that, when executed by one or more processors, cause the one or more processors to perform an operation, the instructions may be configured to cause the one or more processors to perform a method according to the present disclosure.

[0028] According to various embodiments of the present disclosure, three-dimensional data of the oral cavity can be precisely aligned to the occlusal plane.

[0029] According to various embodiments of the present disclosure, a transformation matrix for aligning three-dimensional data to an occlusal plane can be generated.

[0030] According to various embodiments of the present disclosure, three-dimensional data can be precisely aligned to an occlusal plane even if there are no corresponding tooth pairs in the three-dimensional data.

[0031] The effects according to the technical idea of ​​the present disclosure are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by a person skilled in the art from the description of the specification.

[0032] FIG. 1 is a drawing illustrating an example of obtaining an image of a patient's oral cavity using an oral scanner according to one embodiment of the present disclosure.

[0033] FIG. 2A is a block diagram of an electronic device and an oral scanner according to one embodiment of the present disclosure. FIG. 2B is a perspective view of an oral scanner according to one embodiment of the present disclosure.

[0034] FIG. 3 is a diagram illustrating a method for generating a three-dimensional image of an oral cavity according to one embodiment of the present disclosure.

[0035] FIG. 4 is a diagram showing the alignment of first data on an occlusal plane according to one embodiment of the present disclosure.

[0036] FIG. 5A illustrates first data acquired according to one embodiment of the present disclosure.

[0037] FIG. 5b illustrates first data rotated based on a normal vector of an occlusal plane according to one embodiment of the present disclosure.

[0038] FIG. 6A and FIG. 6B illustrate second data according to one embodiment of the present disclosure.

[0039] FIG. 7a and FIG. 7b illustrate inference results of a learning model for second data according to one embodiment of the present disclosure.

[0040] FIG. 8 is an example of an inference result of a learning model according to one embodiment of the present disclosure.

[0041] FIG. 9 is a drawing for explaining a process of determining the three-dimensional center coordinates of each of a plurality of teeth using a projection method according to one embodiment of the present disclosure.

[0042] FIG. 10A illustrates one or more reference center points on an occlusal plane according to one embodiment of the present disclosure.

[0043] FIG. 10b and FIG. 10c are drawings for explaining a process of aligning first data to an occlusal plane according to one embodiment of the present disclosure.

[0044] FIG. 11a illustrates full arch data before alignment to the occlusal plane according to one embodiment of the present disclosure.

[0045] FIG. 11b illustrates full arch data aligned to the occlusal plane according to one embodiment of the present disclosure.

[0046] FIG. 12a illustrates the arch data before alignment to the occlusal plane according to one embodiment of the present disclosure.

[0047] FIG. 12b illustrates the arch data aligned to the occlusal plane according to one embodiment of the present disclosure.

[0048] FIG. 13 is a flowchart of a method for aligning three-dimensional data of an oral cavity to an occlusal plane according to one embodiment of the present disclosure.

[0049] The various embodiments described in this document are exemplified for the purpose of clearly explaining the technical concept of the present disclosure and are not intended to limit it to a specific embodiment. The technical concept of the present disclosure includes various modifications, equivalents, alternatives, and embodiments selectively combining all or part of each embodiment described in this document. Furthermore, the scope of the technical concept of the present disclosure is not limited to the various embodiments presented below or the specific descriptions thereof.

[0050] Terms used in this document, including technical or scientific terms, unless otherwise defined, may have the meaning commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0051] The expressions "includes," "may include," "comprises," "may have," "have," and "may have" used in this document imply the presence of a target feature (e.g., a function, operation, or component), but do not exclude the presence of other additional features. In other words, such expressions should be understood as open-ended terms that imply the possibility of including other embodiments.

[0052] The singular forms used in this document may include the plural form unless the context clearly indicates otherwise, and this also applies to the singular forms set forth in the claims.

[0053] The expressions "first," "second," or "first", "second", etc. used in this document, unless the context indicates otherwise, are used to refer to multiple similar objects and to distinguish one object from another, and do not limit the order or importance among the objects.

[0054] As used herein, the expressions "A, B, and C", "A, B, or C", "A, B, and / or C", or "at least one of A, B, and C", "at least one of A, B, or C", "at least one of A, B, and / or C", etc., may refer to each of the listed items or to all possible combinations of the listed items. For example, "at least one of A or B" may refer to (1) at least one A, (2) at least one B, (3) at least one A and at least one B.

[0055] The expression "based on" as used in this document is used to describe one or more factors that influence the decision, act of judgment, or action described in the phrase or sentence containing the expression, and this expression does not exclude additional factors that influence the decision, act of judgment, or action.

[0056] In the present disclosure, determining C information based on A information and B information means taking A information and B information into consideration in determining C information, and does not exclude that other information besides A information and B information is additionally considered.

[0057] As used herein, the expression that a component (e.g., a first component) is “connected” or “connected” to another component (e.g., a second component) may mean that the component is directly connected or connected to the other component, as well as connected or connected via a new other component (e.g., a third component).

[0058] The expression "configured to" as used in this document can mean "set to do", "having the ability to do", "modified to do", "made to do", "capable of doing", etc., depending on the context. The expression is not limited to the meaning of "specifically designed in hardware", and for example, a processor configured to perform a specific operation can mean a generic-purpose processor that can perform the specific operation by executing software.

[0059] In the present disclosure, a "learning model" may be designed to implement the structure of a human brain on a computer, and may include a plurality of network nodes that simulate neurons of a human neural network and have weights. The plurality of network nodes simulate the synaptic activity of neurons that exchange signals through synapses and may have connections among themselves. In the learning model, the plurality of network nodes may be located at layers of different depths and exchange data according to convolutional connections. For example, the learning model may be an artificial neural network model, a time series analysis model, etc. Here, the time series analysis model is a model that analyzes time series data, and may be a recurrent neural network (RNN) model, a long short-term memory (LSTM) model, or an Autoregressive Integrated Moving Average (ARIMA) model. Meanwhile, the present disclosure is not limited thereto, and various types of models for analyzing data may be used.

[0060] In the present disclosure, a “training process” may mean a process in which a learning model extracts and analyzes features (patterns) of input data and output data pairs of learning data, repeats the process of deriving correlations between input and output data, and optimizes parameters of the learning model based on the correlations between input and output data.

[0061] In the present disclosure, an “inference process” may mean a process in which a learning model applies a previously learned pattern to new input data to generate output data as a result of prediction or classification of the input data.

[0062] The directional terms "upper" and "upper" used in this document, unless otherwise specifically defined in the description, refer to the direction in which the occlusal plane is positioned relative to the teeth in the attached drawings, and the directional terms "lower" and "lower" refer to the opposite direction. The multiple teeth included in the oral cavity depicted in the attached drawings may be included in the upper jaw or the lower jaw, and the directional terms may be interpreted accordingly.

[0063] Hereinafter, various embodiments of the present disclosure will be described with reference to the attached drawings. In the attached drawings and the description of the drawings, identical or substantially equivalent components may be assigned the same reference numerals. Furthermore, in the description of various embodiments below, duplicate descriptions of identical or corresponding components may be omitted, but this does not mean that the corresponding components are not included in the embodiments.

[0064] FIG. 1 is a drawing illustrating a method of obtaining an image of a patient's oral cavity using a three-dimensional scanner (200) according to one embodiment of the present disclosure. According to one embodiment, the three-dimensional scanner (200) may be a dental medical device for obtaining an image of the oral cavity of a subject (20). For example, the three-dimensional scanner (200) may be an intraoral scanner. As illustrated in FIG. 1, a user (10) (e.g., a dentist or a dental hygienist) may obtain an image of the oral cavity of a subject (20) (e.g., a patient) from the subject (20) using the three-dimensional scanner (200). As another example, the user (10) may obtain an image of the oral cavity of the subject (20) from a diagnostic model (e.g., a plaster model or an impression model) modeled after the shape of the oral cavity of the subject (20). Hereinafter, for convenience of explanation, the oral cavity of the subject (20) is scanned to obtain an image of the oral cavity of the subject (20), but the present invention is not limited thereto, and it is also possible to obtain an image of another part of the subject (20) (e.g., the ear of the subject (20)). The 3D scanner (200) may have a form that can be inserted and extracted into the oral cavity, and may be a handheld scanner in which the user (10) can freely adjust the scanning distance and scanning angle.

[0065] According to one embodiment, a three-dimensional scanner (200) can be inserted into the oral cavity of a subject (20) and scan the inside of the oral cavity in a non-contact manner to obtain an image of the oral cavity. The image of the oral cavity may include at least one tooth and gums. The three-dimensional scanner (200) can irradiate light onto the oral cavity of the subject (20) (e.g., at least one tooth and gums of the subject (20)) using a light source (or projector), and can receive light reflected from the oral cavity of the subject (20) through a camera (or at least one image sensor). According to another embodiment, the three-dimensional scanner (200) can also obtain an image of a diagnostic model of the oral cavity by scanning the diagnostic model of the oral cavity. If the diagnostic model of the oral cavity is a diagnostic model modeled after the shape of the oral cavity of the subject (20), the image of the diagnostic model of the oral cavity may be an image of the oral cavity of the subject. In the following, for convenience of explanation, it is assumed that an image of the oral cavity is obtained by scanning the inside of the oral cavity of the subject (20), but the present invention is not limited thereto.

[0066] According to one embodiment, a three-dimensional scanner (200) can acquire a surface image of the oral cavity of a subject (20) as a two-dimensional image based on information received through a camera. The surface image of the oral cavity of the subject (20) can include at least one tooth, gums, an artificial structure, a cheek, a tongue, or lips of the subject (20). The surface image of the oral cavity of the subject (20) can be a two-dimensional image.

[0067] According to one embodiment, a two-dimensional image of the oral cavity acquired from a three-dimensional scanner (200) may be transmitted to an electronic device (100) connected via a wired or wireless communication network. The electronic device (100) may be a computer device or a portable communication device. The electronic device (100) may generate a three-dimensional image of the oral cavity (or may also be referred to as a three-dimensional oral cavity image, a three-dimensional oral cavity model, a three-dimensional scan model, a tooth model, a three-dimensional oral cavity representation, a three-dimensional numerical entity, etc.) that represents the oral cavity three-dimensionally based on the two-dimensional image of the oral cavity received from the three-dimensional scanner (200). Hereinafter, the three-dimensional image may be used interchangeably with the first data in this document. The electronic device (100) may generate a three-dimensional image of the oral cavity by three-dimensionally modeling the internal structure of the oral cavity based on the two-dimensional image of the oral cavity received.

[0068] A 3D image can contain not only geometric information of an object within its 3D space, but also information such as color, texture, and material. The format of a 3D image can be, for example, STL (Standard Triangle Language), OBJ, or polygon file format, but is not limited to these examples. "Polygon" can refer to a polygon, the smallest unit used to express the three-dimensional shape of a 3D oral model. For example, the surface of a 3D image can be expressed with triangular or quadrilateral polygons. For example, a polygon can be composed of at least three vertices and one face. A 3D image can contain information such as the position, color, and normal in 3D space for each vertex. A mesh is made up of multiple polygons and can represent the geometry of an object in 3D space. As the number of polygons representing a 3D image increases, the object can be expressed in greater detail.

[0069] According to another embodiment, a 3D scanner (200) scans the oral cavity of a subject (20) to obtain a 2D image of the oral cavity, generates a 3D image of the oral cavity based on the obtained 2D image of the oral cavity, and may transmit the generated 3D image of the oral cavity to an electronic device (100).

[0070] An electronic device (100) according to one embodiment may be connected to a cloud server (not shown). In the above case, the electronic device (100) may transmit a two-dimensional image of the oral cavity of the subject (20) or a three-dimensional image of the oral cavity to the cloud server, and the cloud server may store the two-dimensional image of the oral cavity of the subject (20) or the three-dimensional image of the oral cavity received from the electronic device (100).

[0071] According to another embodiment, in addition to a handheld scanner that is inserted into the oral cavity of a subject (20) and used, a table scanner (not shown) that is fixed and used at a specific location may be used as a 3D scanner. The table scanner can create a 3D image of the oral cavity diagnostic model by scanning the oral cavity diagnostic model. In the above case, since the light source (or projector) and the camera of the table scanner are fixed, the user can scan the oral cavity diagnostic model by moving the arm that fixes the oral cavity diagnostic model. Compared to a handheld scanner, a table scanner is relatively less likely to cause noise by interfering with another object between the camera and the diagnostic model during the scanning operation. However, embodiments of the present disclosure are applicable not only to a handheld scanner but also to a table scanner and other 3D scanners.

[0072] FIG. 2A is a block diagram of an electronic device (100) and a 3D scanner (200) according to one embodiment of the present disclosure. The electronic device (100) and the 3D scanner (200) can be connected to each other via a wired or wireless communication network and can transmit and receive various data.

[0073] According to one embodiment, a 3D scanner (200) may include a processor (201), a memory (202), a communication circuit (203), a light source (204), a camera (205), an input device (206), and / or a sensor module (207). At least one of the components included in the 3D scanner (200) may be omitted, or another component may be added to the 3D scanner (200). Additionally or alternatively, some of the components may be implemented in an integrated manner, or may be implemented as a single or multiple entities. At least some of the components within the 3D scanner (200) may be connected to each other via a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI), and may exchange data and / or signals.

[0074] According to one embodiment, a processor (201) of a 3D scanner (200) is a configuration capable of performing calculations or data processing related to control and / or communication of each component of the 3D scanner (200), and may be operatively connected to the components of the 3D scanner (200). The processor (201) may load commands or data received from other components of the 3D scanner (200) into a memory (202), process commands or data stored in the memory (202), and store result data. The memory (202) of the 3D scanner (200) according to one embodiment may store instructions for the operation of the processor (201) described above.

[0075] According to one embodiment, the communication circuit (203) of the 3D scanner (200) can establish a wired or wireless communication channel with an external device (e.g., an electronic device (100)) and transmit and receive various data with the external device. According to one embodiment, the communication circuit (203) can include at least one port for connecting to the external device via a wired cable in order to communicate with the external device via a wire. In the above case, the communication circuit (203) can communicate with the external device connected via a wire through the at least one port. According to one embodiment, the communication circuit (203) can be configured to include a cellular communication module and be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). According to one embodiment, the communication circuit (203) can include a short-range communication module and transmit and receive data with the external device using short-range communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB), but is not limited thereto. According to one embodiment, the communication circuit (203) may include a contactless communication module for contactless communication. The contactless communication may include at least one contactless proximity communication technology, such as, for example, near field communication (NFC), radio frequency identification (RFID), or magnetic secure transmission (MST).

[0076] According to one embodiment, the light source (204) of the 3D scanner (200) can irradiate light toward the oral cavity of the subject (20). For example, the light irradiated from the light source (204) can be structured light having a predetermined pattern (e.g., a stripe pattern in which straight lines of different colors appear continuously). The pattern of the structured light can be generated, for example, using a pattern mask or a digital micro-mirror device (DMD), but is not limited thereto. The camera (205) of the 3D scanner (200) according to one embodiment can obtain an image of the oral cavity of the subject (20) by receiving reflected light reflected by the oral cavity of the subject (20). The camera (205) can include, for example, a left camera corresponding to the left-eye field of view and a right camera corresponding to the right-eye field of view in order to construct a 3D image according to an optical triangulation method. The camera (205) may include at least one image sensor, such as a CCD sensor or a CMOS sensor.

[0077] An input device (206) of a 3D scanner (200) according to one embodiment can receive user input for controlling the 3D scanner (200). The input device (206) can include a button for receiving a push operation from a user (10), a touch panel for detecting a touch from a user (10), and a voice recognition device including a microphone. For example, the user (10) can control the start or stop of scanning using the input device (206).

[0078] According to one embodiment, the sensor module (207) of the 3D scanner (200) can detect the operating state of the 3D scanner (200) or the external environmental state (e.g., the user's movement) and generate an electrical signal corresponding to the detected state. The sensor module (207) can include, for example, at least one of a gyro sensor, an acceleration sensor, a gesture sensor, a proximity sensor, or an infrared sensor. The user (10) can control the start or stop of scanning using the sensor module (207). For example, when the user (10) holds the 3D scanner (200) in his / her hand and moves it, the 3D scanner (200) can control the processor (201) to start a scanning operation when the angular velocity measured by the sensor module (207) exceeds a preset threshold value.

[0079] According to one embodiment, the three-dimensional scanner (200) may receive a user input for starting a scan through an input device (206) of the three-dimensional scanner (200) or an input device (109) of the electronic device (100), or may start a scan according to processing in a processor (201) of the three-dimensional scanner (200) or a processor (101) of the electronic device (100). When a user (10) scans the inside of an oral cavity of a subject (20) through the three-dimensional scanner (200), the three-dimensional scanner (200) may generate a two-dimensional image of the oral cavity of the subject (20) and transmit the two-dimensional image of the oral cavity of the subject (20) to the electronic device (100) in real time. The electronic device (100) may display the received two-dimensional image of the oral cavity of the subject (20) through a display (107). In addition, the electronic device (100) can generate (build) a three-dimensional image of the oral cavity of the object (20) based on a two-dimensional image of the oral cavity of the object (20), and can display the three-dimensional image of the oral cavity through a display (107). The electronic device (100) can also display the three-dimensional image being generated in real time through the display (107).

[0080] An electronic device (100) according to one embodiment may include one or more processors (101), one or more memories (103), a communication circuit (105), a display (107), and / or an input device (109). At least one of the components included in the electronic device (100) may be omitted, or another component may be added to the electronic device (100). Additionally or alternatively, some of the components may be implemented in an integrated manner, or may be implemented as a single or multiple entities. At least some of the components within the electronic device (100) may be connected to each other via a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI), and may exchange data and / or signals.

[0081] According to one embodiment, one or more processors (101) of the electronic device (100) may be configured to perform operations or data processing related to control and / or communication of each component (e.g., memory (103)) of the electronic device (100). The one or more processors (101) may be operatively connected to, for example, components of the electronic device (100). The one or more processors (101) may load commands or data received from other components of the electronic device (100) into one or more memories (103), process commands or data stored in the one or more memories (103), and store result data.

[0082] According to one embodiment, one or more memories (103) of the electronic device (100) can store instructions for the operation of one or more processors (101). One or more memories (103) can store data received from a three-dimensional scanner (200) (e.g., a two-dimensional image of the oral cavity obtained through an oral scan).

[0083] According to one embodiment, the communication circuit (105) of the electronic device (100) can establish a wired or wireless communication channel with an external device (e.g., a 3D scanner (200), a cloud server) and transmit and receive various data with the external device. According to one embodiment, the communication circuit (105) can include at least one port for connecting to the external device via a wired cable in order to communicate with the external device via a wire. In the above case, the communication circuit (105) can perform communication with the external device connected via a wire through the at least one port. According to one embodiment, the communication circuit (105) can be configured to include a cellular communication module and be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). According to one embodiment, the communication circuit (105) can include a short-range communication module and transmit and receive data with the external device using short-range communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB), but is not limited thereto. According to one embodiment, the communication circuit (105) may include a contactless communication module for contactless communication. The contactless communication may include at least one contactless proximity communication technology, such as, for example, near field communication (NFC), radio frequency identification (RFID), or magnetic secure transmission (MST).

[0084] According to one embodiment, the display (107) of the electronic device (100) can display various screens based on the control of the processor (101). The processor (101) can display a two-dimensional image of the oral cavity of the object (20) received from the three-dimensional scanner (200) and / or a three-dimensional image of the oral cavity that is a three-dimensional model of the internal structure of the oral cavity through the display (107). For example, the two-dimensional image and / or the three-dimensional image of the oral cavity can be displayed through a specific application program. In the above case, the user (10) can edit, save, and delete the two-dimensional image and / or the three-dimensional image of the oral cavity.

[0085] An input device (109) of an electronic device (100) according to one embodiment can receive commands or data to be used in components of the electronic device (100) (e.g., one or more processors (101)) from an external source (e.g., a user) of the electronic device (100). The input device (109) may include, for example, a microphone, a mouse, or a keyboard. According to one embodiment, the input device (109) may be implemented in the form of a touch sensor panel that is coupled with a display (107) and can recognize contact or proximity of various external objects.

[0086] FIG. 2B is a perspective view of a 3D scanner (200) according to one embodiment of the present disclosure. The 3D scanner (200) according to one embodiment may include a main body (210) and a probe tip (220). The main body (210) of the 3D scanner (200) may be formed in a shape that is easy for a user (10) to grip by hand and use. The probe tip (220) may be formed in a shape that is easy to insert into and withdraw from the oral cavity of the subject (20). In addition, the main body (210) may be coupled to and separated from the probe tip (220). Components of the 3D scanner (200) described in FIG. 2A may be arranged inside the main body (210). An opening may be formed at one end of the main body (210) so that light output from a light source (204) may be irradiated to the subject (20). Light irradiated through the opening may be reflected by the object (20) and then re-entered through the opening. The reflected light entering through the opening may be captured by the camera (205) to generate an image of the object (20). The user (10) may initiate scanning using an input device (206) (e.g., a button) of the 3D scanner (200). For example, when the user (10) touches or presses the input device (206), light from the light source (204) may be irradiated to the object (20).

[0087] FIG. 3 is a diagram illustrating a method for generating a three-dimensional image (320) of an oral cavity according to one embodiment of the present disclosure. A user (10) can scan the inside of an oral cavity of a subject (20) by moving a three-dimensional scanner (200), and in this case, the three-dimensional scanner (200) can obtain a plurality of two-dimensional images (310) of the oral cavity of the subject (20). For example, the three-dimensional scanner (200) can obtain a two-dimensional image of an area including the front teeth of the subject (20), a two-dimensional image of an area including the molars of the subject (20), etc. The three-dimensional scanner (200) can transmit the obtained multiple two-dimensional images (310) to an electronic device (100). According to another embodiment, the user (10) can scan a diagnostic model of an oral cavity and obtain a plurality of two-dimensional images of the diagnostic model of the oral cavity by moving the three-dimensional scanner (200). In the following, for convenience of explanation, it is assumed that an image of the oral cavity of the subject (20) is obtained by scanning the inside of the oral cavity of the subject (20), but the present invention is not limited thereto.

[0088] An electronic device (100) according to one embodiment can convert each of a plurality of two-dimensional images (310) of the oral cavity of a subject (20) into a set of a plurality of points having three-dimensional coordinate values. For example, the electronic device (100) can convert each of a plurality of two-dimensional images (310) into a point cloud, which is a set of data points having three-dimensional coordinate values. For example, a set of point clouds, which are three-dimensional coordinate values ​​based on a plurality of two-dimensional images (310), can be stored as raw data for the oral cavity of the subject (20). The electronic device (100) can complete a full tooth model by aligning the point cloud, which is a set of data points having three-dimensional coordinate values.

[0089] Hereinafter, the operation of aligning the three-dimensional data for the oral cavity to the occlusal plane in FIGS. 4 to 13 can be performed by the processor (101) of the electronic device (100).

[0090] FIG. 4 is a diagram showing three-dimensional data for the oral cavity aligned on an occlusal plane according to one embodiment of the present disclosure.

[0091] The processor can obtain first data (410), which is three-dimensional data of the patient's oral cavity (e.g., the three-dimensional image of FIG. 1). By aligning the first data (410) to the occlusal plane (400), the processor can precisely perform subsequent processes for designing an oral appliance for prosthetic treatment of the patient. The occlusal plane (400) refers to a virtual plane established based on the occlusion of the maxillary and mandibular teeth in the oral cavity. For example, the processor can perform various processes for designing a prosthesis, crown, bridge, implant, orthodontic device, etc., having a shape suitable for the patient's oral structure, using the first data (410) aligned to the occlusal plane (400). Hereinafter, a method for aligning the first data (410) to the occlusal plane (400) will be described in detail with reference to FIGS. 5 to 13.

[0092] FIG. 5A illustrates first data acquired according to one embodiment of the present disclosure.

[0093] FIG. 5b illustrates first data rotated based on a normal vector of an occlusal plane according to one embodiment of the present disclosure.

[0094] In one embodiment, the first data (510a, 510b) may be data about a certain area of ​​the patient's oral cavity. In one embodiment, the first data (510a, 510b) may be data about an area of ​​the patient's oral cavity into which a prosthesis is to be inserted. For example, the first data (510a, 510b) may be data about the lower jaw (or data about the upper jaw), or data about an area of ​​the lower jaw data that includes some teeth. In one embodiment, the first data (510a, 510b) may be in a polygon file format. In this case, the surface represented by the first data (510a, 510b) may be expressed as one or more triangles or squares.

[0095] In one embodiment, the processor may determine a normal vector represented by the first data (510a, 510b). The processor may determine the normal vector represented by the first data (510a, 510b) based on a normal vector of each of one or more polygons forming the surface represented by the first data (510a, 510b). For example, the processor may identify a tooth region in the oral cavity represented by the first data (510a, 510b), determine a normal vector of each of one or more polygons within the tooth region, and determine a sum of the respective normal vectors as the normal vector represented by the first data (510a, 510b). In another embodiment, the processor may determine a normal vector of the surface represented by the first data (510a, 510b) based on a central line represented by the first data (510a, 510b). The centerline may refer to a curve formed along the arch curve represented by the first data (510a, 510b). The arch curve represented by the first data (510a, 510b) may be determined in any manner based on the positions of each of the plurality of teeth represented by the first data. The processor may define a plane including the curve formed by the arch curve, and determine the normal vector of the defined plane as the normal vector represented by the first data (510a, 510b).

[0096] According to one embodiment, the first data (510a) may be arranged in a three-dimensional space in a state that is twisted with respect to the occlusal plane (500). The twisted state refers to a state in which a normal vector of the occlusal plane (500) and a normal vector indicated by the first data (510a) do not match. At this time, the occlusal plane (500) may be arranged parallel to a screen plane on which the occlusal plane (500) and the first data (510a) are displayed, and a normal direction of the occlusal plane (500) may match a normal direction of the screen plane. Fig. 5a illustrates the first data (510a) arranged in a three-dimensional space in a state that is twisted with respect to the occlusal plane (500).

[0097] The processor can generate a rotation matrix to align the normal vector indicated by the first data (510a) with the normal vector of the occlusion plane (500). The processor can calculate the normal vector of the occlusion plane (500) and generate a rotation matrix for rotating the first data (510a) based on the normal vector indicated by the first data (510a) and the normal vector of the occlusion plane (500). Based on the generated rotation matrix, the processor can rotate the first data (510a) so that the surface indicated by the first data (510a) faces the direction in which the occlusion plane (500) is facing. Referring to FIG. 5b, the processor generates a rotation matrix for rotating the first data (510b) so that the surface indicated by the acquired first data (510b) faces the direction in which the occlusion plane (500) is facing, and can rotate the first data (510b) using the rotation matrix so that the normal vector indicated by the first data (510b) matches the normal vector of the occlusion plane (500). Hereinafter, a description will be given assuming a state in which the normal vector indicated by the first data (510a, 510b) matches the normal vector of the occlusion plane (500).

[0098] FIG. 6A and FIG. 6B illustrate second data according to one embodiment of the present disclosure.

[0099] The processor can generate second data (600, 610), which is two-dimensional data about the oral cavity, based on the first data, which is three-dimensional data. According to one embodiment, the first data may include data about teeth as well as data about gums, and the processor can separate the tooth region and the gum region of the oral cavity represented by the first data and extract the first data corresponding to the tooth region.

[0100] The processor can add a color indicating the curvature of each point to the first data for the extracted tooth region. The curvature of the tooth may refer to the degree of curvature of the tooth surface. In one embodiment, the curvature of the tooth surface may be used by the learning model to identify the tooth number of each tooth.

[0101] The processor can capture a three-dimensional image represented by the first data to generate second data (600, 610). For example, the processor can generate second data (600, 610) representing a two-dimensional image captured by looking at a three-dimensional image of a tooth region extracted from the first data and having a curvature indicated in the direction of a normal vector of the occlusal plane (i.e., exposed on a display screen). Referring to FIGS. 6A and 6B, the processor can generate second data (600, 610) based on the first data. FIG. 6A may be second data (600) for full arch data, and FIG. 6B may be second data (610) for partial arch data. Full arch data refers to data representing the entire maxillary or mandibular arch, and partial arch data refers to data representing only a portion of the maxillary arch. As illustrated in FIGS. 6A and 6B , the second data (600, 610) may further include colors indicating the curvature of each point. In the examples of FIGS. 6A and 6B , as the color of each tooth changes from blue to yellow and from yellow to red, the relative size of the curvature of the corresponding point may increase.

[0102] In one embodiment, the processor may determine the size of the image corresponding to the second data to a predetermined size. For example, the processor may determine the size of the two-dimensional image represented by the second data to be the size of the two-dimensional image represented by the input data of the learning model (e.g., 420 pixels wide and 380 pixels high), which will be described later.

[0103] FIG. 7a and FIG. 7b illustrate inference results of a learning model for second data according to one embodiment of the present disclosure.

[0104] The processor can identify multiple teeth included in the second data using a learning model and determine the two-dimensional center coordinates of each tooth. The learning model may be an artificial neural network model, and can receive second data about the oral cavity and perform an inference process to identify the teeth represented by the second data. Based on the inference results (700, 710) of the learning model, the processor can identify each of the multiple teeth represented by the second data.

[0105] In one embodiment, the processor may use the learning model to determine an identification number for each of the plurality of teeth represented by the second data. The identification numbers may be determined based on any system for classifying the plurality of teeth (e.g., Federation Dentaire International (FDI) or a universal numbering system). For example, the learning model may receive the second data and infer the numbers of the plurality of teeth represented by the second data based on at least one of colors indicating the shape and curvature of the plurality of teeth represented by the second data. The processor may identify the plurality of teeth represented by the second data based on the inference results (700, 710) of the learning model including the identification numbers of each tooth.

[0106] In one embodiment, the processor may determine the two-dimensional center coordinates of each of the plurality of teeth represented by the second data. The two-dimensional center coordinates may refer to the center of the teeth in the state represented by the second data. The processor may determine the two-dimensional center coordinates of each of the plurality of teeth based on a bounding box. For example, the processor may generate a bounding box for each of the plurality of teeth. The bounding box refers to the smallest rectangle that includes an area in which each of the plurality of teeth is located. The processor may determine the center of each of the plurality of bounding boxes as the two-dimensional center coordinates of the corresponding tooth. For example, the processor may determine the intersection of two diagonals of the bounding boxes as the center of the bounding box. In one embodiment, the processor may determine the two-dimensional center coordinates of each of the plurality of teeth using a learning model.

[0107] According to one embodiment, the learning model can be trained using a plurality of two-dimensional images of the oral cavity as input data, and data including a tooth number and two-dimensional center coordinates for each of a plurality of teeth included in each image as label data.

[0108] Fig. 7a is an inference result (700) obtained by inputting the second data corresponding to Fig. 6a into the learning model, and Fig. 7b is an inference result (710) obtained by inputting the second data corresponding to Fig. 6b into the learning model. Hereinafter, the description is made assuming that the learning model determines the tooth identification number based on the International Dental Notation (FDI), but it should not be interpreted that embodiments in which the learning model determines the tooth identification number by other methods are excluded.

[0109] Referring to FIG. 7a, the inference result (700) indicates the identification numbers of each of the plurality of teeth represented by the second data (in order from the left, 17, 14, 13, 12, 11, 21, 22, 23, 24, 26, 27). Based on this inference result (700), the processor can determine that the second data is full arch data for the maxilla. Referring to FIG. 7b, the inference result (710) can indicate the identification numbers of each of the plurality of teeth represented by the second data (in order from the left, 17, 16, 15, 14, 13, 12, 11). Based on this inference result (710), the processor can determine that the second data is partial arch data for the right maxilla.

[0110] FIG. 8 is an example of output data of a learning model according to one embodiment of the present disclosure.

[0111] Referring to FIG. 8, the learning model can output output data (800) including the identification number and two-dimensional center coordinates of each of the plurality of teeth included in the second data. For example, the output data (800) includes a central incisor (identification number 11), a lateral incisor (identification number 12), a canine (identification number 13), a first premolar (identification number 14), and a second premolar (identification number 15) of the right maxilla in a two-dimensional image represented by the second data, and a central incisor (identification number 21), a lateral incisor (identification number 22), a canine (identification number 23), and a first premolar (identification number 24) of the left maxilla are identified, and the two-dimensional center coordinate of the central incisor (identification number 11) of the right maxilla is (217, 120), the two-dimensional center coordinate of the lateral incisor (identification number 12) is (173, 134), the two-dimensional center coordinate of the canine (identification number 13) is (140, 157), and the two-dimensional center coordinate of the first premolar (identification number 14) The coordinates can be expressed as (121, 196), the two-dimensional center coordinates of the second premolar (identification number 15) are (101, 236), the two-dimensional center coordinates of the left maxillary central incisor (identification number 21) are (267, 127), the two-dimensional center coordinates of the lateral incisor (identification number 22) are (306, 152), the two-dimensional center coordinates of the canine (identification number 23) are (332, 188), and the two-dimensional center coordinates of the first premolar (identification number 24) are (340, 226).

[0112] The processor can determine the identification number and two-dimensional center coordinates of each of the plurality of teeth included in the second data based on the output data of the learning model corresponding to the second data.

[0113] FIG. 9 is a drawing for explaining a process of determining the three-dimensional center coordinates of each of a plurality of teeth using a projection method according to one embodiment of the present disclosure.

[0114] According to one embodiment, the processor can determine three-dimensional center coordinates corresponding to two-dimensional center coordinates of each of the plurality of teeth. For example, the processor can determine a plurality of first points (P1) on an arbitrary plane that is identical to or parallel to the occlusal plane and that correspond to two-dimensional center coordinates of a two-dimensional image represented by the second data. Next, the processor can determine three-dimensional center coordinates of the plurality of teeth according to a ray projection method based on the plurality of first points (P1). For example, the processor can determine, for each of the plurality of first points (P1), a projection line (L) that is parallel to a normal to the occlusal plane and passes through the first points (P1). p ) and define the corresponding straight line (L p ) and the intersection points of the surfaces of each of the plurality of teeth can be determined as a plurality of second points (P2). The projection line (L p ) and the intersection points of the surfaces of each tooth exist one each at the lower (P2') and upper (P2) sides of the tooth, the processor can determine the occlusal intersection point located at the upper side as the second point (P2). The processor can determine a plurality of second points (P2) as the three-dimensional center coordinates of each of the plurality of teeth.

[0115] In another embodiment, the processor may determine the cusp point of each of the plurality of teeth as the three-dimensional center coordinates of each of the plurality of teeth. For example, the processor may determine the point with the maximum curvature among the points obtained by moving the plurality of second points (P2) along the surfaces of each of the plurality of teeth in the buccal and lingual directions, as the third point (P3). The processor may determine the third point (P3) as the three-dimensional center coordinates of each of the plurality of teeth. The method for finding the cusp point of a tooth in the present disclosure is not limited to what has been described above, and various methods for determining the cusp point may be considered regardless of the second point (P2).

[0116] FIG. 10a illustrates reference center points on an occlusal plane according to one embodiment of the present disclosure.

[0117] The processor may align the first data to the occlusal plane (1000) based on the three-dimensional center coordinates of each of the plurality of teeth indicated by the first data and the reference center points corresponding to each of the plurality of teeth on the occlusal plane (1000). According to one embodiment, the reference center points on the occlusal plane (1000) may be determined in the same manner as the two-dimensional center coordinates of the plurality of teeth are determined. Referring to FIG. 10, the processor may determine a reference center point (1012) for each of the maxillary teeth and the mandibular teeth that occlude at a first position (1010) on the occlusal plane (1000). In FIG. 10, among the two reference center points of each tooth position, the point (hollow point) located further lingually represents the reference center point for the mandibular teeth, and the other point (solid point) represents the reference center point for the maxillary teeth. The processor can determine a transformation matrix for aligning the first data to the occlusal plane (1000) based on reference center points corresponding to a plurality of teeth on the occlusal plane (1000).

[0118] According to one embodiment, the processor may determine one or more target teeth that serve as a reference for determining a transformation matrix. The processor may classify a plurality of teeth indicated by the first data into either a first tooth group or a second tooth group. For example, the processor may classify each of the plurality of teeth indicated by the first data into the first tooth group if the corresponding tooth is an anterior tooth (e.g., a central incisor, a lateral incisor, a canine), and may classify each of the plurality of teeth into the second tooth group if the corresponding tooth is a posterior tooth (e.g., a premolar, a molar). The sum of the number of teeth in the first tooth group and the number of teeth in the second tooth group may be equal to the number of the plurality of teeth indicated by the first data. The number of teeth in the first tooth group may be 0 or greater. The number of teeth in the second tooth group may be 0 or greater.

[0119] In one embodiment, the processor may determine one or more target teeth based on whether a pair of teeth corresponding to each other with respect to the midline of the oral cavity exists in each of the first tooth group and the second tooth group. In one embodiment, the pair of teeth corresponding to each other with respect to the midline may refer to a pair of teeth that are located in different quadrants according to the FDI notation and thus have different tens digits of the tooth numbers but the same ones digit of the tooth numbers. For example, a central incisor (tooth number 11) in quadrant 1 (upper right) and a central incisor (tooth number 21) in quadrant 2 (upper left) may be a pair of teeth corresponding to each other, and a canine (tooth number 33) in quadrant 3 (lower left) and a canine (tooth number 43) in quadrant 4 (lower right) may be a pair of teeth corresponding to each other.

[0120] In one embodiment, the processor may determine whether a first tooth pair exists in a first tooth group and whether a second tooth pair exists in a second tooth group. If the first tooth pair and the second tooth pair exist, the processor may determine the teeth included in the first tooth pair and the second tooth pair (i.e., four teeth) as target teeth. If the first tooth pair or the second tooth pair does not exist, the processor may determine all teeth indicated by the first data as target teeth.

[0121] According to one embodiment, if there are multiple pairs of teeth in the first tooth group that are located at corresponding positions, the processor may determine the pair of teeth that includes the teeth that are closest in distance from the midline as the first tooth pair. For example, if the pairs of teeth that are located at corresponding positions in the first tooth group are (tooth 11, tooth 21), (tooth 12, tooth 22), and (tooth 13, tooth 23), the processor may determine the pair of teeth (tooth 11, tooth 21) as the first tooth pair. By determining the pair of teeth that includes the teeth that are closest in distance from the midline as the first tooth pair, the processor may more precisely align the center line when aligning the first data and the occlusal plane.

[0122] According to one embodiment, if there are multiple tooth pairs in corresponding positions in the second tooth group, the processor may determine two teeth as the second tooth pair based on any criterion. For example, the processor may set priorities for molar teeth and determine a tooth pair including a tooth with the highest priority among the multiple tooth pairs as the second tooth pair. For example, the processor may set priorities in the order of (tooth 16, tooth 26), (tooth 17, tooth 27), (tooth 18, tooth 28), (tooth 15, tooth 25), (tooth 14, tooth 24). For example, if the tooth pairs of (tooth 17, tooth 27), (tooth 18, tooth 28), and (tooth 14, tooth 24) all exist in the second tooth group, the processor may determine (tooth 17, tooth 27) with the highest priority among them as the second tooth pair. In one embodiment, teeth (teeth 16, 26) that are generally known to have the deepest roots among the teeth and thus to move the least during orthodontic treatment may have the highest priority. The method by which the processor determines one of a plurality of tooth pairs located in corresponding positions in the second tooth group as the second tooth pair is not limited to what has been described above, and the processor may determine the second tooth pair in the same manner as it determines the first tooth pair.

[0123] FIG. 10b and FIG. 10c are drawings for explaining a process of aligning first data to an occlusal plane according to one embodiment of the present disclosure.

[0124] According to one embodiment, when both a first tooth pair and a second tooth pair are present, any method may be used to align three-dimensional data representing the oral cavity to the occlusal plane based on two tooth pairs at corresponding positions relative to the midline of the oral cavity. For example, a transformation matrix may be generated to align two or more straight lines derived based on the three-dimensional center coordinates of four teeth of the two tooth pairs to corresponding straight lines on the occlusal plane. As a specific example, referring to FIGS. 10B and 10C, the processor may determine four teeth included in the first tooth pair (TT1, TT2) and the second tooth pair (TT3, TT4) as target teeth. The processor may determine a straight line connecting the three-dimensional center coordinates of each of the two teeth (TT1, TT2) of the first tooth pair as the first straight line (L1), and may determine a perpendicular bisector of a line segment (L2') connecting the three-dimensional center coordinates of each of the two teeth (TT3, TT4) of the second tooth pair as the second straight line (L2). Alternatively, the processor may obtain a cusp point based on the three-dimensional center coordinates for each tooth (TT1, TT2, TT3, TT4), and then determine the first straight line (L1) and the second straight line (L2) using the cusp point instead of the three-dimensional center coordinates. The processor may determine a third straight line (L3) and a fourth straight line (L4) on the occlusal plane corresponding to the first straight line (L1) and the second straight line (L2), respectively. The processor may generate a transformation matrix that aligns the first straight line (L1) with the third straight line (L3), and that aligns the second straight line (L2) with the fourth straight line (L4). According to one embodiment, the processor determines the intersection (P) of a first straight line (L1) and a second straight line (L2). i1 ) is determined, and the intersection point (P i1 ) can be used to align three-dimensional data representing the oral cavity to the occlusal plane. For example, the processor may align the intersection point (P i1 ) on the occlusal plane corresponding to the point (P i2) is determined (e.g., the intersection of the third straight line (L3) and the fourth straight line (L4)), the second straight line (L2) is aligned with the fourth straight line (L4), and the intersection of the first straight line (L1) and the second straight line (L2) (P i1 ) and the corresponding point (P) on the occlusal plane i2 ) can be used to generate a transformation matrix that matches the original data.

[0125] If either the first tooth pair or the second tooth pair is absent, the processor may generate a transformation matrix to move the three-dimensional center coordinates of each of one or more target teeth (i.e., all of the teeth represented by the first data) as close as possible to a corresponding reference center point on the occlusal plane while minimizing the amount of movement of the oral cavity represented by the first data. For example, for each of the plurality of teeth represented by the first data, a matrix may be determined for moving the three-dimensional center coordinates to a reference center point on the occlusal plane (1000) corresponding to each tooth. As a specific example, when the first data represents five teeth, the processor may determine a first matrix for moving the three-dimensional center coordinate of the first tooth to a reference center point on the occlusal plane (1000) of the first tooth, determine a second matrix for moving the three-dimensional center coordinate of the second tooth to a reference center point on the occlusal plane (1000) of the second tooth, determine a third matrix for moving the three-dimensional center coordinate of the third tooth to a reference center point on the occlusal plane (1000) of the third tooth, determine a fourth matrix for moving the three-dimensional center coordinate of the fourth tooth to a reference center point on the occlusal plane (1000) of the fourth tooth, and determine a fifth matrix for moving the three-dimensional center coordinate of the fifth tooth to a reference center point on the occlusal plane (1000) of the fifth tooth. The processor may generate a transformation matrix that minimizes the amount of movement in the three-dimensional space of the oral cavity represented by the first data based on the first to fifth matrices. Accordingly, it becomes possible to align the first data to the occlusal plane even when the first tooth pair and the second tooth pair do not exist.

[0126] FIG. 11a illustrates full arch data before alignment to the occlusal plane according to one embodiment of the present disclosure.

[0127] FIG. 11b illustrates full arch data aligned to the occlusal plane according to one embodiment of the present disclosure.

[0128] The processor may obtain first data (1110a), which is full-arch data, and rotate the first data (1110a) toward the occlusal plane (1100) based on a rotation matrix. The processor may generate second data based on the first data (1110a), and determine an identification number and a two-dimensional center coordinate of each tooth using the generated second data and a learning model. The processor may determine a three-dimensional center coordinate based on the two-dimensional center coordinate, and determine one or more target teeth based on the presence of a first tooth pair and a second tooth pair. Referring to FIG. 11A, since the full-arch data includes both a first tooth pair and a second tooth pair, the processor may generate a transformation matrix based on four teeth included in the first tooth pair and the second tooth pair. The processor may align the first data (1110b) to the occlusal plane (1100) based on the transformation matrix.

[0129] FIG. 12a illustrates the arch data before alignment to the occlusal plane according to one embodiment of the present disclosure.

[0130] FIG. 12b illustrates the arch data aligned to the occlusal plane according to one embodiment of the present disclosure.

[0131] Descriptions of overlapping content described in FIGS. 4 to 11 will be omitted. The processor can obtain first data (1210a), which is arch data. Since tooth pairs do not exist in the arch data, the processor can generate a transformation matrix based on all teeth included in the first data (1210a). The processor can align the first data (1210b) to the occlusal plane (1200) based on the transformation matrix. Using the method of the present disclosure, even if tooth pairs do not exist at corresponding positions with respect to the midline in the first data, the first data can be precisely aligned to the occlusal plane.

[0132] FIG. 13 is a flowchart of a method for aligning three-dimensional data of an oral cavity to an occlusal plane according to one embodiment of the present disclosure.

[0133] The electronic device can obtain first data, which is three-dimensional data about an oral cavity including a plurality of teeth, in operation S1310.

[0134] The electronic device can determine the three-dimensional center coordinates of each of the plurality of teeth in operation S1320.

[0135] The electronic device may determine, in operation S1330, one or more target teeth among the plurality of teeth.

[0136] The electronic device can determine, in operation S1340, one or more reference center points, each corresponding to one or more target teeth, in an occlusal plane for aligning a plurality of teeth.

[0137] The electronic device may generate a transformation matrix for aligning the first data to the occlusal plane based on the three-dimensional center coordinates of each of one or more target teeth and one or more reference center points, in operation S1350.

[0138] Although the steps of the method or algorithm according to the present disclosure are described in a sequential order in the flowchart illustrated in FIG. 13, the steps may be performed in any order that can be arbitrarily combined according to the present disclosure, in addition to being performed sequentially. The description according to this flowchart does not exclude changes or modifications to the method or algorithm, and does not imply that any step is essential or desirable. In one embodiment, at least some of the steps may be performed in parallel, iteratively, or heuristically. In one embodiment, at least some of the steps may be omitted, or other steps may be added.

[0139] Various embodiments of the present disclosure may be implemented as software on a machine-readable storage medium. The software may be software for implementing various embodiments of the present disclosure. The software may be inferred from various embodiments of the present disclosure by programmers skilled in the art to which the present disclosure pertains. For example, the software may be a program including machine-readable instructions (e.g., code or code segments). The device may be a device capable of operating according to instructions called from a storage medium, such as a computer. In one embodiment, the device may be an electronic device (100) according to embodiments of the present disclosure. In one embodiment, the processor of the device may execute the called instruction, causing components of the device to perform functions corresponding to the instruction. In one embodiment, the processor may be a processor (101) according to embodiments of the present disclosure. The storage medium may refer to any type of recording medium that stores data and can be read by the device. The storage medium may include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, etc. In one embodiment, the storage medium may be memory (103). In one embodiment, the storage medium may be implemented in a distributed form, such as in a network-connected computer system. The software may be distributed and stored and executed in a computer system, etc. The storage medium may be a non-transitory storage medium. A non-transitory storage medium means a tangible medium regardless of whether data is stored semi-permanently or temporarily, and does not include a signal that is propagated transitorily.

[0140] While the technical concept of the present disclosure has been described through various embodiments, it should be understood that the technical concept of the present disclosure encompasses various substitutions, modifications, and variations that can be made within the scope of those skilled in the art to which the present disclosure pertains. Furthermore, it should be understood that such substitutions, modifications, and variations are encompassed within the scope of the appended claims.

Claims

1. In a method performed by an electronic device, A step of acquiring first data, which is three-dimensional data for an oral cavity including a plurality of teeth; A step of determining the three-dimensional center coordinates of each of the plurality of teeth indicated by the first data; A step of determining one or more target teeth among the plurality of teeth; A step of determining one or more reference center points each corresponding to one or more target teeth in an occlusal plane for aligning the plurality of teeth; and A method comprising the step of generating a transformation matrix for aligning the first data to the occlusal plane based on the three-dimensional center coordinates of each of the one or more target teeth and the one or more reference center points.

2. In paragraph 1, The step of determining one or more target teeth among the above plurality of teeth is: A step of classifying each of the plurality of teeth into either a first tooth group or a second tooth group; A step of determining whether a first pair of teeth belonging to the first tooth group exists while being in corresponding positions based on the midline of the oral cavity; and A method comprising the step of determining whether a second pair of teeth belonging to the second tooth group exists while being in corresponding positions based on the midline.

3. In paragraph 2, Further comprising a step of determining an identification number for each of the plurality of teeth, A method wherein the step of determining whether the first pair of teeth exists and the step of determining whether the second pair of teeth exists are based on the identification number of each of the plurality of teeth.

4. In paragraph 2, The step of determining one or more target teeth among the above plurality of teeth is: A method comprising, in response to determining that both the first tooth pair and the second tooth pair are present, determining four teeth included in the first tooth pair and the second tooth pair as the one or more target teeth.

5. In paragraph 3, The step of determining one or more target teeth among the above plurality of teeth is: A method further comprising the step of determining, as a first tooth pair, a tooth pair including a tooth with the closest distance from the midline among the plurality of tooth pairs that are positioned corresponding to each other with respect to the midline, when there are multiple tooth pairs among one or more teeth included in the first tooth group.

6. In paragraph 3, The step of determining one or more target teeth among the above plurality of teeth is: A method further comprising the step of determining, as a second tooth pair, a tooth pair including a tooth with the closest distance from the midline among the plurality of tooth pairs that are in corresponding positions with respect to the midline among one or more teeth included in the second tooth group, if there are multiple tooth pairs that are in corresponding positions with respect to the midline.

7. In paragraph 2, The step of generating the above transformation matrix is: A method comprising, in response to a determination that both the first tooth pair and the second tooth pair are present, generating a transformation matrix to align two or more straight lines derived based on the three-dimensional center coordinates of the four teeth with corresponding straight lines on the occlusal plane.

8. In paragraph 2, The step of determining one or more target teeth among the above plurality of teeth is: A method comprising, in response to a determination that at least one of the first tooth pair or the second tooth pair is not present, determining all of the plurality of teeth as the one or more target teeth.

9. In paragraph 8, The step of generating the above transformation matrix is: A method comprising: generating a transformation matrix to minimize the amount of movement of the oral cavity indicated by the first data while moving the three-dimensional center coordinates of each of the one or more target teeth as close as possible to a corresponding reference center point of the occlusal plane in response to a determination that at least one of the first tooth pair or the second tooth pair is not present.

10. In paragraph 1, The step of determining the three-dimensional center coordinates of each of the plurality of teeth indicated by the first data is: A step of generating second data, which is two-dimensional data about the oral cavity, based on the first data; A step of identifying each of the plurality of teeth from the second data; a step of determining the two-dimensional center coordinates of each of the plurality of teeth; and A method comprising the step of determining the three-dimensional center coordinates of each of the plurality of teeth based on the two-dimensional center coordinates.

11. In paragraph 10, The step of generating the second data, which is two-dimensional data about the oral cavity, is: A step of identifying a tooth region indicated by the first data, wherein the tooth region includes the plurality of teeth; A step of adding a color indicating curvature to each point of the above tooth area; A method comprising the step of generating the second data based on data for the tooth region to which the color has been added among the first data.

12. In paragraph 11, The step of determining the two-dimensional center coordinates of each of the plurality of teeth is: A step of identifying each of the plurality of teeth indicated by the second data; A step of generating a plurality of bounding boxes corresponding to each of the plurality of teeth, wherein the plurality of bounding boxes are in the shape of the smallest rectangle that includes an area in which each of the plurality of teeth is located in the second data; and A method comprising the step of determining the center of each of the plurality of bounding boxes as the two-dimensional center coordinates of the corresponding tooth.

13. In paragraph 10, The step of determining the two-dimensional center coordinates of each of the plurality of teeth is: A step of applying the above second data to a learning model to obtain output data. A method comprising: including: the output data identifying each of the plurality of teeth and indicating the two-dimensional center coordinates of each of the plurality of teeth.

14. In paragraph 10, The step of determining the three-dimensional center coordinates of each of the plurality of teeth is: A step of determining a plurality of first points corresponding to the two-dimensional center coordinates of each of the plurality of teeth and located on a plane identical to or parallel to the occlusal plane; A step of determining a plurality of second points on the surfaces of the plurality of teeth corresponding to the plurality of first points, wherein each of the plurality of second points is an intersection of a straight line passing through the corresponding first point and a surface of the plurality of teeth that is parallel to a normal of the occlusal plane; and A method comprising the step of determining the plurality of second points as the three-dimensional center coordinates of each of the plurality of teeth.

15. In paragraph 14, The step of determining the three-dimensional center coordinates of each of the plurality of teeth is: A step of determining a plurality of third points, which are cusp points of each of the plurality of teeth, based on the plurality of second points; and A method comprising the step of determining the plurality of third points as the three-dimensional center coordinates of each of the plurality of teeth.

16. In paragraph 15, The step of determining the third point of each of the plurality of teeth is: A method comprising the step of determining a point with a maximum curvature among points obtained by moving the plurality of second points in the buccal and lingual directions along the surfaces of each of the plurality of teeth as a third point of each of the plurality of teeth.

17. In paragraph 1, The above method, A method further comprising a step of aligning the first data to the occlusion plane based on the transformation matrix.

18. In paragraph 1, The above method, A step of dividing the surfaces of a plurality of teeth represented by the first data into one or more polygons; determining at least one first normal vector for each of said at least one polygon; A step of determining a second normal vector corresponding to the first data based on the one or more first normal vectors; A step of determining a third normal vector which is a normal vector of the above occlusal plane; and A method further comprising the step of generating a rotation matrix for rotating the first data so that the second normal vector coincides with the third normal vector.

19. In electronic devices, One or more processors, comprising one or more memories storing instructions executed by the one or more processors; An electronic device, wherein when the instructions are executed by the one or more processors, the one or more processors are configured to execute a method according to any one of claims 1 to 18.

20. In a non-transitory computer-readable recording medium recording instructions that, when executed by one or more processors, cause the one or more processors to perform an operation, A non-transitory computer-readable recording medium, wherein the instructions are configured to cause the one or more processors to execute a method according to any one of claims 1 to 18.

Images