Scan image processing method of intraoral scanner, device therefor, and recording medium on which instructions thereof are recorded

Abstract

A method of the present disclosure may comprise the steps of: receiving an input for selecting a plurality of teeth of a target oral cavity; arranging first library teeth on a three-dimensional space; moving a plurality of second library teeth on the three-dimensional space such that the plurality of second library teeth corresponding to the plurality of teeth among the first library teeth are arranged in a space corresponding to the plurality of teeth in a three-dimensional image of the target oral cavity; and overlapping the three-dimensional image of the target oral cavity and the plurality of second library teeth, and displaying same.

Description

Method for processing scanned images of an oral scanner, device thereof and recording medium recording commands thereof

[0001] The present disclosure relates to a method for processing a scan image of an oral scanner, a device thereof, and a recording medium recording a command thereof.

[0002] A 3D image of the oral cavity refers to data obtained by scanning teeth, the oral cavity, or a model or reconstruction thereof using a 3D scanner. 3D images of a patient's oral cavity can be acquired during prosthetic treatments such as inlays, onlays, crowns, and implants, as well as during other dental treatments such as orthodontics. These images can be used to design prosthetics or implants, and to manufacture braces.

[0003] Traditionally, the primary method involved taking a mold of the oral cavity using materials like alginate and then manually fabricating prosthetics, implants, and orthodontic appliances. Recently, digital methods have become widespread, using 3D scanners to acquire 3D scan data of the patient's oral cavity, computers to design prosthetics, implants, and orthodontic appliances, and then 3D printing them.

[0004] The problem to be solved in the present disclosure is to provide a method for displaying a plurality of first library teeth selected by a user by overlaying them on a three-dimensional image of the oral cavity.

[0005] The problem to be solved in the present disclosure is to provide a method for determining positional information of a plurality of library teeth and aligning the library teeth by using segmentation and / or margin lines that separate the tooth region and the gingival region in a three-dimensional image of the oral cavity.

[0006] The problem to be solved in the present disclosure is to provide a method for enabling a user to edit multiple library teeth without switching screens by displaying multiple library teeth in a superimposed manner on a three-dimensional image of the oral cavity.

[0007] The problem to be solved in the present disclosure is to provide a method for forming customized library teeth using actual dental data of a patient (a three-dimensional image of the oral cavity).

[0008] The problems to be solved in the present disclosure are not limited to the problems mentioned above, and other problems not mentioned can be clearly understood by those skilled in the art from the description below.

[0009] In one embodiment, an image processing method performed by an electronic device may include: arranging a three-dimensional image of a target oral cavity in a three-dimensional space; receiving an input for selecting a plurality of teeth of the target oral cavity; arranging first library teeth corresponding to the three-dimensional image of the target oral cavity in the three-dimensional space; moving a plurality of second library teeth in the three-dimensional space such that a plurality of second library teeth corresponding to a plurality of teeth among the first library teeth are arranged in a space corresponding to a plurality of teeth among the three-dimensional image of the target oral cavity; and displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth in an overlapping manner.

[0010] In one embodiment, the method further comprises the step of obtaining a plurality of margin lines, each corresponding to a plurality of teeth, and the step of arranging the first library tooth in the three-dimensional space comprises the step of arranging the first library tooth in the three-dimensional space based on the plurality of margin lines, wherein each of the plurality of margin lines may represent a boundary at which an intact tooth begins to be exposed in a gingival region of a three-dimensional image of a target oral cavity.

[0011] In one embodiment, the step of obtaining a plurality of margin lines may include the step of obtaining a plurality of margin lines based on a boundary between a tooth region and a gingival region obtained as a result of segmentation; or the step of obtaining a plurality of margin lines based on a user input.

[0012] In one embodiment, the step of arranging the first library tooth in the three-dimensional space may include the step of identifying a tooth number corresponding to a tooth region obtained as a result of segmentation; and the step of arranging the first library tooth in the three-dimensional space based on the tooth number.

[0013] In one embodiment, the method may further comprise the step of generating a first library tooth.

[0014] In one embodiment, the step of generating the first library tooth may include the step of increasing the first library tooth in the direction of the gingival region based on mesh information of at least a portion of the first library tooth corresponding to the tooth region among the tooth region and gingival region included in the three-dimensional image of the target oral cavity.

[0015] In one embodiment, the step of generating the first library tooth may further include the step of machining an increased area in the increased first library tooth based on a plurality of margin lines.

[0016] In one embodiment, the step of receiving an input for selecting a plurality of teeth in the target oral cavity may include the step of receiving an input for selecting a tooth number for each of the plurality of teeth in the target oral cavity.

[0017] In one embodiment, the step of moving a plurality of second library teeth in a three-dimensional space may include: generating first position information including a center point of a first bounding box for each of the plurality of second library teeth based on each of the plurality of margin lines; generating second position information including a center point of a second bounding box for each of the teeth of a three-dimensional image of the target oral cavity; and generating a correlation between the first position information and the second position information for each of the plurality of second library teeth based on an origin located in a dental arch of the target oral cavity.

[0018] In one embodiment, the step of generating a correlation may include the step of generating a tooth-by-tooth transformation matrix indicating a correlation between first positional information of each of the plurality of second library teeth relative to the origin and second positional information having the same tooth number as each of the plurality of second library teeth.

[0019] In one embodiment, the step of superimposing and displaying a three-dimensional image of the target oral cavity and a plurality of second library teeth may include a step of superimposing and displaying a plurality of second library teeth on a three-dimensional image of the target oral cavity, each having a tooth number identical to that of each of the plurality of second library teeth in three-dimensional space, based on a tooth-by-tooth transformation matrix.

[0020] In one embodiment, the origin may be a point between two central incisors in the arch.

[0021] In one embodiment, the step of superimposing and displaying a three-dimensional image of the target oral cavity and a plurality of second library teeth may include a step of superimposing and displaying a library tooth corresponding to the deleted tooth on the three-dimensional image of the target oral cavity based on a tooth-specific transformation matrix corresponding to the deleted tooth.

[0022] In one embodiment, a tooth-specific transformation matrix corresponding to a deleted tooth is generated based on second position information including a point in the arch corresponding to a tooth number, and a library tooth corresponding to the deleted tooth may include a prosthesis corresponding to the deleted tooth.

[0023] In one embodiment, the step of superimposing and displaying a three-dimensional image of the target oral cavity and a plurality of second library teeth may include a step of superimposing and displaying a library tooth corresponding to a prep tooth on the three-dimensional image of the target oral cavity based on a tooth-specific transformation matrix corresponding to the prep tooth.

[0024] In one embodiment, a tooth-specific transformation matrix corresponding to a prep tooth is generated based on second position information in which a center point of a second bounding box is changed based on a difference between the uppermost height of the prep tooth and the uppermost height of a library tooth corresponding to the prep tooth, and the library tooth corresponding to the prep tooth may include a prosthesis corresponding to the prep tooth.

[0025] In one embodiment, the step of superimposing and displaying a three-dimensional image of the target oral cavity and a plurality of second library teeth may include a step of displaying connectors between the plurality of second library teeth based on a determination that the plurality of second library teeth corresponding to the plurality of teeth are connected by connectors.

[0026] In another embodiment, an electronic device includes a processor; a network interface communicating with an oral scanner; a display; a memory; and a computer program loaded into the memory and executed by the processor, wherein the computer program is configured to: place a three-dimensional image of a target oral cavity in a three-dimensional space; receive an input for selecting a plurality of teeth of the target oral cavity; place first library teeth corresponding to the three-dimensional image of the target oral cavity in the three-dimensional space; move a plurality of second library teeth in the three-dimensional space such that a plurality of teeth among the first library teeth are placed in spaces corresponding to the plurality of teeth among the three-dimensional image of the target oral cavity; and display the three-dimensional image of the target oral cavity and the plurality of second library teeth in an overlapping manner.

[0027] In another embodiment, a non-transitory computer-readable recording medium having recorded thereon a computer program to be executed by a processor, the computer program may include instructions for arranging a three-dimensional image of a target oral cavity in a three-dimensional space; instructions for receiving an input for selecting a plurality of teeth of the target oral cavity; instructions for arranging first library teeth corresponding to the three-dimensional image of the target oral cavity in the three-dimensional space; instructions for moving a plurality of second library teeth in the three-dimensional space such that a plurality of second library teeth corresponding to a plurality of teeth among the first library teeth are arranged in spaces corresponding to the plurality of teeth among the three-dimensional image of the target oral cavity; and instructions for superimposing and displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth.

[0028] According to various embodiments of the present disclosure, a method for displaying a plurality of first library teeth selected by a user by overlapping them on a three-dimensional image of an oral cavity can be provided.

[0029] According to various embodiments of the present disclosure, a method for determining positional information of a plurality of library teeth and aligning the library teeth can be provided by using segmentation and / or margin lines that separate a tooth region and a gingival region in a three-dimensional image of an oral cavity.

[0030] According to various embodiments of the present disclosure, a method can be provided that allows a user to edit a plurality of library teeth without switching screens by displaying the plurality of library teeth in an overlapping manner on a three-dimensional image of the oral cavity.

[0031] According to various embodiments of the present disclosure, a method for forming customized library teeth using actual dental data of a patient (a three-dimensional image of the oral cavity) can be provided.

[0032] FIG. 1 is an exemplary diagram illustrating a scanning environment according to some embodiments of the present disclosure.

[0033] FIG. 2 is an exemplary block diagram illustrating each of the electronic device and the oral scanner illustrated in FIG. 1.

[0034] FIG. 3 is an exemplary drawing of the oral scanner described with reference to FIG. 2.

[0035] Figures 4a and 4b are drawings for explaining a three-dimensional image of the target concept of the present disclosure.

[0036] FIG. 4c is a drawing for explaining the first library tooth of the present disclosure.

[0037] Figure 5 is a drawing for explaining the margin line of the present disclosure.

[0038] FIG. 6 is a drawing for explaining a method for selecting multiple teeth of the present disclosure.

[0039] FIG. 7 is a drawing for explaining second position information in a three-dimensional image of the oral cavity of the present disclosure.

[0040] Fig. 8 is a drawing for explaining the first position information of the library teeth of the present disclosure.

[0041] FIG. 9 is an exemplary drawing illustrating how the library teeth described with reference to FIG. 8 are superimposed on the three-dimensional image of the oral cavity described with reference to FIG. 7.

[0042] FIG. 10 is a flowchart illustrating a method in which a server superimposes library teeth onto a three-dimensional image of an oral cavity in the present disclosure.

[0043] The embodiments of this disclosure are provided for the purpose of illustrating the technical concepts of this disclosure. Therefore, the scope of rights under this disclosure is not limited to the embodiments presented below or the specific descriptions of these embodiments.

[0044] All technical and scientific terms used in this disclosure, unless otherwise defined, have the meanings commonly understood by those of ordinary skill in the art to which this disclosure pertains. All terms used in this disclosure have been selected for the purpose of more clearly explaining this disclosure and are not intended to limit the scope of rights under this disclosure.

[0045] Expressions such as “including,” “comprising,” “having,” and the like used in this disclosure should be understood as open-ended terms that imply the possibility of including other embodiments, unless otherwise stated in the phrase or sentence in which the expression is included.

[0046] Unless otherwise stated, singular expressions described herein may include plural meanings, and this also applies to singular expressions described in the claims. Furthermore, expressions such as "first," "second," etc., used herein are used to distinguish between multiple components and do not limit the order or importance of the components.

[0047] The term "unit" as used herein refers to software or hardware components such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). However, the "unit" is not limited to hardware and software, and the "unit" may be configured to reside on an addressable storage medium, or may be configured to execute one or more processors. Accordingly, as examples of what the "unit" may mean, the "unit" includes components such as software components, object-oriented software components, class components, and task components, as well as processors, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. Furthermore, the functionality provided within a component and a "unit" may be combined into a smaller number of components and "units" or further separated into additional components and "units."

[0048] The expression "based on" as used in this disclosure is used to describe one or more factors that influence the decision, act of judgment, or action described in the phrase or sentence containing the expression, and this expression does not exclude additional factors that influence the decision, act of judgment, or action.

[0049] When it is said in this disclosure that a component is "connected" or "connected" to another component, it should be understood that the component can be directly connected or connected to the other component, or can be connected or connected via another new component.

[0050] Hereinafter, embodiments of the present disclosure will be described with reference to the attached drawings. In the attached drawings, identical or corresponding components are assigned the same reference numerals. Furthermore, in the description of the embodiments below, redundant descriptions of identical or corresponding components may be omitted. However, even if a description of a component is omitted, it is not intended that such component is not included in any embodiment.

[0051] FIG. 1 is an exemplary diagram illustrating a scanning environment according to some embodiments of the present disclosure. Specifically, the scanning environment illustrated in FIG. 1 is an environment for acquiring an image of a patient's oral cavity using an oral scanner (200) according to some embodiments of the present disclosure, wherein the oral scanner (200) may be a dental medical device for acquiring an image of the oral cavity of a subject (20) (e.g., a patient).

[0052] As illustrated in FIG. 1, a user (10) (e.g., a dentist, a dental hygienist) can obtain an image of the oral cavity of the subject (20) from the subject (20) using an oral scanner (200). As another example, the user (10) can also obtain an image of the oral cavity of the subject (20) from a diagnostic model (e.g., a plaster model, an impression model) that models the shape of the oral cavity of the subject (20). For convenience of explanation, the following description will be based on an example of obtaining an image of the oral cavity of the subject (20), but the scope of the present disclosure is not limited by such an example. In addition, the area to be imaged is not limited to the oral cavity of the subject (20), and it is also possible to obtain images of other areas of the subject (20) (e.g., the ear of the subject (20)).

[0053] Hereinafter, the operation of the components illustrated in Fig. 1 will be described in detail.

[0054] The electronic device (100) illustrated in FIG. 1 can receive a two-dimensional image of the oral cavity of a subject (20) from an oral scanner (200). Here, the electronic device (100) can communicate with the oral scanner (200) via a wired or wireless communication network. In addition, the electronic device (100) can generate a three-dimensional image of the oral cavity by three-dimensionally modeling the internal structure of the oral cavity based on the received two-dimensional image of the oral cavity, and specific operations related thereto will be described later with reference to FIG. 4. The three-dimensional image of the oral cavity thus generated can be displayed to a user (10) or the subject (20) through the display of the electronic device (100). At this time, the user (10) can provide appropriate medical services, etc. to the subject (20) by referring to the three-dimensional image displayed on the display of the electronic device (100).

[0055] In some embodiments, the electronic device (100) may receive a three-dimensional image of the oral cavity generated by the oral scanner (200). Here, the oral scanner (200) may scan the oral cavity of the subject (20) to obtain a two-dimensional image of the oral cavity, and may generate a three-dimensional image of the oral cavity based on the obtained two-dimensional image of the oral cavity. That is, it should be noted that the operation of generating a three-dimensional image is included within the scope of the present disclosure regardless of the location where it is processed.

[0056] Additionally, the electronic device (100) may be connected to a cloud server (not shown). In this case, the electronic device (100) may transmit a two-dimensional image of the oral cavity of the subject (20) or a three-dimensional image of the oral cavity to the cloud server, and the cloud server may store the two-dimensional image of the oral cavity of the subject (20) or the three-dimensional image of the oral cavity received from the electronic device (100).

[0057] The electronic device (100) described so far can be implemented as a computing device, and an example of such a computing device will be described in detail later with reference to FIG. 2.

[0058] The oral scanner (200) illustrated in FIG. 1 may have a form that can be inserted and withdrawn into the oral cavity, and may be a handheld scanner in which the user (10) can freely adjust the scanning distance and scanning angle.

[0059] This oral scanner (200) can obtain an image of the oral cavity by being inserted into the oral cavity and scanning the inside of the oral cavity in a non-contact manner. The image of the oral cavity can include at least one tooth, gingiva, or an artificial structure that can be inserted into the oral cavity (e.g., an orthodontic device including a bracket and a wire, an implant, a denture, an orthodontic auxiliary tool inserted into the oral cavity, etc.). Specifically, the oral scanner (200) can irradiate light into the oral cavity of the subject (20) using a light source (or a projector), and can receive light reflected from the oral cavity of the subject (20) through a camera (or at least one image sensor).

[0060] In addition, the oral scanner (200) can acquire a surface image of the oral cavity of the subject (20) as a two-dimensional image based on information received through the camera. Here, the surface image of the oral cavity of the subject (20) can include at least one of the teeth, gums, artificial structures, cheeks, tongue, or lips of the subject (20).

[0061] As described above, in some embodiments, the oral scanner (200) may scan the oral cavity to obtain a two-dimensional image of the oral cavity, and may also generate a three-dimensional image of the oral cavity based on the obtained two-dimensional image of the oral cavity.

[0062] The oral scanner (200) described so far can be implemented as a computing device, and an example of such a computing device will be described in detail below with reference to FIG. 2.

[0063] FIG. 2 is an exemplary block diagram illustrating each of the electronic device (100) and the oral scanner (200) illustrated in FIG. 1. The block diagram illustrated in FIG. 2 merely illustrates a preferred embodiment for achieving the purpose of the present disclosure, and some components may be added or deleted as needed. In addition, it should be noted that the components in the block diagram illustrated in FIG. 2 represent functionally distinct functional elements, and that multiple different components may be implemented in an integrated form in an actual physical environment. Hereinafter, each component illustrated in the block diagram will be described in detail.

[0064] The electronic device (100) illustrated in FIG. 2 may include one or more processors (101), one or more memories (103), a communication circuit (105), a display (107), and / or an input device (109). As described above, at least one of the components included in the electronic device (100) may be omitted, or other components may be added to the electronic device (100). Additionally or alternatively, some of the components may be implemented in an integrated manner, or may be implemented as a single or multiple entities. At least some of the components within the electronic device (100) may be connected to each other via a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI), and may exchange data and / or signals.

[0065] One or more processors (101) of the electronic device (100) may be configured to perform calculations or data processing related to control and / or communication of each component (e.g., memory (103)) of the electronic device (100). The one or more processors (101) may be operatively connected to, for example, components of the electronic device (100). In addition, the one or more processors (101) may load commands or data received from other components of the electronic device (100) into one or more memories (103), process commands or data stored in the one or more memories (103), and store result data.

[0066] Next, one or more memories (103) of the electronic device (100) can store various data, commands, and / or information. As a specific example, one or more memories (103) can store instructions for the operation of the processor (101) as a computer program. In addition, one or more memories (103) can store correlation models built according to a machine learning algorithm. In addition, one or more memories (103) can store data received from the oral scanner (200) (e.g., a two-dimensional image of the oral cavity, a three-dimensional image of the oral cavity).

[0067] Next, the communication circuit (network interface, 105) of the electronic device (100) can establish a wired or wireless communication channel with an external device (e.g., an oral scanner (200), a cloud server (not shown)) and transmit and receive various data with the external device. In some embodiments, the communication circuit (105) may include at least one port for connecting to the external device via a wired cable in order to communicate with the external device via a wire. In this case, the communication circuit (105) can communicate with the external device connected via a wire through the at least one port. In some other embodiments, the communication circuit (105) may be configured to be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax) by including a cellular communication module. In some other embodiments, the communication circuit (105) may include a short-range communication module to transmit and receive data with the external device using short-range communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB). In some other embodiments, the communication circuit (105) may include a contactless communication module for contactless communication. The contactless communication may include at least one contactless proximity communication technology, such as near field communication (NFC), radio frequency identification (RFID), or magnetic secure transmission (MST). In addition to the various examples described above, the electronic device (100) may be implemented in various known ways for communicating with external devices, and it should be noted that the scope of the present disclosure is not limited to the examples described above.

[0068] Next, the display (107) of the electronic device (100) can display various screens based on the control of the processor (101). Here, based on the control of the processor (101), a two-dimensional image of the oral cavity of the object (20) received from the oral scanner (200) and / or a three-dimensional image of the oral cavity that is a three-dimensional model of the internal structure of the oral cavity can be displayed through the display (107). At this time, in order to display the two-dimensional image and / or the three-dimensional image of the oral cavity on the display (107), for example, a web browser or a dedicated application can be installed in the electronic device (100). In some embodiments, the aforementioned web browser or dedicated application can be implemented to provide the user (10) with an editing function, a saving function, and a deleting function of the two-dimensional image and / or the three-dimensional image of the oral cavity through a user interface.

[0069] Next, the input device (109) of the electronic device (100) can receive commands or data to be used in components of the electronic device (100) (e.g., the processor (101)) from an external source (e.g., a user) of the electronic device (100). The input device (109) may include, for example, a microphone, a mouse, or a keyboard. In some embodiments, the input device (109) may be implemented in the form of a touch sensor panel that is coupled with the display (107) and can recognize contact or proximity of various external objects. However, the scope of the present disclosure is not limited to the examples described above, and various known input devices (109) may be included in the scope of the present disclosure for the convenience of the user.

[0070] The oral scanner (200) illustrated in FIG. 2 may include a processor (201), a memory (202), a communication circuit (203), a light source (204), a camera (205), and / or an input device (206). As described above, at least one of the components included in the oral scanner (200) may be omitted, or other components may be added to the oral scanner (200). Additionally or alternatively, some of the components may be implemented in an integrated manner, or may be implemented as a single or multiple entities. At least some of the components within the oral scanner (200) may be connected to each other via a bus, a general purpose input / output (GPIO), a serial peripheral interface (SPI), or a mobile industry processor interface (MIPI), and may exchange data and / or signals.

[0071] The processor (201) of the oral scanner (200) is a configuration capable of performing calculations or data processing related to control and / or communication of each component of the oral scanner (200), and may be operatively connected to the components of the oral scanner (200). In addition, the processor (201) may load commands or data received from other components of the oral scanner (200) into the memory (202), process the commands or data stored in the memory (202), and store the result data.

[0072] Next, the memory (202) of the oral scanner (200) can store instructions for the operation of the processor (201) described above.

[0073] Next, the communication circuit (203) of the oral scanner (200) can establish a wired or wireless communication channel with an external device (e.g., an electronic device (100)) and transmit and receive various data with the external device. In some embodiments, the communication circuit (203) can include at least one port for connecting to the external device via a wired cable in order to communicate with the external device via a wire. In this case, the communication circuit (203) can communicate with the external device connected via a wire through the at least one port. In some other embodiments, the communication circuit (203) can be configured to include a cellular communication module to be connected to a cellular network (e.g., 3G, LTE, 5G, Wibro, or Wimax). In some other embodiments, the communication circuit (203) can include a short-range communication module to transmit and receive data with the external device using short-range communication (e.g., Wi-Fi, Bluetooth, Bluetooth Low Energy (BLE), UWB). In some other embodiments, the communication circuit (203) may include a contactless communication module for contactless communication. Here, the contactless communication may include at least one contactless proximity communication technology, such as near field communication (NFC), radio frequency identification (RFID), or magnetic secure transmission (MST). In addition to the various examples described above, the oral scanner (200) may be implemented in various known ways for communicating with external devices, and it should be noted that the scope of the present disclosure is not limited to the examples described above.

[0074] Next, the light source (204) of the oral scanner (200) can irradiate light toward the oral cavity of the subject (20). For example, the light irradiated from the light source (204) can be structured light having a predetermined pattern (e.g., a stripe pattern in which straight lines of different colors appear continuously). Here, the pattern of the structured light can be generated using, for example, a pattern mask or a digital micro-mirror device (DMD), but is not limited thereto.

[0075] Next, the camera (205) of the oral scanner (200) can obtain a three-dimensional image of the oral cavity of the subject (20) by receiving reflected light reflected by the oral cavity of the subject (20). Here, the camera (205) may include a left camera corresponding to the left eye field of view and a right camera corresponding to the right eye field of view, for example, to construct a three-dimensional image according to an optical triangulation method. In addition, the camera (205) may include at least one image sensor, such as a CCD sensor or a CMOS sensor.

[0076] Next, the input device (206) of the oral scanner (200) can receive user input for controlling the oral scanner (200). For example, the input device (206) can include a button for receiving a push operation from the user (10), a touch panel for detecting the touch of the user (10), and a voice recognition device including a microphone. At this time, the user (10) can control the start or stop of scanning using the input device (206).

[0077] To describe in more detail the operation of the oral scanner (200) controlled through the input device (206), the oral scanner (200) may receive a user input for starting a scan through the input device (206) of the oral scanner (200) or the input device (206) of the electronic device (100), and may start the scan according to processing in the processor (201) of the oral scanner (200) or the processor (201) of the electronic device (100). Here, when the user (10) scans the inside of the oral cavity of the subject (20) through the oral scanner (200), the oral scanner (200) may generate a two-dimensional image of the oral cavity of the subject (20) and transmit the two-dimensional image of the oral cavity of the subject (20) to the electronic device (100) in real time. At this time, the electronic device (100) may display the two-dimensional image of the oral cavity of the subject (20) received through the display (107). In addition, the electronic device (100) can generate (build) a three-dimensional image of the oral cavity of the object (20) based on a two-dimensional image of the oral cavity of the object (20), and can display the three-dimensional image of the oral cavity through a display (107). At this time, the electronic device (100) can also display the three-dimensional image being generated in real time through the display (107).

[0078] Next, the sensor module (207) of the oral scanner (200) can detect the operating status of the oral scanner (200) or the external environmental status (e.g., the user's movement) and generate an electrical signal corresponding to the detected status. The sensor module (207) can include, for example, at least one of a gyro sensor, an acceleration sensor, a gesture sensor, a proximity sensor, or an infrared sensor. Here, the user (10) can control the start or stop of scanning using the sensor module (207). For a specific example, when the user (10) holds the oral scanner (200) in his / her hand and moves it, the oral scanner (200) can control the processor (201) to start a scanning operation when the angular velocity measured by the sensor module (207) exceeds a set value.

[0079] Referring to FIG. 3 below, the oral scanner (200) described so far will be described in more detail. FIG. 3 is an exemplary drawing of the oral scanner (200) described with reference to FIG. 2.

[0080] The oral scanner (200) illustrated in FIG. 3 may include a main body (210) and a probe tip (220). Here, the main body (210) of the oral scanner (200) may be formed in a shape that is easy for a user (10) to grip by hand and use, and the probe tip (220) may be formed in a shape that is easy to insert and withdraw into the oral cavity of the subject (20). In addition, the main body (210) may be coupled to and separated from the probe tip (220). In addition, components of the oral scanner (200) described in FIG. 2 may be arranged inside the main body (210). An opening may be formed at one end of the main body (210) so that light output from a light source (204) may be irradiated to the subject (20). The light irradiated through the opening may be reflected by the subject (20) and then re-entered through the opening. Here, the reflected light entering through the opening can be captured by the camera to create an image of the object (20). Additionally, the user (10) can initiate scanning using the input device (206) (e.g., a button) of the oral scanner (200). For example, when the user (10) touches or presses the input device (206), light from the light source (204) can be irradiated onto the object (20).

[0081] With reference to FIGS. 1 to 3, the scanning environment and components included therein according to some embodiments of the present disclosure have been specifically described.

[0082] In the present disclosure, library teeth are sample teeth (standard teeth) used to manufacture prosthetics, implants, braces, etc., and may have a typical tooth shape. The library teeth are expressed as three-dimensional images and can be displayed based on mesh information. The library teeth may have one sample tooth (standard tooth) for each tooth number. Each sample tooth may include general characteristics of the shape of the natural tooth of the corresponding tooth number. For example, if a specific tooth of a patient is lost or removed for treatment and a prosthesis needs to be made to replace it, utilizing the shape of the corresponding sample tooth can efficiently manufacture a prosthesis with a natural shape. In addition, the three-dimensional image of the oral cavity is captured by an oral scanner (200), and the mesh may have a somewhat low degree of completion. If the mesh is low in completion, it may be unsuitable for manufacturing prosthetics, implants, braces, etc. by 3D printing. Conversely, the library teeth may be tooth models with a high degree of completion of the mesh. Therefore, library teeth are ideally suited for 3D printing when modifying them to manufacture prosthetics, implants, orthodontics, and other devices. Aligning library teeth with a 3D image of the patient's oral cavity can serve as an appropriate intermediate model for digitally manufacturing prosthetics, implants, orthodontics, and other devices.

[0083] In another embodiment, library teeth may be generated based on a three-dimensional image of the oral cavity. The library teeth may be generated based on mesh information corresponding to the tooth region among the gingival region and tooth region segmented from the three-dimensional image of the oral cavity. Through this, the electronic device (100) can generate library teeth similar to the teeth of an actual patient.

[0084] In another embodiment, the library teeth may be generated based on a three-dimensional image of a specific subject's past oral cavity. For example, a three-dimensional image of the patient's oral cavity obtained before a specific tooth was lost or removed may be stored in memory (103). The electronic device (100) may also generate the library teeth based on this three-dimensional image of the oral cavity containing the specific intact teeth. In this case, the library teeth may include the shape of the patient's natural teeth.

[0085] Since the library teeth can be used in the production of prosthetics, implants, braces, dental treatment devices, etc., the effort and time required for the production of prosthetics, implants, braces, dental treatment devices, etc. can be reduced, the accuracy and productivity of prosthetics, implants, braces, dental treatment devices, etc. can be improved, and the consumption of computing resources to obtain the same results can be reduced.

[0086] In the present disclosure, library teeth, which are pre-designed to a certain extent by tooth number (dental type), can be used. The library teeth can be displayed by overlaying (or aligning) them onto a three-dimensional image of the oral cavity to produce prosthetics, implants, and orthodontic appliances. If the process of overlaying or aligning library teeth onto a three-dimensional image of the oral cavity is performed manually, the work fatigue of the dentist or dental technician may increase, the accuracy and productivity of the results may decrease, and consequently, the consumption of computing resources to achieve the same result may increase.

[0087] Additionally, if there are two or more library teeth to be worked on, manual alignment may be required by selecting each library tooth individually and editing them in a separate editing screen. Therefore, a method for aligning and displaying two or more library teeth on a three-dimensional image of the oral cavity and simultaneously editing two or more library teeth on the same screen without a separate screen transition is described in detail below.

[0088] In one embodiment, the electronic device (100) can place a three-dimensional image of the target oral cavity in three-dimensional space.

[0089] Figures 4a and 4b are drawings for explaining a three-dimensional image of the target concept of the present disclosure.

[0090] FIG. 4A illustrates a three-dimensional image (400) of an oral cavity as an example. The three-dimensional image (400) of the oral cavity may include a gingival region and a tooth region. The electronic device (100) may distinguish the gingival region and the tooth region in the three-dimensional image (400) through segmentation. The segmentation may be image segmentation. Image segmentation is a technical process of dividing a digital image into individually distinct parts or objects. This process aims to divide the image into multiple meaningful regions so that each part can be more easily analyzed and processed. Specifically, image segmentation is achieved by classifying pixels into groups that share the same properties or classes. These divided regions may include, for example, object boundaries, surfaces, and other important elements.

[0091] FIG. 4B illustrates, as an example, only a tooth region (420) derived from a three-dimensional image (400) of an oral cavity. The electronic device (100) can create a library tooth using the tooth region (420). By creating a library tooth from a three-dimensional image of an oral cavity scanned from a specific object's oral cavity, the electronic device (100) can create a library tooth that matches the individual characteristics of the object. In one embodiment, the electronic device (100) can create a surface of the library tooth using the tooth region (420). The surface of the library tooth may be composed of a mesh. In one embodiment, the interior of the library tooth may be expressed in the form of an empty space. In another embodiment, the interior of the library tooth may be expressed in a filled form based on mesh information related to the tooth.

[0092] FIG. 4c is a drawing for explaining the first library tooth of the present disclosure.

[0093] The library tooth may be a sample tooth shape having a typical tooth shape and / or a tooth shape generated based on a three-dimensional image of the oral cavity of a specific subject. Accordingly, the electronic device (100) may obtain a library tooth having a sample tooth shape from the memory (103), or may obtain a newly generated library tooth based on a three-dimensional image of the oral cavity of a specific subject.

[0094] FIG. 4C illustrates a first library tooth (440) generated based on a tooth region (420) acquired from a three-dimensional image (400) of an oral cavity. The first library tooth (440) may be a three-dimensional image including all library teeth corresponding to the three-dimensional image of the oral cavity. For example, the first library tooth (440) may be a three-dimensional image including all library teeth from tooth numbers 1 to 32.

[0095] In the present disclosure, the second library tooth may include at least some library teeth from the first library tooth, and is distinguished for convenience of description. The second library tooth may be a library tooth corresponding to a plurality of selected teeth.

[0096] In one embodiment, the electronic device (100) may generate a first library tooth. The tooth may be composed of a crown region (e.g., an area exposed from the gingiva) and a root region (e.g., an area not exposed due to being covered by the gingiva). The electronic device (100) may increase the library tooth in the direction of the gingival region based on mesh information of the crown region of the library tooth. Since an actual tooth has a root region as well as a crown region, the electronic device (100) may form a root region by extending the library tooth in the direction of the gingival region so that the library tooth has a shape similar to that of an actual tooth. A three-dimensional image of the oral cavity may not include the root region. Since the root region of an actual tooth is covered by the gingiva, the scan result may not include the root region. Therefore, the electronic device (100) may acquire the tooth region and the gingival region as segmentation results from the three-dimensional image of the oral cavity. In this case, the tooth region in the three-dimensional image of the oral cavity may be the crown region of the tooth. The electronic device (100) can generate a crown region of a library tooth based on the tooth region of a three-dimensional image of the oral cavity. Furthermore, the electronic device (100) can increase the first library tooth toward the gingival region based on mesh information of a portion of the crown region. Through this, the electronic device (100) can also express the root region while generating the first library tooth.

[0097] The electronic device (100) may increase the library tooth in the direction of the gingival region based on the mesh information of the region (461), which is at least a portion of the library tooth among the tooth regions. For example, the electronic device (100) may form a root region by elongating the mesh of the region (461) in the direction of the gingival region based on a central axis that penetrates the tooth vertically. The mesh information of the increased region (462) may be identical to the mesh information of the region (461). The increased region (462) may be intended to represent the root region of the library tooth. By increasing the library tooth, the electronic device (100) may make the empty space between the tooth region and the gingival region invisible when the library tooth is superimposed on the three-dimensional image (400) of the oral cavity. This enables realistic simulation through the three-dimensional image of the oral cavity where the library tooth is superimposed, reduces worker confusion, and leads to high work productivity. In addition, the consumption of computing resources to obtain the same result may be reduced.

[0098] In another embodiment, a library tooth may represent a crown area but not a root area. If the library tooth only represents a crown area, the height of the prosthesis included in the library tooth may be lower than the prosthesis that is actually to be installed. In such a case, the user may need to arbitrarily adjust the height of the prosthesis. Therefore, for realistic simulation, the electronic device may augment the library tooth based on mesh information of the crown area of ​​the library tooth so that it can represent not only the crown area of ​​the library tooth but also the root area.

[0099] In one embodiment, the electronic device (100) can process an increased area of ​​an increased first library tooth based on a plurality of margin lines. For example, the electronic device (100) can change the mesh information of the margin line (450) so as to distinguish the increased area from the existing area based on the margin line (450). The electronic device (100) can smoothly express the shape extending from the crown area to the root area of ​​the library tooth through an implicit surfacing technique. Implied surfacing is a technique for defining a surface through a mathematical function, particularly a point where the function value at an arbitrary point (x, y, z) becomes a specific value. This method can ensure the continuity and smoothness of the surface because the surface is mathematically defined. In addition, the implicit surfacing method can efficiently process local details of the model.

[0100] For example, the electronic device (100) can cut the library tooth in the direction of the gum area based on the margin line to create a tooth shape of an actually required length. In addition, the battery device (100) can process the library tooth into a shape similar to an actual tooth shape.

[0101] In one embodiment, the electronic device (100) may generate a library of teeth including prostheses corresponding to the plurality of teeth based on a determination that the plurality of teeth include at least one of a deleted tooth or a prepped tooth. In dental treatment, a deleted tooth may be removed due to an external factor or through a dental procedure. A prepped tooth refers to a tooth that has been prepared in advance for dental restoration or surgery. Preparing involves shaping or carving a tooth into a shape that meets specific needs. A prepped tooth is essential for mounting various dental prostheses, such as crowns, bridges, and veneers. A dental prosthesis refers to a medical device designed to replace a deleted tooth or restore a damaged tooth. Prostheses include crowns, bridges, and dentures, each manufactured according to specific purposes and designs. Crowns encase and protect damaged teeth, while bridges fill missing areas using adjacent teeth. Dentures are used to restore function and aesthetics in cases where teeth are partially or completely missing. The electronic device (100) can recognize whether a tooth corresponding to a specific tooth number is a deleted tooth or a prep tooth through a three-dimensional image of the oral cavity. As another example, the electronic device (100) can determine whether a plurality of teeth are at least one of a deleted tooth or a prep tooth based on user input.

[0102] In one embodiment, the electronic device (100) may receive an input for selecting multiple teeth in the target oral cavity. The user may view the three-dimensional images of FIGS. 4A and 4B and determine which teeth to create and align as library teeth. The input for selecting multiple teeth is described below in FIG. 6.

[0103] Figure 5 is a drawing for explaining the margin line of the present disclosure.

[0104] The margin line represents the boundary where an intact tooth begins to be exposed in the gingival region of a three-dimensional image of the target oral cavity. For an intact tooth in the target oral cavity, the margin line may also refer to the boundary line between the gingival region and the tooth region in the three-dimensional image of the oral cavity. For example, as illustrated in FIG. 5 , a margin line (510) may be displayed in the three-dimensional image (400) of the oral cavity. On the other hand, for a deleted tooth or a prepared tooth in the target oral cavity, the margin line may be generated assuming that an intact tooth exists at that location.

[0105] In one embodiment, the electronic device (100) can acquire multiple margin lines, each corresponding to a plurality of teeth. The electronic device (100) can acquire the margin lines based on the boundaries between the tooth region and the gingival region acquired as a result of segmentation. In another embodiment, the electronic device (100) can set the margin lines on a three-dimensional image of the oral cavity, each corresponding to a plurality of teeth, based on user input. Through this, the user can edit the margin lines automatically set by the electronic device (100).

[0106] In one embodiment, the electronic device (100) may place the first library tooth (440) in a three-dimensional space. The first library tooth (440) may be located in the same three-dimensional space as the three-dimensional image (400) of the oral cavity, but may be placed apart from the three-dimensional image (400) of the oral cavity. In addition, the first library tooth (440) may be omitted from the screen, or may be displayed.

[0107] In one embodiment, the electronic device (100) may place the first library teeth in a three-dimensional space based on a plurality of margin lines. The electronic device (100) may use the plurality of margin lines so that the first library teeth (440) in the three-dimensional space may be placed identically to the oral structure in the three-dimensional image (400) of the oral cavity. Each of the plurality of margin lines may correspond to a tooth number. For example, in the three-dimensional image of the oral cavity, there may be a margin line A corresponding to tooth number 1, a margin line B corresponding to tooth number 2, etc. Each of the margin lines may include arrangement information of the plurality of teeth in the three-dimensional image of the oral cavity. For example, the positions of the margin lines in the three-dimensional image of the oral cavity may indicate the positions of the teeth in the dental arch. Accordingly, the electronic device (100) may place the first library teeth in the three-dimensional space by matching the margin lines corresponding to the tooth numbers of the library teeth. For example, in the library teeth, margin line A may be assigned to tooth number 1, and margin line B may be assigned to tooth number 2. Accordingly, the electronic device (100) may store the margin lines corresponding to each of the first library teeth as position information of the three-dimensional tooth image. The position information may include the center point of the margin line. In addition, the electronic device (100) may place the first library teeth in three-dimensional space based on the position information (e.g., margin lines) corresponding to each of the first library teeth.

[0108] In another embodiment, the electronic device (100) may generate a two-dimensional image of the oral cavity based on a three-dimensional image of the oral cavity. The electronic device (100) may generate a two-dimensional image of the oral cavity corresponding to the tooth region using the tooth region obtained as a result of segmentation in the three-dimensional image of the oral cavity. The electronic device (100) may identify a plurality of teeth included in the two-dimensional image using a tooth number identification model and determine the two-dimensional center coordinates of each tooth. For example, the electronic device (100) may determine the two-dimensional center coordinates of tooth number 6, tooth number 7, etc. The tooth number identification model may be an artificial neural network model, and may receive a two-dimensional image of the oral cavity and perform an inference process to identify teeth in the two-dimensional image of the oral cavity. The electronic device (100) may identify each of the plurality of teeth in the two-dimensional image using the tooth number identification model. The electronic device (100) may determine the tooth number of each of the plurality of teeth in the two-dimensional image using the tooth number identification model. The tooth number can be determined based on any system that classifies multiple teeth (e.g., Federation Dentaire International (FDI) or a universal numbering system). For example, the tooth number identification model can receive a two-dimensional image and output multiple tooth numbers from the two-dimensional image based on at least one of colors indicating the shape and curvature of the teeth in the two-dimensional image. The tooth number identification model can be a model in which a correlation between the two-dimensional image and the tooth number is learned. The electronic device (100) can identify multiple teeth from the two-dimensional image based on the inference result of the tooth number identification model including the tooth number of each tooth.The electronic device (100) can place the first library teeth in a three-dimensional space using position information corresponding to the tooth number in the two-dimensional image.

[0109] In another embodiment, the electronic device (100) may place the first library teeth in a three-dimensional space using the tooth region obtained as a result of segmentation. The electronic device (100) may identify a tooth number corresponding to the tooth region obtained as a result of segmentation. The electronic device (100) may compare a first tooth shape corresponding to the tooth region of a three-dimensional image of the oral cavity with a second tooth shape corresponding to the tooth number using a tooth number identification model, and may identify a tooth number having a tooth shape most similar to the first tooth shape. The tooth number identification model may be a model trained to obtain similarity between images. The tooth number identification model may identify a second tooth shape similar to the first tooth shape, which is an input three-dimensional image, and output a specific tooth number corresponding to the second tooth shape. The tooth number identification model may be a model that has learned a correlation between a tooth shape, which is a three-dimensional image, and a tooth number. In one embodiment, the electronic device (100) may place the first library teeth in a three-dimensional space based on the tooth number. Tooth numbers can indicate the location information of teeth in the dental arch. Accordingly, the electronic device (100) can know the location information corresponding to a specific tooth number. For example, the electronic device (100) can know where tooth number 6 is located in the dental arch and where tooth number 7 is located in the dental arch. The electronic device (100) can use the location information corresponding to the tooth numbers to place the first library teeth in a three-dimensional space. Through this, the first library teeth (440) can be placed in the three-dimensional space in the same manner as the oral structure in the three-dimensional image (400) of the oral cavity.

[0110] FIG. 6 is a drawing for explaining a method for selecting multiple teeth of the present disclosure.

[0111] The screen (600) may display a tooth arrangement and tooth numbers. A user may select multiple tooth numbers. The electronic device (100) may receive an input for selecting a tooth number for each of a plurality of teeth in the target oral cavity. For example, the user may select adjacent teeth numbers 6 (610), 7 (620), 8 (630), and 9 (640). As another example, the user may select non-adjacent teeth numbers 1, 5, 14, and 19.

[0112] In one embodiment, the electronic device (100) may receive an input for selecting multiple teeth and display library teeth corresponding to the selected tooth numbers by overlaying them on a three-dimensional image (400) of the oral cavity.

[0113] Through the screen (600), the user can select the type of prosthesis to be created for the selected tooth number. For example, if the user selects a crown after selecting a tooth number, subsequent processing may follow to place a corresponding library tooth on the selected tooth to generate data for manufacturing a crown. In another example, if the user selects an inlay after selecting a tooth number, subsequent processing may follow to place a corresponding library tooth on the selected tooth to generate data for manufacturing an inlay. Additionally, the user may also add a connector (650) to the selected tooth number.

[0114] FIG. 7 is a drawing for explaining second position information in a three-dimensional image of the oral cavity of the present disclosure.

[0115] The second library teeth and the three-dimensional image of the oral cavity may be arranged on different planes. Accordingly, the electronic device (100) may move the second library teeth so that the second library teeth corresponding to the plurality of teeth are arranged in the space corresponding to the plurality of teeth in the three-dimensional image of the oral cavity. To this end, the electronic device (100) may use first position information corresponding to each second library tooth and second position information for each tooth in the three-dimensional image of the oral cavity.

[0116] In one embodiment, the electronic device (100) can move a plurality of second library teeth in a three-dimensional space so that a plurality of second library teeth corresponding to a plurality of teeth among the first library teeth are arranged in a space corresponding to a plurality of teeth among the three-dimensional image of the oral cavity.

[0117] In one embodiment, the electronic device (100) may generate second position information including the midpoint of a second bounding box for each tooth in a three-dimensional image of a target oral cavity. The second position information may be generated for each tooth included in the three-dimensional image of the oral cavity. For example, tooth number 6 (610), tooth number 8 (630), and tooth number 9 (640) may respectively correspond to a second bounding box A (710), a second bounding box C (730), and a second bounding box D (740). A bounding box refers to a minimum rectangular outline that includes an object and is mainly used in computer graphics and data analysis. A three-dimensional bounding box is an extension of this concept to a three-dimensional space. It may be defined as a three-dimensional rectangular parallelepiped that includes a specified object as narrowly as possible. The first bounding box is for library teeth, and the second bounding box is for teeth included in the three-dimensional image of the oral cavity.

[0118] In another embodiment, the electronic device (110) may cause the second position information of the deleted tooth to include a point in the arch corresponding to the tooth number. The deleted tooth may not generate a second bounding box. Therefore, the electronic device (110) may determine the point at which the tooth number corresponding to the deleted tooth is located in the arch as the second position information of the deleted tooth. For example, if tooth number 7 (620) is the deleted tooth, the electronic device (100) may determine the point (725) corresponding to tooth number 7 in the arch to include the second position information. By locating the point corresponding to the deleted tooth in the arch, the electronic device (100) can superimpose the library tooth corresponding to the deleted tooth on a three-dimensional image of the oral cavity in its normal position.

[0119] The arch can be represented by a curve. The electronic device (100) can determine the arch based on the center point of the second bounding box for each tooth in the three-dimensional image of the oral cavity. The arch can be positioned and shaped to pass through the center point of as many second bounding boxes as possible. The curve representing the arch may not be displayed on the screen.

[0120] In another embodiment, the electronic device (110) may store second location information by changing the midpoint of the second bounding box of the prep tooth. The electronic device (100) may store second location information in which the midpoint of the second bounding box is changed based on the difference between the top height of the prep tooth and the top height of the library tooth corresponding to the prep tooth. For example, tooth number 6 (610), tooth number 8 (630), and tooth number 9 (640) may each be prep teeth. In addition, the midpoints of the second bounding box A (710), the second bounding box C (730), and the second bounding box D (740) may be 713, 733, and 743, respectively. However, due to the characteristics of the prep tooth, the size or height may be smaller than that of a normal tooth. Therefore, if the first position information of the library tooth is changed based on the midpoint of the second bounding box of the prep tooth, a large area of ​​the library tooth may be covered by the gingival area. Therefore, the electronic device (100) may change the second position information of the prep tooth to include a corrected point rather than the midpoint of the second bounding box. For example, the electronic device (100) may move the midpoint of the second bounding box by the amount of the difference between the top height of the prep tooth and the top height of the library tooth corresponding to the prep tooth. For example, the electronic device (100) may move the midpoints 713, 733, and 743 of the second bounding box A (710), the second bounding box C (730), and the second bounding box D (740), respectively, to positions 715, 735, and 745, and include the corresponding points in the second position information. For example, the height difference between points (713) and (715) may be the difference between the top height of tooth number 6 (610) and the top height of the library tooth number 6.

[0121] Fig. 8 is a drawing for explaining the first position information of the library teeth of the present disclosure.

[0122] In one embodiment, the electronic device (100) may generate first position information including the midpoint of a first bounding box for each of the plurality of library teeth based on each of the plurality of margin lines. The first position information may refer to position information of the library teeth in a three-dimensional space. The first library teeth may be aligned based on the plurality of margin lines. Accordingly, the plurality of library teeth may be arranged in the three-dimensional space to be identical to the tooth structure in a three-dimensional image of the oral cavity. As a result of the arrangement, each library tooth may have a coordinate in the three-dimensional space. As one example, a bounding box may be used to determine the coordinate of the library tooth. The electronic device (100) may generate a first bounding box for each of the plurality of library teeth (e.g., the second library tooth). For example, the electronic device (100) can generate a first bounding box A (810), a first bounding box B (820), a first bounding box C (830), and a first bounding box D (840) for tooth numbers 6 (610), 7 (620), 8 (630), and 9 (640), respectively. The electronic device (100) can include the midpoint of the first bounding box in the first location information. For example, the electronic device (100) can include the midpoint (815) of the first bounding box A (810) of the 6th library tooth (611) corresponding to tooth number 6 (610) in the first location information. Similarly, the first position information of the 7th library tooth (621) may include the center point (825) of the first bounding box B (820), the first position information of the 8th library tooth (631) may include the center point (835) of the first bounding box C (830), and the first position information of the 9th library tooth (641) may include the center point (845) of the first bounding box D (840).

[0123] In one embodiment, a library tooth corresponding to a deleted tooth may serve as the basis for the shape of a prosthesis corresponding to the deleted tooth. For example, referring to FIG. 7 , tooth 7 (620) may be the deleted tooth. Accordingly, library tooth 7 (621) may serve as the basis for the shape of a prosthesis corresponding to the deleted tooth.

[0124] In one embodiment, the library teeth corresponding to the prep teeth may serve as the basis for the shape of the prosthesis corresponding to the prep teeth. For example, referring to FIG. 7 , tooth 6 (610), tooth 8 (630), and tooth 9 (640) may be prep teeth. Accordingly, library tooth 6 (611), library tooth 8 (631), and library tooth 9 (641) may serve as the basis for the shape of the prosthesis corresponding to the prep teeth.

[0125] In one embodiment, upon determining that a plurality of library teeth corresponding to a plurality of teeth are connected by connectors, the electronic device (100) may include connectors between the library teeth. For example, upon determining that library teeth 6 (611), 7 (621), 8 (631), and 9 (641) are connected by connectors, the electronic device (100) may represent connectors between each library tooth as a three-dimensional image. The connectors may be represented based on mesh information. A bridge is one of the dental treatments used to replace a deleted tooth. Unlike a prep tooth, a deleted tooth has no target to which a prosthesis is attached, so a bridge is usually configured as a structure that secures an artificial tooth using adjacent teeth as support. A bridge may include connectors that connect prostheses to each other. Depending on the material, the bridge may be made of metal, porcelain, or a composite material thereof, and is provided in various forms depending on the patient's needs and oral condition.

[0126] FIG. 9 is an exemplary drawing illustrating how the library teeth described with reference to FIG. 8 are superimposed on the three-dimensional image of the oral cavity described with reference to FIG. 7.

[0127] In one embodiment, the electronic device (100) can move a library tooth (e.g., a second library tooth) to a position of a tooth in a three-dimensional image of the oral cavity with the same tooth number relative to an origin (e.g., an origin located in the arch of the oral cavity).

[0128] In one embodiment, the electronic device (100) may generate a correlation between first position information and second position information of each of a plurality of second library teeth based on an origin located in the dental arch of the oral cavity. The correlation between the first position information and the second position information may be generated for each tooth. For example, a correlation between the first position information and the second position information may be generated for each tooth numbered 1, 2, . . . 32. The dental arch refers to an anatomical structure related to the dental row in which the teeth of the upper and lower jaws are arranged, and includes a form in which the teeth are arranged continuously in an arch shape. The upper and lower jaws are each divided into the maxillary arch and the mandibular arch, and each arch has the characteristic of being arranged symmetrically left and right with respect to the midline. The origin located in the dental arch may be determined as a point between two central incisors in the dental arch. For example, in the maxillary arch, the point between teeth numbered 8 and 9 may be determined as the origin, and in the mandibular arch, the point between teeth numbered 24 and 25 may be determined as the origin. As another example, the origin may be the intersection of the maxillary arch and the midline. The origin may exist differently in the maxilla and the mandible. For example, when the teeth are located in the maxilla, the electronic device (1000) may generate a correlation based on the origin of the maxilla. When the teeth are located in the mandible, the electronic device (100) may generate a correlation based on the origin of the mandible.

[0129] In one embodiment, the electronic device (100) may generate a tooth-specific transformation matrix indicating a correlation between first position information of each of a plurality of library teeth with respect to the origin and second position information having the same tooth number as each of a plurality of second library teeth. The tooth-specific transformation matrix may be a matrix that moves a point included in the first position information to the origin and then moves the point to a point included in the second position information.

[0130] For example, the transformation matrix of tooth number 6 (610) can indicate a correlation between the first position information of the library tooth number 6 (611) with respect to the origin (601) of the maxillary arch and the second position information of the tooth number 6 (610) in the three-dimensional image of the oral cavity. As another example, the transformation matrix of tooth number 8 (630) can indicate a correlation between the first position information of the library tooth number 8 (631) with respect to the origin of the maxillary arch and the second position information of the tooth number 8 (630) in the three-dimensional image of the oral cavity.

[0131] In one embodiment, the electronic device (100) can display a three-dimensional image of a target oral cavity and a plurality of second library teeth by overlapping them. The electronic device (100) can display the plurality of second library teeth by overlapping them on a three-dimensional image of the oral cavity having the same tooth number as each of the plurality of second library teeth in a three-dimensional space based on a tooth-specific transformation matrix. Specifically, the electronic device (100) can apply the tooth-specific transformation matrix to all points of the mesh included in the second library teeth to position all points of the mesh included in the second library teeth in the three-dimensional image of the oral cavity. For example, the electronic device (100) can apply the tooth-6 transformation matrix to all points of the mesh included in the library tooth (611) to move all points of the mesh included in the library tooth (611) to a location where the tooth 6 is located in the three-dimensional image of the oral cavity.

[0132] Through this, the electronic device (100) can display the 6th library tooth (611) by overlapping the position of the 6th tooth (610) in the 3D image of the oral cavity (e.g., the position based on the second position information). Similarly, the electronic device (100) can display the 7th, 8th, and 9th library teeth (621, 631, 641) by overlapping the positions of the 7th, 8th, and 9th teeth (620, 630, 640) in the 3D image of the oral cavity, respectively.

[0133] In the present disclosure, the operation of displaying a plurality of library teeth by overlapping them on a three-dimensional image of the oral cavity is to display the library teeth by overlapping them while maintaining the three-dimensional image of the oral cavity, so that when the library teeth are moved, the teeth of the existing three-dimensional image of the oral cavity can be displayed.

[0134] In one embodiment, the electronic device (100) may display a library tooth corresponding to the deleted tooth by overlaying it on a three-dimensional image of the target oral cavity based on a tooth-specific transformation matrix corresponding to the deleted tooth. For example, if the deleted tooth is tooth number 7 (620), the electronic device (100) may move library tooth number 7 (621) to the position of tooth number 7 (620) based on the transformation matrix of tooth number 7 (620).

[0135] In one embodiment, the electronic device (100) may display a library tooth corresponding to the prep tooth by overlaying it on a three-dimensional image of the target oral cavity based on a tooth-specific transformation matrix corresponding to the prep tooth. For example, if the prep tooth is tooth number 6 (610), the electronic device (100) may move the library tooth number 6 (611) to the position of tooth number 6 (610) based on the transformation matrix of tooth number 6 (610).

[0136] In one embodiment, the electronic device (100) may display connectors between a plurality of library teeth based on a determination that a plurality of library teeth corresponding to a plurality of adjacent teeth are connected by connectors. For example, the electronic device (100) may display a connector (910) to connect between the 7th library tooth (621) and the 8th library tooth (631) based on a determination that a connector is connected between the 7th library tooth (621) and the 8th library tooth (631).

[0137] FIG. 10 is a flowchart illustrating a method in which a server superimposes library teeth onto a three-dimensional image of an oral cavity in the present disclosure.

[0138] In one embodiment, the server can place a three-dimensional image of the target oral cavity in a three-dimensional space (S1010).

[0139] In one embodiment, the server may receive an input for selecting multiple teeth of a target oral cavity (S1020).

[0140] In one embodiment, the server can place the first library teeth corresponding to the three-dimensional image of the target oral cavity in a three-dimensional space (S1030).

[0141] In one embodiment, the server can move a plurality of second library teeth in a three-dimensional space so that a plurality of second library teeth corresponding to a plurality of teeth among the first library teeth are placed in a space corresponding to a plurality of teeth among the three-dimensional image of the target oral cavity (S1040).

[0142] In one embodiment, the server can display a three-dimensional image of the target oral cavity and a plurality of second library teeth by overlapping them (S1050).

[0143] Various embodiments of the present disclosure and effects according to the embodiments have been described with reference to FIGS. 1 through 10 so far. The effects according to the technical concept of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description in the specification.

[0144] In addition, the technical idea of ​​the present disclosure described with reference to FIGS. 1 to 10 so far may be implemented as a computer-readable code on a recording medium that a computing device (e.g., an electronic device) can read. Here, a computer program implemented as a code may include at least one instruction that, when loaded into a memory of the computing device, causes a processor of the computing device to perform operations according to various embodiments of the present disclosure. Such a computer-readable recording medium may be, for example, a removable recording medium (e.g., a CD, a DVD, a Blu-ray disc, a USB storage device, a removable hard disk, etc.) or a fixed recording medium (e.g., a ROM, a RAM, a computer-attached hard disk, etc.). In addition, the recording medium may be a non-transitory recording medium. Here, a non-transitory recording medium means a tangible medium that exists regardless of whether data is stored semi-permanently or temporarily, and may not include a signal that is propagated transitorily. Furthermore, a computer program recorded on a recording medium can be transmitted to another computing device via a network such as the Internet and installed on the other computing device, thereby allowing the computer program to be used on the other computing device.

[0145] While the technical concepts of the present disclosure have been described above with reference to certain embodiments and examples illustrated in the accompanying drawings, it should be understood that various substitutions, modifications, and variations may be made without departing from the technical scope of the present disclosure, which would be understood by those skilled in the art. Furthermore, such substitutions, modifications, and variations should be considered to fall within the scope of the appended claims.

Claims

1. In an image processing method performed by an electronic device, A step of arranging a three-dimensional image of a target oral cavity in a three-dimensional space; A step of receiving an input for selecting a plurality of teeth of the target oral cavity; A step of arranging a first library tooth corresponding to a three-dimensional image of the target oral cavity in the three-dimensional space; A step of moving a plurality of second library teeth in the three-dimensional space so that a plurality of second library teeth corresponding to the plurality of teeth among the first library teeth are arranged in a space corresponding to the plurality of teeth among the three-dimensional image of the target oral cavity; and A step of displaying a three-dimensional image of the above target oral cavity and the plurality of second library teeth by overlapping them. How to include.

2. In paragraph 1, A step of obtaining multiple margin lines corresponding to each of the multiple teeth Including more, The step of placing the first library tooth on the three-dimensional space is: A step of arranging the first library tooth on the three-dimensional space based on the plurality of margin lines, A method wherein each of the plurality of margin lines represents a boundary where an intact tooth begins to be exposed in the gingival area of ​​the three-dimensional image of the target oral cavity.

3. In paragraph 2, The step of obtaining the above multiple margin lines is A step of obtaining the plurality of margin lines based on the boundary between the tooth area and the gingival area obtained as a result of segmentation; or A step of obtaining the plurality of margin lines based on user input A method comprising:

4. In paragraph 1, The step of placing the first library tooth on the three-dimensional space is: A step of identifying a tooth number corresponding to a tooth area obtained as a result of segmentation; and A step of arranging the first library tooth in the three-dimensional space based on the tooth number. A method comprising:

5. In paragraph 1, Step of creating the first library tooth A method further comprising:

6. In paragraph 5, The step of creating the first library tooth is as follows: A step of increasing the first library tooth in the direction of the gingival region based on mesh information of at least a part of the first library tooth corresponding to the tooth region and gingival region included in the three-dimensional image of the target oral cavity. A method comprising:

7. In paragraph 6, The step of creating the first library tooth is as follows: A step of processing an increased area in the increased first library tooth based on a plurality of margin lines. A method further comprising:

8. In paragraph 1, The step of receiving an input for selecting multiple teeth of the target oral cavity is as follows: A step of receiving an input for selecting a tooth number for each of a plurality of teeth in the above target oral cavity. A method comprising:

9. In paragraph 1, The step of moving the plurality of second library teeth in the three-dimensional space is: A step of generating first position information including the center of the first bounding box for each of the plurality of second library teeth based on each of the plurality of margin lines; A step of generating second position information including the center of the second bounding box for each tooth of the three-dimensional image of the target oral cavity; and A step of generating a correlation between the first position information of each of the plurality of second library teeth and the second position information based on the origin located in the arch of the target oral cavity. A method comprising:

10. In paragraph 9, The steps for generating the above correlation are: A step of generating a tooth-specific transformation matrix indicating a correlation between first position information of each of the plurality of second library teeth based on the origin and second position information having the same tooth number as each of the plurality of second library teeth; A method comprising:

11. In paragraph 10, The step of displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth by overlapping them is: A step of displaying the plurality of second library teeth by overlapping them on a three-dimensional image of the target oral cavity having the same tooth number as each of the plurality of second library teeth in the three-dimensional space based on the tooth-by-teeth transformation matrix; A method comprising:

12. In paragraph 11, The above origin is the point between the two central incisors in the above arch, method.

13. In paragraph 1, The step of displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth by overlapping them is: A method comprising the step of displaying a library tooth corresponding to a deleted tooth by overlaying it on a three-dimensional image of the target oral cavity based on a tooth-by-tooth transformation matrix corresponding to the deleted tooth.

14. In paragraph 13, The tooth-specific transformation matrix corresponding to the deleted tooth is, It is generated based on second location information including a point in the arch corresponding to the tooth number, The library tooth corresponding to the deleted tooth includes a prosthesis corresponding to the deleted tooth. method.

15. In paragraph 1, The step of displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth by overlapping them is: A step of displaying a library tooth corresponding to a prep tooth by overlaying it on a three-dimensional image of the target oral cavity based on a tooth-specific transformation matrix corresponding to the prep tooth. A method comprising:

16. In paragraph 15, The tooth-specific transformation matrix corresponding to the above prep tooth is, The second position information is generated based on the changed center point of the second bounding box based on the difference between the top height of the above prep tooth and the top height of the library tooth corresponding to the above prep tooth, The library tooth corresponding to the above prep tooth includes a prosthesis corresponding to the above prep tooth. method.

17. In paragraph 1, The step of displaying the three-dimensional image of the target oral cavity and the plurality of second library teeth by overlapping them is: A method comprising the step of displaying the connector between the plurality of second library teeth, based on a determination that a plurality of second library teeth corresponding to the plurality of teeth are connected by connectors.

18. Processor; A network interface that communicates with the oral scanner; display; memory; and A computer program loaded into the above memory and executed by the above processor, The above computer program, Place the 3D image of the target oral cavity in 3D space, Receive an input for selecting multiple teeth of the target oral cavity, The first library tooth corresponding to the three-dimensional image of the target oral cavity is placed in the three-dimensional space, Moving the plurality of second library teeth in the three-dimensional space so that the plurality of second library teeth corresponding to the plurality of teeth among the first library teeth are arranged in the space corresponding to the plurality of teeth among the three-dimensional image of the target oral cavity, and Displaying a three-dimensional image of the target oral cavity and the plurality of second library teeth by overlapping them, Electronic devices.

19. In a non-transitory computer-readable recording medium recording a computer program to be executed by a processor, The above computer program, Instructions for placing a three-dimensional image of a target oral cavity in three-dimensional space; Instructions for receiving an input for selecting a plurality of teeth of the target oral cavity; An instruction for placing a first library tooth corresponding to a three-dimensional image of the target oral cavity in the three-dimensional space; An instruction for moving a plurality of second library teeth in the three-dimensional space so that a plurality of second library teeth corresponding to the plurality of teeth among the first library teeth are arranged in a space corresponding to the plurality of teeth among the three-dimensional image of the target oral cavity; and An instruction for displaying a three-dimensional image of the target oral cavity and the plurality of second library teeth in an overlapping manner, Computer readable recording medium.

Images