Oral scanning system

Abstract

The present invention relates to an oral scanning system comprising: an oral scanner for scanning the oral cavity of a patient so as to generate a plurality of scanning images; a smart glass having a camera module; and a scanning management terminal linked with the oral scanner and the smart glass, wherein the scanning management terminal receives the plurality of scanning images from the oral scanner so as to generate a three-dimensional image model for the oral cavity of the patient, divides, into a first area and a second area, a camera image captured by the camera module and sets the first area to include an oral area of the patient, and extracts a scanner area corresponding to the oral scanner from the camera image so that when deviation of the scanner area from the first area during operation is detected, the scanner is controlled to stop operation.

Description

Oral scanning system

[0001] The present invention relates to an oral scanning system, and more specifically, to an oral scanning system to which augmented reality technology is applied.

[0002] In general, human teeth facilitate digestion by chewing food and play an important role in pronunciation. When abnormalities occur in these teeth, the condition of the teeth is diagnosed and treated, and if necessary, artificial implants are implanted to replace the teeth.

[0003] When a tooth is lost, an artificial tooth is implanted through a dental implant procedure. During the implant procedure, when fabricating a prosthesis using an abutment, the final prosthesis, the round, is placed over a custom abutment; therefore, it is necessary to accurately measure the shape of the abutment in three dimensions.

[0004] Recently, there have been active attempts to directly measure abutments in the patient's oral cavity using an oral scanner, that is, to perform 3D scanning and utilize this for prosthetic design.

[0005] For example, Korean Published Patent No. 2021-0101936 discloses "method, apparatus, and recording medium for acquiring oral scanner images." The technology disclosed in the above Korean Published Patent encrypts only the original image and oral scanner information acquired through an oral scanner at a user terminal and transmits them to a server, and the server generates an oral scanner image using the received original image and oral scanner information.

[0006] The device disclosed in the aforementioned Korean published patent includes an oral scanner and an information processing device for processing an image scanned by the oral scanner, such as a computer-based terminal.

[0007] It is common for the above terminal to have a program installed to manage the oral scanner, such as processing images scanned by the oral scanner through communication with a server and transmitting the scanned images to the server.

[0008] Therefore, it is common for manufacturers of oral scanners to sell the scanner along with a terminal equipped with a management program, or to install the program on a PC provided at the dental clinic.

[0009] In the process of scanning a patient's oral cavity using the oral scanner described above, the scanned information or images are displayed on the monitor of a terminal with the program installed. Accordingly, during the scanning process, as illustrated in Fig. 1, the dentist or nurse scans while looking at the side of the patient's oral cavity where the oral scanner is inserted, and looks at the monitor to check the scanned images. In other words, there is the inconvenience of having to alternate between looking at the side of the patient's oral cavity and the monitor.

[0010] In addition, oral scanners are typically manufactured in a size that can be held by hand; therefore, instead of implementing various control buttons on the scanner, input signals during the scanning process using the oral scanner are generated through a program installed on the terminal.

[0011] Even with such a configuration, since input devices of the terminal, such as a keyboard, mouse, or touchscreen, must be operated to generate input signals during the scanning process, it causes inconvenience in a restricted usage environment where the oral scanner is being held.

[0012] Accordingly, the present invention has been devised to resolve the above-mentioned problems and aims to provide an oral scanning system that provides an environment in which scanned images or various information can be easily visually recognized.

[0013] In addition, another objective of the present invention is to provide an oral scanning system that can automatically terminate based on the user's movement during the oral scanning process.

[0014] In addition, another objective of the present invention is to provide an oral scanning system capable of providing a user environment that ensures user convenience in the generation of input signals during the oral scanning process.

[0015] The above objective is achieved by an oral scanning system according to the present invention, comprising: an oral scanner that scans a patient's oral cavity to generate a plurality of scanned images; smart glasses having a camera module; and a scanning management terminal that interacts with the oral scanner and the smart glasses; wherein the scanning management terminal receives a plurality of scanned images from the oral scanner to generate a three-dimensional image model of the patient's oral cavity, distinguishes a camera image captured by the camera module into a first region and a second region such that the first region includes the patient's oral cavity region within the camera image, extracts a scanner region corresponding to the oral scanner from the camera image, and controls the oral scanner to stop the operation of the oral scanner when it is detected that the scanner region deviates from the first region while the oral scanner is in operation.

[0016] Here, the smart glasses include a transparent display portion on which an image is displayed; and the scanning management terminal can generate a glass image to be displayed on the transparent display portion of the smart glasses and control the smart glasses so that the glass image is displayed on the second area on the transparent display portion.

[0017] In addition, the scanning management terminal operates in either a first mode or a second mode to divide the camera image into the first area and the second area; the first mode operates when the patient's face is covered with a bleaching cloth, and the second mode can operate when the patient's face is not covered with the bleaching cloth.

[0018] In one embodiment, the scanning management terminal detects whether a small pore area corresponding to the small pore is included within the camera image, operates in the first mode when the small pore area is detected, and operates in the second mode when the small pore area is not detected.

[0019] In one embodiment, at least one marker is attached to the small pore; and the scanning management terminal can operate in the first mode by determining that the small pore area has been detected when the marker is recognized within the camera image.

[0020] Here, when the scanning management terminal operates in the first mode, it extracts a perforated area among the small hole areas corresponding to the small hole within the camera image; classifies the perforated area as the first area, and classifies an area other than the perforated area as the second area.

[0021] And, a plurality of the above markers are attached around the perforated area of ​​the above small hole; and the scanning management terminal can extract the perforated area based on the positions of the plurality of markers recognized from the camera image.

[0022] In one embodiment, when the scanning management terminal operates in the second mode, it extracts a face region for the patient's face within the camera image; divides the face region into the first region and divides the region other than the face region into the second region.

[0023] In one embodiment, the glass image may include a 2D oral image of the 3D image model viewed from a specific view direction, and at least one of an input menu for a control input of at least one of the oral scanner and the scanning management terminal.

[0024] In addition, the scanning management terminal can control the oral scanner to operate when it detects that the scanner area is being brought into the first area.

[0025] According to the above configuration, the present invention provides an oral scanning system in which, based on an image captured by a camera module, the oral scanner detects that it has moved out of the patient's oral area, thereby allowing the scanning to be automatically terminated.

[0026] In addition, an oral scanning system is provided that displays scanned images or various information on smart glasses worn by the user, thereby providing an environment that can be easily visually perceived.

[0027] In addition, an oral scanning system is provided that can provide a user environment where user convenience is ensured in the generation of input signals during the oral scanning process.

[0028] FIG. 1 is a diagram showing an example of an oral scanning environment in a typical dental clinic, and

[0029] FIG. 2 is a diagram showing an example of the configuration of an oral scanning system according to an embodiment of the present invention, and

[0030] FIG. 3 is a diagram showing an example of the configuration of smart glasses according to an embodiment of the present invention, and

[0031] FIG. 4 is a diagram showing an example of the configuration of a scanning management terminal according to an embodiment of the present invention, and

[0032] FIG. 5 is a diagram illustrating the usage environment depending on whether a small gun is applied, and

[0033] FIGS. 6 to 8 are drawings showing examples of glass images displayed on smart glasses according to an embodiment of the present invention, and

[0034] FIG. 9 is a diagram showing an example in which a marker is attached to a small pore in an oral scanning system according to an embodiment of the present invention, and

[0035] FIG. 10 is a diagram illustrating an example of a method for recognizing a marker in an oral scanning system according to an embodiment of the present invention, and

[0036] FIG. 11 is a diagram illustrating the automatic termination of an oral scanner in an oral scanning system according to an embodiment of the present invention, and

[0037] FIG. 12 is a diagram showing an example of an input menu displayed on smart glasses according to an embodiment of the present invention, and

[0038] FIG. 13 is a diagram showing an example of a gesture applied to a smart glass system according to an embodiment of the present invention.

[0039] The present invention relates to an oral scanning system comprising an oral scanner that scans a patient's oral cavity to generate a plurality of scanning images, smart glasses having a camera module, and a scanning management terminal linked with the oral scanner and the smart glasses; wherein the scanning management terminal receives a plurality of the scanning images from the oral scanner and generates a three-dimensional image model of the patient's oral cavity, distinguishes a camera image captured by the camera module into a first area and a second area such that the first area includes the patient's oral cavity area within the camera image, extracts a scanner area corresponding to the oral scanner from the camera image, and controls the oral scanner to stop the operation of the oral scanner when it is detected that the scanner area deviates from the first area while the oral scanner is in operation.

[0040] The advantages and features of the present invention and the methods for achieving them will become clear by referring to the embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below but may be implemented in various different forms. These embodiments are provided merely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the present invention, and the present invention is defined only by the scope of the claims.

[0041] The terms used in this specification are for describing embodiments and are not intended to limit the invention. In this specification, the singular form includes the plural form unless specifically stated otherwise in the text. The terms "comprises" and / or "comprising" used in this specification do not exclude the presence or addition of one or more other components in addition to the components mentioned. Throughout the specification, the same reference numerals refer to the same components, and "and / or" includes each of the mentioned components and all combinations of one or more. Although terms such as "first," "second," etc., are used to describe various components, these components are not limited by these terms. These terms are used merely to distinguish one component from another. Therefore, the first component mentioned below may be the second component within the technical scope of the invention.

[0042] Unless otherwise defined, all terms used herein (including technical and scientific terms) may be used in a meaning commonly understood by those skilled in the art to which the present invention pertains. Additionally, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise.

[0043] Hereinafter, embodiments according to the present invention will be described in detail with reference to the attached drawings.

[0044] FIG. 2 is a diagram showing an example of the configuration of an oral scanning system (100) according to an embodiment of the present invention.

[0045] Referring to FIG. 2, an oral scanning system (100) according to an embodiment of the present invention may be configured to include an oral scanner (110), smart glasses (150), and a scanning management terminal (130). In one embodiment, the oral scanning system (100) may include a scanning management terminal (130) for managing the oral scanner (110) together with the manufacturer of the oral scanner (110).

[0046] An oral scanner (110) according to an embodiment of the present invention can scan a patient's oral cavity to generate a plurality of scanned images. In one embodiment, the oral scanner (110) may be equipped with various methods capable of capturing scanned images of teeth constituting the patient's oral cavity, such as a confocal microscope method, triangulation method, optical tomography system method, and fringe interferometry method, and the technical concept of the present invention is not limited to such methods.

[0047] A scanning management terminal (130) according to an embodiment of the present invention can manage scanned images scanned by an oral scanner (110). Here, the scanning management terminal (130) can communicate with a dental management terminal (not shown) by being connected to the dental management terminal through a communication network within the dental clinic. In one embodiment, the scanning management terminal (130) and the dental management terminal can communicate with each other through a wireless communication network such as wireless WI-FI, Bluetooth, or WiBro. In addition, the scanning management terminal (130) and the dental management terminal may be interconnected through a wired communication network such as a LAN.

[0048] The smart glasses (150) according to an embodiment of the present invention may be provided in the form of glasses that can be worn by a user, such as a dentist or a nurse, as shown in FIG. 2. For example, the smart glasses (150) may include a glass frame (151) forming a skeleton and a lens (151a) to which the glass frame (151) is fixed so as to be positioned in front of the wearer's eyes when the glass frame (151) is worn.

[0049] FIG. 3 is a diagram showing an example of the configuration of smart glasses (150) according to an embodiment of the present invention.

[0050] Referring to FIG. 3, a smart glass (150) according to an embodiment of the present invention may be configured to include a camera module (153) and a transparent display unit (152). Additionally, the smart glass (150) may be configured to include a glass interface unit (154) and a glass processor (155).

[0051] A transparent display unit (152) according to an embodiment of the present invention is made of a transparent material and displays an image on its surface. In an embodiment of the present invention, the transparent display unit (152) is exemplified as being composed of the entire lens (151a) constituting the smart glass (150), but it is understood that the transparent display unit (152) may be provided only in a part of the lens (151a).

[0052] In this way, depending on the configuration of the transparent display unit (152), even when the user is wearing smart glasses (150) or when the glass image (GI) described later is displayed on the transparent display unit (152), the actual image that passes through the transparent display unit (152), that is, the actual image seen by the user's eyes, can be viewed together with the glass image (GI) displayed on the transparent display unit (152).

[0053] A camera module (153) according to an embodiment of the present invention is installed in a smart glass (150) and can capture the front of the smart glass (150). In an embodiment of the present invention, as shown in FIG. 2, the camera module (153) is installed in the glass frame (151) of the smart glass (150) as an example, and is installed to capture an image in a viewing direction corresponding to the direction in which the user's eyes look when wearing the smart glass (150).

[0054] A glass interface unit (154) according to an embodiment of the present invention may be connected to a scanning management terminal (130) via a communication network. Here, wireless communication or wired communication may be applied for communication between the scanning management terminal (130) and the glass interface unit (154). In one embodiment, the glass interface unit (154) may communicate with the scanning management terminal (130) via TCP / IP-based Wi-Fi communication or Bluetooth communication, and other types of wired or wireless communication may also be applied.

[0055] A glass processor (155) according to an embodiment of the present invention can control the entire function of the smart glass (150). In one embodiment, the glass processor (155) controls communication with a scanning management terminal (130) through a glass interface unit (154) and can transmit a camera image captured by a camera module (153) to the scanning management terminal (130) through the glass interface unit (154).

[0056] In addition, the glass processor (155) can display a glass image (GI) to be described later, which is transmitted from the scanning management terminal (130) and received through the glass interface unit (154), on the transparent display unit (152), and a detailed explanation thereof will be provided later.

[0057] Meanwhile, the scanning management terminal (130) according to an embodiment of the present invention can receive a plurality of scanning images from an oral scanner (110) and generate a three-dimensional image model of the patient's oral cavity.

[0058] And, the scanning management terminal (130) can generate a glass image (GI) to be displayed on the transparent display part (152) of the smart glass (150) and transmit the generated image to the smart glass (150) through a communication network.

[0059] FIG. 4 is a diagram showing an example of the configuration of a scanning management terminal (130) according to an embodiment of the present invention.

[0060] Referring to FIG. 4, a scanning management terminal (130) according to an embodiment of the present invention may be configured to include a scanning interface unit (131), a scanning management program (132), and a scanning terminal processor (133). Additionally, the scanning management terminal (130) may be configured to include a data storage unit (136) and a scanning terminal monitor (134).

[0061] A scanning interface unit (131) according to an embodiment of the present invention is connected to an oral scanner (110) via a communication network and can receive a scanning image captured by the oral scanner (110). In one embodiment, the scanning interface unit (131) may be connected to the oral scanner (110) via a wired or wireless communication method. Here, the technical concept of the present invention is not limited to the method of connection between the scanning interface unit (131) and the oral scanner (110).

[0062] Additionally, the scanning interface unit (131) can communicate with the smart glasses (150) through a communication network. Here, the communication method between the smart glasses (150) and the scanning management terminal (130) can be wired or wireless as described above.

[0063] In FIG. 2, a single communication network is illustrated and described as a form in which the smart glasses (150) and the scanning management terminal (130) and the oral scanner (110) and the scanning management terminal (130) communicate, but it is obvious that the communication method between the smart glasses (150) and the scanning management terminal (130) and the communication method between the oral scanner (110) and the scanning management terminal (130) may be the same or different from each other.

[0064] A scanning terminal processor (133) according to an embodiment of the present invention may include a hardware configuration and a software configuration for controlling the overall functions of a scanning management terminal (130). In one embodiment, when the scanning management terminal (130) is implemented in the form of a PC, the hardware configuration may include a CPU, RAM, a main board, etc., and the software configuration may include an operating system.

[0065] The scanning management program (132) can be installed on the scanning management terminal (130). The scanning management program (132) can manage the scanning images captured and received by the oral scanner (110) through the scanning interface unit (131) by linking with the scanning terminal processor (133).

[0066] In one embodiment, the scanning management program (132) may be configured to include a 3D model generation unit (132a). The 3D model generation unit (132a) can generate a three-dimensional image model of a patient's oral cavity using a plurality of scanned images transmitted from an oral scanner (110).

[0067] In an embodiment of the present invention, the scanning image transmitted from the oral scanner (110) is exemplified as having an RGB-D data format. That is, it may be a three-dimensional image format in which depth information is included in each pixel within the RGB image. Here, the 3D model generation unit (132a) generates a three-dimensional image model of the oral cavity by aligning the scanning image containing depth information through an image processing process.

[0068] Here, the 3D model generation unit (132a) according to an embodiment of the present invention can store at least a portion of a plurality of scanning images received from an oral scanner (110) and a generated 3D image model in a data storage unit (136).

[0069] A scanning management program (132) according to an embodiment of the present invention may be configured to include a glass image generation unit (132d). In one embodiment, the glass image generation unit (132d) may generate a glass image (GI) to be displayed on a transparent display unit (152) of a smart glass (150), and a detailed description thereof will be provided later.

[0070] Meanwhile, the scanning management program (132) according to an embodiment of the present invention can divide the camera image captured by the camera module (153) of the smart glasses (150) and received through the scanning interface unit (131) into a first area and a second area. In addition, the scanning management program (132) can control the smart glasses (150) so that the glass image (GI) generated by the glass image generation unit (132d) is displayed on the transparent display unit (152) of the smart glasses (150) at a location corresponding to the second area.

[0071] Here, the first region includes the patient's oral region within the camera image, and the second region corresponds to the region within the camera image that does not include the patient's oral region.

[0072] Through this, when displaying a glass image (GI) on the transparent display part (152) of the smart glasses (150) worn by the user during the process of scanning or treating the patient's oral cavity, the image is displayed in a second area that is an area other than the oral cavity area that is the target of treatment or scanning, so that the scanning image displayed on the transparent display part (152) does not obstruct the view of the user, that is, the dentist or nurse, during the scanning or treatment process.

[0073] In an embodiment of the present invention, the scanning management program (132) of the scanning management terminal (130) operates in either a first mode or a second mode to divide the camera image into a first area and a second area.

[0074] Referring to FIG. 5, as shown in FIG. 5(b), the first mode is operated with the patient's face covered by a small veil (200), and the second mode is operated with the patient's face not covered by a small veil (200), that is, with the patient's face exposed, as shown in FIG. 5(a).

[0075] To explain more specifically, generally, the dental treatment process is performed with the patient's face covered with a diaphragm (200), but the oral scanning process can be performed without covering the patient's face with a diaphragm (200). That is, during the oral scanning process, a situation may occur where the patient's face is covered with a diaphragm (200) or not.

[0076] As previously explained, the first region includes the patient's oral region and the second region does not include the patient's oral region, and it is desirable to display a glass image (GI) on the patient's face region excluding the oral region, and a detailed explanation thereof will be provided later.

[0077] Referring to FIGS. 6 to 8, as shown in FIG. 6, it is preferable to display a glass image (GI) in an area excluding the face area of ​​a patient not covered by a small veil (200) within the gaze area (VA) of a user wearing smart glasses (150).

[0078] On the other hand, as illustrated in FIG. 7, in the case of a patient covered with a small veil (200), the small veil (200) covers the face area excluding the oral area, so it is acceptable for a glass image (GI) to be displayed on the face area.

[0079] Accordingly, in an embodiment of the present invention, the scanning management program (132) of the scanning management terminal (130) operates in a first mode or a second mode depending on whether the small sack (200) is covered, thereby reflecting the situation as illustrated in FIGS. 6 and FIGS. 7.

[0080] In one embodiment, the scanning management program (132) is exemplified as operating in either a first mode or a second mode based on user input through the user input unit (135).

[0081] Referring to FIG. 4, the scanning management terminal (130) may be configured to include a user input unit (135). In one embodiment, a user may input various input signals to the scanning management terminal (130) through the user input unit (135). The user input unit (135) may be configured with at least one input device among a touch pad, a foot switch, a keyboard / mouse, or a microphone for voice recognition.

[0082] The user can select a first mode when the small veil (200) is covered on the patient's face through the user input section (135), and select a second mode when the small veil (200) is not covered on the patient's face.

[0083] In another embodiment, the scanning management program (132) of the scanning management terminal (130) can detect whether a small pore area corresponding to a small pore (200) is included within a camera image captured by the camera module (153) of the smart glasses (150). For example, the scanning management terminal (130) can detect an area with a specific color as a small pore area by using the color of the small pore (200), or detect whether a small pore area exists by detecting the boundary of a specific color.

[0084] And, the scanning management program (132) can operate in a first mode when a small gun area is detected and in a second mode when a small gun area is not detected.

[0085] Again, referring to FIG. 4, the scanning management program (132) according to an embodiment of the present invention may be configured to include a first detection model (132b) and a second detection model (132c) as shown in FIG. 4.

[0086] When the first detection model (132b) according to an embodiment of the present invention operates in a first mode, it can divide the camera image into a first area and a second area. And, when the second detection model (132c) operates in a second mode, it can divide the camera image into a first area and a second area.

[0087] In one embodiment, the first detection model (132b) can operate in a first mode and extract a perforated area (210) among the pore areas corresponding to the pore (200) within the camera image. That is, it can extract a perforated area (210) formed so that the oral cavity can be viewed by cutting out a portion of the interior of the pore (200). Then, the first detection model (132b) can classify the perforated area (210) as a first area and classify the area other than the perforated area (210) as a second area.

[0088] Through this, as shown in FIG. 7, areas other than the first area, the perforated area (210), for example, the small pore area, are also included in the area where the glass image (GI) is displayed, even if they correspond to the patient's face area, so that the glass image (GI) can be displayed.

[0089] Meanwhile, the second detection model (132c) can operate in a second mode and divide the camera image into a first area and a second area.

[0090] In one embodiment, the second detection model (132c) can extract a face region of the patient's face within a camera image while operating in a second mode. The second detection model (132c) can classify the face region as a first region and classify the region other than the face region as a second region.

[0091] Through this, as shown in FIG. 6, the glass image (GI) is displayed in an area other than the first area, which is the face area, so that the glass image (GI) is not displayed in the entire face area, including the oral area which is the patient's treatment or scanning target, and thus does not interfere with the user's scanning operation or treatment.

[0092] Here, the first detection model (132b) and the second detection model (132c) can be implemented as learning models trained based on artificial intelligence. For example, the first detection model (132b) can be generated by being trained to detect perforation areas (210) within pore areas using an image of a patient's face covered with various types of pores (200) as training data.

[0093] And, the second detection model (132c) can be provided as an artificial intelligence model capable of detecting a human face region within an image. Here, the model for detecting a human face region within an image may be applied in various forms, for example, a method for detecting the boundary region of a human face, a method for detecting eyes, nose, mouth, etc. and detecting the face region based thereon, etc.

[0094] In the above-described embodiment, the scanning management program (132) detects the presence of a small pore area based on a specific color when detecting a small pore area.

[0095] As another example, in an embodiment of the present invention, as shown in FIG. 9, at least one marker (M) is attached to the small pore (200). Then, the scanning management program (132) of the scanning management terminal (130) may be configured to determine whether the small pore area is detected based on whether the marker (M) is recognized within the camera image.

[0096] In the embodiment illustrated in FIG. 9, three markers (M) are attached to the small pouch (200) as an example, but it goes without saying that the technical concept of the present invention is not limited to such a number.

[0097] With the above configuration, the scanning management program (132) recognizes that the small pore (200) is covered when the marker (M) is recognized within the camera image and operates in a first mode, and when the marker (M) is not recognized, recognizes that the small pore (200) is not covered and operates in a first mode.

[0098] Additionally, in an embodiment of the present invention, a scanning management program (132) can extract a perforation area (210) based on the location of a marker (M) recognized within a camera image. As shown in FIG. 9, when three or more markers (M) are attached around the perforation area (210) of a small hole (200), the scanning management program (132) can extract the perforation area (210) based on the location of the recognized marker (M).

[0099] In one embodiment, when three markers (M) are attached to the small hole (200), as shown in FIG. 10 (a), an area (A2) of a certain size inside the triangle formed by connecting the three markers (M) can be recognized as a perforation area (210).

[0100] As another example, when three markers (M) are attached to the small hole (200), as shown in Fig. 10 (b), the triangular area (A2) formed by connecting the three markers (M) can be recognized as the perforation area (210).

[0101] Here, the method of recognizing the perforation area (210) based on a plurality of markers (M) can be implemented in various ways in addition to the embodiment described above, and it goes without saying that the technical concept of the present invention is not limited thereto.

[0102] Meanwhile, the glass image generation unit (132d) can generate a 2D oral image of a 3D image model viewed from a specific view direction as a glass image (GI), as shown in FIGS. 6 and 7, and transmit it to the smart glass (150) through the scanning interface unit (131).

[0103] And, the glass processor (155) can display a 2D oral image received through the glass interface unit (154) as a glass image (GI) on the transparent display unit (152). At this time, the scanning management program (132) can control the smart glass (150) so that the 2D oral image to be displayed as a glass image (GI) is displayed in the second area.

[0104] The above configuration is reflected in real time during treatment or oral scanning using smart glasses (150). That is, a camera image captured in real time by the camera module (153) of the smart glasses (150) is transmitted to a scanning management terminal (130), and the scanning management program (132) distinguishes a first area and a second area from the camera image transmitted in real time, so that the position of the second area where the glass image (GI) is to be displayed changes in real time in accordance with the movement of the user wearing the smart glasses (150).

[0105] As illustrated in FIGS. 7 and 8, when the user's gaze area (VA) changes according to the movement of the user wearing the smart glasses (150), a second area is distinguished in real time within the changing gaze area (VA), so that the position where the glass image (GI) is displayed on the transparent display unit (152) can be changed.

[0106] Meanwhile, the scanning management program (132) of the scanning management terminal (130) according to an embodiment of the present invention can extract a scanner area corresponding to the oral scanner (110) from a camera image.

[0107] And, the scanning management program (132) can control the oral scanner (110) so that when the scanner area is detected moving away from the first area while the oral scanner (110) is in operation, the operation of the oral scanner (110) is stopped.

[0108] Referring to FIG. 11, when the oral scanner (110) is in operation, the scanning management program (132) can extract a scanner area corresponding to the oral scanner (110) from a camera image captured and transmitted by the camera module (153). At this time, when oral scanning is in progress, the scanner area for the oral scanner (110) is located within a first area containing the oral region, as shown in FIG. 11 (a) within the camera image.

[0109] Here, the scanning management program (132) distinguishes a first region and a second region within the camera image and extracts a scanner region from the camera image, thereby recognizing that the scanner region is located within the first region.

[0110] As described above, during the scanning process, if the user completes the oral scanning process or removes the oral scanner (110) from the patient's oral cavity for other reasons, the scanner area in the camera image moves away from the first area, as shown in Fig. 11 (b).

[0111] At this time, the scanning management program (132) controls the oral scanner (110) so that the operation of the oral scanner is stopped when it is detected that the scanner area is moving away from the first area, thereby allowing the operation of the oral scanner (110) to be stopped without any separate operation by simply removing the oral scanner (110) from the patient's oral cavity.

[0112] In one embodiment, the scanning management program (132) of the scanning management terminal (130) can control the oral scanner (110) to operate when it is detected that the scanner area is being brought into the first area.

[0113] As shown in Fig. 11(b), if the oral scanner (110) is detected to be introduced into the first area while not located in the first area through analysis of the camera image, the oral scanner (110) can be controlled to operate.

[0114] Meanwhile, the glass image generation unit (132d) according to an embodiment of the present invention can generate an input menu for control input of an oral scanner (110) or a scanning management terminal (130) as a glass image (GI) to be displayed on the smart glass (150).

[0115] FIG. 12 is a diagram showing an example in which an input menu is displayed on a smart glass (150) according to an embodiment of the present invention. As shown in FIG. 9, the input menu is displayed in a second area as described above.

[0116] In one embodiment, the selection of an input menu can be made using the glass input portion (156) of the smart glasses (150). The glass input portion (156) can be applied in a touch manner to the outside of the glass frame (151) of the smart glasses (150).

[0117] As another example, the input menu can be selected through gesture recognition by a user wearing smart glasses (150). More specifically, the scanning management terminal (130) may be configured to include a gesture recognition unit (132e).

[0118] The gesture recognition unit (132e) receives a camera image captured by the camera module (153) while displaying an input menu as a glass image (GI), and can extract a hand area from the received camera image.

[0119] In one embodiment, a user wearing smart glasses (150) can execute a pre-set gesture in front of the smart glasses (150) using their hand while an input menu is displayed on the transparent display portion (152) of the smart glasses (150) they are wearing. For example, they can perform gestures such as moving their hand up and down as in (a) of FIG. 13, or moving their hand left and right as shown in (b) of FIG. 13.

[0120] Here, a gesture of moving the hand up and down can be recognized as an operation signal to change the input menu to be selected among multiple input menus, and a gesture of moving left and right can be recognized as an operation signal for selecting the corresponding input menu.

[0121] The gesture illustrated in FIG. 13 is an example, and various inputs and selections may be possible through the setting of various gestures.

[0122] Through the above configuration, the user input section (135) of the scanning management terminal (130) or the glass input section (156) of the smart glasses (150) can be operated and selected via gestures without operation, thereby providing a user environment that ensures user convenience.

[0123] Although some embodiments of the present invention have been illustrated and described, those skilled in the art will understand that modifications can be made to these embodiments without departing from the principles or spirit of the invention. The scope of the invention will be defined by the appended claims and their equivalents.

[0124] [Explanation of the symbol]

[0125] 100: Oral scanning system 110: Oral scanner

[0126] 130: Scanning management terminal 131: Scanning interface section

[0127] 132: Scanning Management Program 132a: 3D Model Creation Unit

[0128] 132b: 1st detection model 132c: 2nd detection model

[0129] 132d: Glass image generation unit 132e: Gesture recognition unit

[0130] 133: Scanning terminal processor 134: Scanning terminal monitor

[0131] 135: User input section 136: Data storage section

[0132] 150 : Smart Glasses 151 : Glass Frame

[0133] 151a: Lens 152: Transparent display part

[0134] 153: Camera module 154: Glass interface part

[0135] 155: Glass processor 156: Glass input section

[0136] 200 : Small hole 210 : Perforation area

[0137] The present invention can be applied to the field of oral scanning used in dental clinics, etc.

Claims

1. In an oral scanning system, An oral scanner that scans a patient's oral cavity to generate multiple scanned images, and Smart glasses having a camera module, and It includes a scanning management terminal linked with the above oral scanner and the above smart glasses; The above scanning management terminal A plurality of the scanning images are received from the oral scanner to generate a three-dimensional image model of the patient's oral cavity, and A camera image captured by the camera module is divided into a first region and a second region, wherein the first region is divided such that it includes the patient's oral region within the camera image. Extract a scanner area corresponding to the oral scanner from the above camera image, and An oral scanning system characterized by controlling the oral scanner to stop operation of the oral scanner when it is detected that the scanner area deviates from the first area while the oral scanner is in operation.

2. In Paragraph 1, The smart glasses described above include a transparent display unit on which an image is displayed; The above scanning management terminal Generating a glass image to be displayed on the transparent display part of the smart glass, and An oral scanning system characterized by controlling the smart glasses so that the glass image is displayed in the second area on the transparent display unit.

3. In Paragraph 2, The above scanning management terminal operates in either a first mode or a second mode to divide the camera image into the first area and the second area; An oral scanning system characterized in that the first mode operates with the patient's face covered with a small bleb, and the second mode operates without the patient's face covered with the small bleb.

4. In Paragraph 3, The above scanning management terminal Detecting whether a small gun region corresponding to the small gun is included within the camera image, An oral scanning system characterized by operating in the first mode when the above-mentioned pore area is detected and operating in the second mode when the above-mentioned pore area is not detected.

5. In Paragraph 4, At least one marker is attached to the above small pouch; An oral scanning system characterized by the above scanning management terminal determining that the small pore area has been detected when the marker is recognized within the camera image, and operating in the first mode.

6. In Paragraph 5, When the above scanning management terminal operates in the above first mode, Extracting a perforated area among the small hole regions corresponding to the small hole within the camera image above; An oral scanning system characterized by dividing the perforated area into the first area and dividing the area other than the perforated area into the second area.

7. In Paragraph 6, A plurality of the above markers are attached around the perforated area of ​​the above small pore; An oral scanning system characterized by the above-described scanning management terminal extracting the perforation area based on the positions of a plurality of markers recognized from the camera image.

8. In Paragraph 4, When the above scanning management terminal operates in the above second mode, Extracting a face region for the patient's face within the above camera image; An oral scanning system characterized by dividing the face area into the first area and dividing the area other than the face area into the second area.

9. In Paragraph 2, The glass image above A 2D oral image of the above 3D image model viewed from a specific viewing direction, An oral scanning system characterized by including at least one input menu for a control input of at least one of the oral scanner and the scanning management terminal.

10. In Paragraph 1, The above scanning management terminal An oral scanning system characterized by controlling the oral scanner to operate when it is detected that the above scanner area is being drawn into the above first area.

Images