Oral model generation method and computer program recorded on recording medium in order to execute same

Abstract

The present invention presents a method for generating an oral model. The method may comprise steps in which a computing device: receives a plurality of images of a subject from an intraoral scanner; generates a depth map on the ㅁbasis of depth information acquired through the disparity between the plurality of received images and depth information acquired through structured light included in each of the plurality of images; and generates the oral model for the subject through the generated depth map.

Description

Method for creating an oral model and a computer program recorded on a recording medium for executing the same

[0001] The present invention relates to digital healthcare. More specifically, it relates to a method for generating an oral cavity model by complementing a depth map acquired in a stereoscopic manner and a depth map acquired in a single manner, and a computer program recorded on a recording medium for executing the method.

[0002] Intraoral scanning is the process of reconstructing the three-dimensional appearance of the teeth and periodontium within the oral cavity and creating a digital 3D oral model. Traditionally, impression-taking was used to recreate the appearance of the oral cavity using impression material. However, advancements in computer-aided design (CAD) and computer-aided manufacturing (CAM) technologies have made it possible to recreate the oral cavity using digital methods.

[0003] The principles for implementing an oral scanner can be broadly categorized into triangulation technique, confocal laser scanner microscopy (CLSM), and active wavefront sampling and device.

[0004] In particular, the method using triangulation can create a depth map by recognizing the distance to an object using an image created based on the detected light when light emitted from a light source is reflected and detected from the surface of teeth and periodontal tissue in the oral cavity.

[0005] Meanwhile, when generating a depth map in stereo based on multiple images, accurate depth map generation is possible because the visual disparity of each image is utilized.

[0006] However, when generating a depth map in a stereo manner, the hardware configuration for acquiring multiple images can become complicated, and there was a problem that the scanning speed was relatively slow because the depth map could only be generated for the area common to each image.

[0007] On the other hand, generating a depth map in a single method based on a single image has the advantage of simpler hardware configuration compared to the stereo method.

[0008] However, when generating a depth map in a single manner, since the shape of the structured light must be recognized, a calibration process between the projector that irradiates the structured light and the image sensor is essential, and not only does this require complex calculations for generating the depth map, but there is also a problem that the accuracy of the generated depth map is low.

[0009] One object of the present invention is to provide a method for generating an oral cavity model by complementing a depth map acquired in a stereo manner and a depth map acquired in a single manner.

[0010] Another object of the present invention is to provide a computer program recorded on a recording medium for executing a method for generating an oral cavity model by complementing a depth map acquired in a stereo manner and a depth map acquired in a single manner.

[0011] The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

[0012] In order to achieve the technical task described above, the present invention proposes a method for generating an oral cavity model. The method may include a step in which a computing device receives a plurality of images of a subject from an oral cavity scanner, a step in which the computing device generates a depth map based on depth information acquired through a visual disparity between the plurality of received images and depth information acquired through structured light included in each of the plurality of images, and a step in which the computing device generates an oral cavity model of the subject using the generated depth map.

[0013] The computer program may be combined with a computing device comprising a transceiver, a memory, and a processor for processing instructions stored in the memory. In addition, the computer program may be a computer program recorded on a recording medium so that the processor executes the steps of receiving a plurality of images of a subject from an oral scanner, generating a depth map based on depth information acquired through a visual disparity between the plurality of received images and depth information acquired through structured light included in each of the plurality of images, and generating an oral cavity model for the subject based on the generated depth map.

[0014] Specific details of other embodiments are included in the detailed description and drawings.

[0015] According to embodiments of the present invention, by complementing a depth map acquired in a stereo manner and a depth map acquired in a single manner, the advantages of each depth map can be utilized in various ways depending on the purpose.

[0016] The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description of the claims.

[0017] FIG. 1 is a schematic diagram showing an oral scanning system according to one embodiment of the present invention.

[0018] FIG. 2 is a perspective view of an oral scanner according to one embodiment of the present invention.

[0019] FIG. 3 is a bottom perspective view of an oral scanner according to one embodiment of the present invention.

[0020] FIG. 4 is a perspective view showing the internal configuration of an oral scanner according to one embodiment of the present invention.

[0021] FIG. 5 is a side view showing the internal configuration of an oral scanner according to one embodiment of the present invention.

[0022] FIG. 6 is a top view showing the internal configuration of an oral scanner according to one embodiment of the present invention.

[0023] FIG. 7 is an exemplary diagram showing a stereo image acquired by an oral scanner according to one embodiment of the present invention.

[0024] Figure 8 is an example diagram for explaining an image area detected according to an image acquisition method.

[0025] Figure 9 is a logical configuration diagram of a computing device according to one embodiment of the present invention.

[0026] FIGS. 10 to 13 are exemplary diagrams illustrating each depth map generated according to one embodiment of the present invention.

[0027] Figure 14 is a flowchart illustrating a method for creating an oral model according to one embodiment of the present invention.

[0028] Figure 15 is a flowchart illustrating a method for creating an oral model according to another embodiment of the present invention.

[0029] It should be noted that the technical terms used in this specification are used merely to describe specific embodiments and are not intended to limit the present invention. Furthermore, unless specifically defined otherwise herein, the technical terms used in this specification should be interpreted as having a meaning generally understood by those skilled in the art to which the present invention pertains, and should not be interpreted in an excessively broad or narrow sense. Furthermore, if a technical term used in this specification is an incorrect technical term that does not accurately express the spirit of the present invention, it should be replaced with a technical term that can be correctly understood by those skilled in the art. Furthermore, general terms used in the present invention should be interpreted according to their dictionary definitions or according to the context, and should not be interpreted in an excessively narrow sense.

[0030] Additionally, the singular expressions used herein include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "consist of" or "have" should not be construed to necessarily include all of the components or steps described in the specification, and should be construed to mean that some of the components or steps may not be included, or that additional components or steps may be included.

[0031] Additionally, terms including ordinal numbers, such as "first," "second," etc., used herein may be used to describe various components, but these components should not be limited by these terms. These terms are used solely to distinguish one component from another. For example, without departing from the scope of the present invention, a first component could be referred to as a "second component," and similarly, a second component could also be referred to as a "first component."

[0032] When a component is referred to as being "connected" or "connected" to another component, it may be directly connected or connected to that other component, but there may also be other components intervening. Conversely, when a component is referred to as being "directly connected" or "connected" to another component, it should be understood that there are no other components intervening.

[0033] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the attached drawings. Regardless of the drawing numbers, identical or similar components will be given the same reference numbers, and redundant descriptions thereof will be omitted. In addition, when describing the present invention, if a detailed description of a related known technology is judged to obscure the gist of the present invention, the detailed description thereof will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and the spirit of the present invention should not be construed as being limited by the attached drawings. The spirit of the present invention should be construed to extend to all modifications, equivalents, and substitutes other than the attached drawings.

[0034]

[0035] FIG. 1 is a schematic diagram showing an oral scanning system according to one embodiment of the present invention.

[0036] As illustrated in FIG. 1, the oral scanning system (300) may include an oral scanner (100) capable of scanning a three-dimensional structure inside the oral cavity of a dental patient and a computing device (200) connected to the oral scanner (100).

[0037] For example, an oral scanner (100) can be inserted into the oral cavity of a dental patient by a dental professional to non-contactly scan the teeth and capture multiple two-dimensional image data. In addition, the oral scanner (100) can transmit the captured multiple two-dimensional image data to a computing device (200) or independently execute three-dimensional oral structure modeling based on the two-dimensional image data.

[0038] The oral scanner (100) may be directly connected to a computing device (200) using a USB cable or the like, or may be connected via a network configured to enable wired or wireless communication. For example, depending on the installation environment, the network may be configured as a wired network such as an electrical connection line such as a copper cable, Ethernet, a wired home network (Power Line Communication), a telephone line communication device, and RS-serial communication, a wireless network such as a mobile communication network, WLAN (Wireless LAN), Wi-Fi, Bluetooth, and ZigBee, or a combination thereof.

[0039] The oral scanner (100) can transmit and receive information and / or data, such as two-dimensional image data and three-dimensional oral structure model data, with the computing device (200). The oral scanner (100) and the computing device (200) may be configured to be physically separate as illustrated, but are not limited thereto. For example, the oral scanner (100) and the computing device (200) may be configured as an integrated unit.

[0040] In the following configuration, the computing device (200) can perform three-dimensional oral structure modeling based on at least two two-dimensional image data or stereo images acquired from the oral scanner (100). To perform this function, the computing device (200) may correspond to a computing device including a processor (e.g., CPU, GPU, AP, NPU, etc.) capable of performing image processing and three-dimensional modeling and a memory capable of storing two-dimensional image data or three-dimensional oral structure model data. In one embodiment, the computing device (200) may be configured to transmit and receive information and / or data with the oral scanner (100). Specifically, the computing device (200) can transmit a command signal to the oral scanner (100) and receive image information of a target oral structure from the oral scanner (100).

[0041] Meanwhile, in the description below, the computing device (200) is described as controlling the overall operation of the oral scanner (100), but this is not limited to the case, and the oral scanner (100) may be configured to perform some or all of the operations.

[0042]

[0043] Hereinafter, an oral scanner according to one embodiment of the present invention will be described in detail.

[0044] FIG. 2 is a perspective view of an oral scanner according to an embodiment of the present invention, and FIG. 3 is a bottom perspective view of an oral scanner according to an embodiment of the present invention.

[0045] Referring to FIGS. 2 and 3, the oral scanner (100) may be configured to be gripped by a user, and may be configured to be inserted at least partially into the oral cavity while gripped by the user.

[0046] Hereinafter, each configuration will be described. The housing (10) forms the exterior of the oral scanner (100), and may be configured to accommodate a light source unit (20), a first optical system (30), a second optical system (40), a third optical system (50), and an image sensor unit (60) which will be described later therein. The housing (10) may be formed in the shape of a long rod formed in one direction as a longitudinal direction, and may be formed in any shape suitable for insertion into the oral cavity. In addition, the cross-section of the oral scanner (100) is illustrated as being approximately rectangular, but is not limited thereto, and the cross-section may be configured to include at least one of a circular, streamlined, and polygonal shape. In addition, when the cross-section of the housing (10) is formed in a polygonal shape, it may be configured to be rounded for a user's grip.

[0047] The housing (10) may include a first housing body (10a) and a second housing body (10b). A light source unit (20), a first optical system (30), a third optical system (50), and an image sensor unit (60) may be arranged inside the first housing body (10a), and a second optical system (40) may be arranged inside the second housing body (10b). The second housing body (10b) may be provided to be detachable from the first housing body (10a). When using an oral scanner (100), at least a portion of the oral scanner (100) may be inserted into the oral cavity of a patient.

[0048] With this configuration, after using the oral scanner (100), the second housing body (10b) located in front of the oral scanner (100) can be separated from the first housing body (10a) and disinfected or cleaned separately.

[0049] The opening forming portion (12) can be formed adjacent to one end of the housing (10). Specifically, the opening forming portion (12) can be positioned adjacent to one end of the housing (10) to form an opening (12a) connecting the inside and the outside of the housing (10) on its inner side.

[0050] The opening forming portion (12) may be configured so that light generated or reflected inside the housing (10) through the opening (12a) is irradiated to the outside, and external light can be introduced into the inside of the housing (10). In one embodiment, when the oral scanner (100) is inserted into the oral cavity, the opening (12a) may be configured to face the teeth that require scanning.

[0051]

[0052] Hereinafter, the internal configuration of an oral scanner according to one embodiment of the present invention will be described in detail.

[0053] FIG. 4 is a perspective view showing the internal configuration of an oral scanner according to an embodiment of the present invention, FIG. 5 is a side view showing the internal configuration of an oral scanner according to an embodiment of the present invention, FIG. 6 is a top view showing the internal configuration of an oral scanner according to an embodiment of the present invention, and FIG. 7 is an exemplary diagram showing a stereo image acquired by an oral scanner according to an embodiment of the present invention.

[0054] Referring to FIGS. 4 to 7, the oral scanner (100) may include a light source unit (20), a first optical system (30), a second optical system (40), a third optical system (50), and an image sensor unit (60).

[0055] The light source unit (20) may be configured to generate light. The light generated by the light source unit (20) may be configured to pass through optical systems. In this case, the light irradiated from the light source unit (20) may correspond to patterned light or structured light. The light pattern may be a fringe pattern, a linear pattern, a dot pattern, or a pattern of any shape. For example, the light source unit (20) may irradiate a pattern that changes according to a preset time. When the patterned light is irradiated on a subject (S), such as teeth in the oral cavity, deformation of the corresponding pattern may occur according to the three-dimensional structure of the surface of the subject (S). Therefore, the three-dimensional structure of the subject (S) can be identified and modeled based on information about deformation of the pattern projected on the surface of the subject (S) or changes in the position of feature points.

[0056] The light source unit (20) may be arranged inside the housing (10). Specifically, the light source unit (20) may be configured to irradiate light to the first optical system (30) described later. The opening (12a) or the second optical system (40) may be arranged adjacent to one end of the housing (10), and the light source unit (20) may be arranged adjacent to the other end of the housing (10). In addition, the light source unit (20) may be arranged at any intermediate point between one end and the other end of the housing (10). The arrangement of the light source unit (20) is not limited, and it is satisfied if the light generated from the light source unit (20) is irradiated toward the first optical system (30) or the second optical system (40), preferably the first optical system (30).

[0057] The first optical system (30) may be configured to reflect light irradiated from the light source unit (20) toward the second optical system (40). The first optical system (30) may reflect light irradiated along the irradiation axis (LS) so that the light moves along a first optical path (L1) spaced apart in the width direction (X2) or the width direction (X2) and the height direction (X3). The width direction (X2) may refer to a direction in the left-right width that is perpendicular to the longitudinal direction (X1) of the oral scanner (100). The height direction (X3) may refer to a direction in the vertical height that is perpendicular to the longitudinal direction (X1) of the oral scanner (100). The width direction (X2) may be defined as a left-right direction, and the height direction (X3) may be defined as an up-down direction. For convenience of explanation, the longitudinal direction (X1) may be defined as the first direction, the width direction (X2) as the second direction, and the height direction (X3) as the third direction.

[0058] The first optical system (30) may include a first reflector (32) and a second reflector (34). The first reflector (32) may be configured to reflect light irradiated from the light source (20) toward the second reflector (34). The first reflector (32) may be arranged on the inner upper portion of the housing (10). In addition, the second reflector (34) may be configured to reflect light reflected by the first reflector (32) toward the second optical system (40). The second reflector (34) may be arranged to be spaced apart from the first reflector (32).

[0059] Light irradiated from the light source unit (20) can be reflected toward the second optical system (40) via the first reflector unit (32) and the second reflector unit (34) of the first optical system (30). The first reflector unit (32) of the first optical system (30) can form an irradiation axis (LS) with the light source unit (20), and the second reflector unit (34) of the first optical system (30) can form a first optical path (L1) with the second optical system (40). The optical path refers to the area through which light passes, the central axis of the path along which light travels in space, and can also be defined as an optical axis.

[0060] The light source unit (20) is arranged to be spaced apart from the first optical path (L1), and the first and second reflectors (32, 34) can be configured to be inclined so that the light emitted from the light source unit (20) can travel to the second optical system (40) along the established first optical path (L1). The inclined arrangement of the first and second reflectors (32, 34) will be described later.

[0061] The second optical system (40) reflects the light irradiated from the second reflector (34) of the first optical system (30) toward the subject (S), and the light reflected from the subject (S) can be reflected by the second optical system (40) and transmitted toward the third optical system (50).

[0062] The second optical system (40) may include at least one reflector. This may be defined as a third reflector (42). For example, the second optical system (40) may be at least one mirror. In one embodiment, the second optical system (40) may be positioned at or around the opening (12a). For example, the second optical system (40) may be fixedly positioned on the inner surface of the housing (10) adjacent to the opening (12a).

[0063] The third optical system (50) may be configured to reflect light reflected from the outside by the second optical system (40) toward the image sensor unit (60). Specifically, the third optical system (50) may reflect light reflected by the second optical system (40) from the subject (S) toward the image sensor unit (60). The third optical system (50) may include one or more reflectors or mirrors for reflecting the light.

[0064] The first to third optical systems (30, 40, 50) can be arranged to be aligned along a reference line (AL) passing through the first optical path (L1). The reference line may also be defined as a reference axis. The first to third optical systems (30, 40, 50) can minimize the travel distance of light by being aligned along the reference line (AL). This minimizes light scattered while passing through the optical systems, thereby minimizing aberrations and obtaining clear and high contrast up to the periphery of the object, thereby improving optical performance.

[0065] In addition, the first to third optical systems (30, 40, 50) may be arranged on a virtual plane through which a reference line (AL) passes. The reference line (AL) may refer to an extension line extending from the first optical path (L1). An image sensor unit (60) may also be arranged on the plane. The virtual plane is arranged so that the reference line (AL) passes through, and the plane may refer to a plane extending in the height direction from the reference line (AL). In the present embodiment, the plane may be configured to pass through the center of the oral scanner (100).

[0066] The third optical system (50) may be arranged adjacent to the second reflector (34) of the first optical system (30). Specifically, the third optical system (50) may be arranged adjacent to the other side of the reflective surface on which light is reflected by the second reflector (34) of the first optical system (30).

[0067] The third optical system (50) may include a fourth reflector (52) and a fifth reflector (54). A pair of fourth reflectors (52) may be provided, and the pair of fourth reflectors (52) may reflect light reflected from the second optical system (40) toward the fifth reflector (54). The second optical system (40) may form a pair of second optical paths (L2) with the third optical system (50). Specifically, the pair of second optical paths (L2) may be formed by the pair of fourth reflectors (52) of the second optical system (40) and the third optical system (50). Since the pair of fourth reflectors (52) are configured to be spaced apart from each other, the pair of second optical paths (L2) may be configured to be spread apart from each other in the width direction (X2). A pair of second optical paths (L2) may be formed identically in the height direction, but is not limited thereto. The center lines of a pair of second optical paths (L2) may also be arranged parallel to the first optical path (L1).

[0068] The second reflector (34) of the first optical system (30) may be arranged between a pair of fourth reflectors (52). Specifically, the second optical system (40) and the pair of fourth reflectors (52) may form a square area (BA), and the second reflector (34) of the first optical system (30) may be arranged in the square area (BA). Through this, the second reflector (34) of the first optical system (30) may be arranged so as not to interfere with the light reflected from the second optical system (40) toward the pair of fourth reflectors (52) of the third optical system (50). Through this configuration, the first optical path (L1) and the pair of second optical paths (L2) may not interfere with each other, and the components inside the housing (10) may be arranged more densely.

[0069] A pair of fourth reflectors (52) can form a gap therebetween. By forming the gap, the pair of fourth reflectors (52) can allow light reflected by the fourth reflectors (52) and irradiated from the fifth reflector (54) to pass through the gap and reach the image sensor unit (60). Here, the position and direction of each of the pair of fourth reflectors (52) can be set so that the two images of the subject (S) reflected by the two reflective surfaces of the fifth reflector (54) and detected by the image sensor unit (60) do not overlap and each image is fully visible.

[0070] The fifth reflector (54) may be provided in pairs, and the pair of fifth reflectors (54) may be configured to reflect the light reflected from the pair of fourth reflectors (52) toward the image sensor unit (60). Specifically, the pair of fifth reflectors (54) may each reflect the light reflected from the pair of fourth reflectors (52) toward the image sensor unit (60). The light reflected from the pair of fifth reflectors (54) may pass through the gap formed between the pair of fourth reflectors (52) and reach the image sensor unit (60). Here, the position and direction of each of the pair of fifth reflectors (54) may be set so that the two images of the subject (S) reflected by the fifth reflectors (54) and detected by the image sensor unit (60) do not overlap, and each image is fully visible.

[0071] A pair of fifth reflectors (54) may be connected to each other at each of their respective corners. For example, a pair of fifth reflectors (54) may be configured in the form of triangular prisms that are arranged adjacently as shown in the drawings.

[0072] In one embodiment, each of the first optical system (30), the second optical system (40), and the third optical system (50) may be fixedly positioned at a predetermined position inside the housing (10). In this case, a driving unit for adjusting the angle of the first optical system (30), the second optical system (40), or the third optical system (50) may not be installed inside the housing (10). In this way, since there is no need to place other electronic or mechanical components in the areas where the first optical system (30), the second optical system (40), and the third optical system (50) are placed inside the housing (10), the components inside the housing (10) can be densely arranged. Accordingly, since the optimal structure of the housing (10) can be designed from the dense structures of the first optical system (30), the second optical system (40), and the third optical system (50), an oral scanner (100) that can freely perform scanning operations inside the oral cavity and has a small volume can be implemented.

[0073] In one embodiment, the angle formed by a pair of fourth reflectors (52) may correspond to a hot angle, i.e., an angle less than 180 degrees, and the angle formed by a pair of fifth reflectors (54) may correspond to a right angle, i.e., an angle greater than 180 degrees.

[0074] The image sensor unit (60) may be configured to detect light reflected from the third optical system (50). Specifically, the image sensor unit (60) may be configured to detect light reflected from the third optical system (50) and passing through the lens assembly (90) and the sensor reflector (94). In one embodiment, the image sensor unit (60) may be configured to acquire two stereo images from the light reflected from the third optical system (50). Specifically, the image sensor unit (60) may acquire two images of light each reflected by the fifth reflectors (54) of the third optical system (50). In this way, the oral scanner (100) may acquire two stereo images with only one image sensor unit (60) through the third optical system (50). The two stereo images acquired from the image sensor unit (60) may be used for 3D oral structure modeling executed by a subsequent processor.

[0075] In this embodiment, the fourth and fifth reflectors (52, 54) are described as each having one reflective surface, but the fourth and fifth reflectors (52, 54) may include two or more reflective surfaces (n). In this case, the stereo image acquired from the image sensor unit (60) may acquire 2n stereo images.

[0076] The image sensor unit (60) may be placed inside the housing (10). Specifically, the image sensor unit (60) may be placed adjacent to the other end of the housing (10). The image sensor unit (60) may be configured so that light reflected from the fifth reflector (54) of the third optical system (50) is incident thereon.

[0077] Through these configurations, light irradiated from the light source unit (20) can be reflected toward the second optical system (40) by the first reflector (32) and the second reflector (34) of the first optical system (30), and the light reflected from the second reflector (34) can be reflected toward the subject (S) located outside the housing (10) through the second optical system (40) and the opening (12a). In addition, the light reflected from the subject (S) can be reflected toward the fourth reflector (52) of the third optical system (50) by the second optical system (40). The light reflected by the fourth reflector (52) can be reflected toward the image sensor unit (60) by the fifth reflector (54).

[0078] Referring to FIG. 7, the image sensor unit (60) may be configured to detect light reflected from the third optical system (50). The image sensor unit (60) may be configured to acquire two stereo images (Ia, Ib) from the light reflected from the third optical system (50). Specifically, the image sensor unit (60) may acquire two stereo images (Ia, Ib) each reflected by a pair of fifth reflectors (54) of the third optical system (50). Based on the two stereo images (Ia, Ib) acquired in this manner, the processor may extract depth data and, based on the depth data, execute 3D modeling of the oral structure, which is the subject (S).

[0079]

[0080] Figure 8 is an example diagram for explaining an image area detected according to an image acquisition method.

[0081] Oral scanners can acquire images of teeth and periodontal tissue through their image sensor. The teeth captured in these images vary in shape and appearance depending on the individual. These images may contain various features, such as cavities, external trauma, or grooves created by procedures like root canal treatment and implants.

[0082] Meanwhile, a conventional oral scanner, as illustrated in (a), irradiates structured light onto a subject through a light source, detects light reflected from the subject through a single image sensor unit (A), and can generate a depth map for the subject based on the detected structured light. Here, a single image sensor unit (A) means that only one image is detected from a single image sensor.

[0083] Conventional oral scanners such as this one may have problems in accurately detecting depth in areas of the nasal cavity, such as the inner side of a groove in a tooth, with only one image sensor unit (A), depending on the location and direction. Furthermore, when a depth map is generated using a single image sensor unit, there is a problem in that the accuracy of the generated depth map is low.

[0084] In another conventional method, as illustrated in (b), the oral scanner can obtain a stereo image by irradiating structured light to the subject through a light source and detecting the light reflected from the subject through two image sensor units (C, D). At this time, the oral scanner can generate a depth map for the teeth by using the visual disparity between the acquired stereo images. In this case, since the oral scanner can generate a depth map only for an area that exists in common in the stereo images, the acquired area is limited, and a problem may occur in which depth detection is impossible for a negative area such as the inner side of a groove existing in the teeth.

[0085] To solve these problems, an oral scanner according to one embodiment of the present invention proposes a method for quickly and accurately covering various shaded areas existing in teeth to generate an accurate depth map.

[0086]

[0087] Hereinafter, the logical configuration of a computing device according to one embodiment of the present invention will be described in detail.

[0088] FIG. 9 is a logical configuration diagram of a computing device according to an embodiment of the present invention, and FIGS. 10 to 13 are exemplary diagrams for explaining each depth map generated according to an embodiment of the present invention.

[0089] As illustrated in FIG. 9, a computing device (200) according to one embodiment of the present invention may be configured to include a communication unit (205), an input / output unit (210), a control unit (215), and a storage unit (220).

[0090] As such, the components of the computing device (200) merely represent functionally distinct elements, so two or more components may be implemented integrated with each other in an actual physical environment, or one component may be implemented separately from each other in an actual physical environment.

[0091] As for each component, the communication unit (205) can transmit and receive data with the oral scanner (200).

[0092] Specifically, the communication unit (205) can receive an image of a subject. Here, the image can be an image generated by irradiating light to teeth and periodontal tissues within the oral cavity and based on the light reflected from the subject. In addition, the communication unit (205) can transmit a guide signal to the oral scanner (100) according to the state of the oral scanner. Here, the guide signal can be a signal that instructs the output of one or more of light, sound, and vibration.

[0093] With the following configuration, the input / output unit (210) can input or output various types of data required or generated during the oral scanning process.

[0094] Specifically, the input / output unit (210) can receive a scan target range related to oral scanning. In this case, the scan target range is a range within which scanning must be performed in order to create a model of teeth and periodontal tissues within the oral cavity.

[0095] In addition, the input / output unit (210) can output in real time an image being scanned by the oral scanner (100). In this case, the image being scanned can be any one of the screens related to the maxilla, mandible, and occlusion of the patient, but is not limited thereto. In addition, the input / output unit (210) can output a composite image generated by oral scanning. In this case, the composite image is a single oral model of teeth and periodontal tissues in the oral cavity generated by synthesizing one or more scanned images.

[0096] In the following configuration, the control unit (215) can generate a depth map based on depth information obtained through visual deviation between multiple images received from the oral scanner (100) and depth information obtained through analysis of structured light included in each of the multiple images.

[0097] Specifically, as illustrated in FIG. 10, light reflected from the first area (a) of the subject may pass through one of the fourth reflectors (52a) and be reflected by the fifth reflector (54) and detected by the image sensor unit (60). Here, the control unit (215) may receive an image of the first area (a) generated through the light detected by the image sensor unit (60) from the oral scanner.

[0098] In addition, as illustrated in FIG. 11, light reflected from the second area (b) of the subject may be reflected by the fifth reflector (54) through one of the fourth reflectors (52b) and detected by the image sensor unit (60). Here, the control unit (215) may receive an image of the second area (b) generated through the light detected by the image sensor unit (60) from the oral scanner.

[0099] That is, the control unit (215) can obtain two stereo images for each of the first area (a) and the second area (b) obtained by the image sensor unit (60) through the fifth reflector (54) by reflecting from each of the pair of fourth reflectors (52a, 52b).

[0100] Although FIGS. 10 and 11 illustrate a method utilizing visual deviation according to the positional structure of a pair of mutually spaced fourth reflectors (52a, 52b), the present invention is not limited to this structure and includes various structures in which visual deviation may occur when acquiring a stereo image.

[0101] Meanwhile, in FIGS. 10 to 14, the image sensor unit (60) is illustrated as a single configuration, but as described above, it is possible to obtain a stereo image through a single image sensor, and it is also possible to obtain a stereo image by providing a separate image sensor for each area.

[0102] As illustrated in FIG. 12, the control unit (215) can generate a depth map for a third region (c) overlapping the first region (a) and the second region (b) through stereo matching between the two stereo images acquired in this manner. Here, the generated depth map can include depth information for the third region (c) commonly included in the first region (a) and the second region (b). That is, the control unit (215) can match pixels for each image of the first region (a) and the second region (b) through stereo matching, and generate a depth map for the third region (c) including depth information based on triangulation.

[0103] Here, since the generated depth map only includes depth information for the third region (c) that is commonly included in the first region (a) and the second region (b), the generation of depth information is inevitably limited for regions that exist in each of the first region (a) and the second region (b) but do not overlap each other.

[0104] Accordingly, as illustrated in FIG. 13, the control unit (215) can supplement depth information for areas that exist in each of the first area (a) and the second area (b) but do not overlap each other by additionally obtaining depth information through the image for the first area (a) and the image for the second area (b), respectively.

[0105] To this end, the control unit (215) can derive depth information for each of the second depth map and the third depth map through triangulation of the pattern of structured light that changes according to the depth of the subject in the image for the first region (a) and the image for the second region (b), respectively. For example, the control unit (215) can obtain three-dimensional coordinates corresponding to the depth information of the subject by calculating the intersection of three-dimensional straight lines passing through the center (center of projection) of the image sensor unit (60) and specific points captured in the image for the first region (a) and the image for the second region (b), respectively.

[0106] Thereafter, the control unit (215) can supplement the depth information of the depth map acquired through stereo matching using the additionally acquired depth information. In the following description, the depth map for the third region (c) acquired through stereo matching will be described as the first depth map, and the depth maps acquired from the image for the first region (a) and the image for the second region (b) will be described as the second depth map and the third depth map, respectively.

[0107] In one embodiment, the control unit (215) can identify at least one pixel in the first depth map that does not include depth information, and extract depth information of a pixel corresponding to the identified at least one pixel from at least one of the second depth map and the third depth map to supplement the first depth map. That is, the control unit (215) can supplement pixels of the first depth map that do not include depth information because they were not simultaneously acquired from the first image and the second image for shaded areas, such as between adjacent teeth and between teeth and periodontal tissue, based on the depth information for the second depth map and the third depth map.

[0108] At this time, the control unit (215) can calculate an average value of the depth information extracted from the second depth map and the third depth map, respectively, and generate depth information for at least one identified pixel based on the calculated average value. For example, if there are corresponding pixels in both the second depth map and the third depth map for a pixel in the first depth map that does not include depth information, the control unit (215) can reflect the depth information of the corresponding pixel in the first depth map based on the average value of the depth information for the corresponding pixel in the second depth map and the third depth map.

[0109] In another embodiment, the control unit (215) may extract a depth map for an area that does not overlap with a third area among the first and second areas from at least one of the second depth map and the third depth map and match the depth map to the first depth map. At this time, the second depth map and the third depth map, whose depth information is acquired through pattern recognition of structured light, have a problem in that their accuracy is relatively low compared to the first depth map acquired in a stereo manner. Accordingly, the control unit (215) may correct the depth information for the matched depth map based on the depth information of the first depth map. Here, the control unit (215) may calculate depth values ​​for the same pixel between the first depth map and the second depth map or the third depth map, and correct the depth information for the matched depth map based on an average value of the calculated depth values.

[0110] For example, the control unit (215) can extract a depth map for an area not included in the first depth map from the second depth map, and align the extracted depth map to the first depth map. At this time, the control unit (215) can calculate depth deviations for the same pixel between the first depth map and the second depth map. Here, the average value of the calculated depth deviations can be a parameter for correcting the depth information of the depth map aligned to the first depth map. Accordingly, the control unit (215) can correct the depth information of the aligned depth map extracted from the second depth map or the third depth map based on the depth information of the first depth map, which has relatively high accuracy, by increasing or decreasing the depth information of the aligned depth map by the average value of the calculated depth deviations.

[0111] In another embodiment, the control unit (215) can evaluate the reliability of the oral model based on the deviation of the generated depth values. That is, the control unit (215) can evaluate the reliability of the oral model to be generated based on the consistency of the depth information of the first depth map, the second depth map, and the third depth map. For example, the control unit (215) can set a grade based on the average value of the depth deviation, and generate guide information such as a calibration request, an optical system inspection request, etc. based on the set grade and output the generated guide information through the oral scanner, thereby providing the user with customized guide information based on the reliability of the oral model.

[0112] And, the control unit (215) can create an oral cavity model for the subject through the generated depth map. Specifically, when the oral scanner moves and continuously scanned images are continuously received and a depth map is generated as described above, the control unit (215) can reconstruct the generated depth maps to create a single 3D oral cavity model. For example, the control unit (215) can extract feature points for each depth map and align the depth maps by matching corresponding feature points among the extracted feature points in 3D.

[0113] In addition, the control unit (215) can reconstruct the aligned images to create a single three-dimensional oral model. At this time, if there is a difference in illumination between the images, the control unit (215) can additionally perform brightness correction work, etc. before reconstructing.

[0114]

[0115] Hereinafter, the operation of the computing device described above will be described in more detail.

[0116] Figure 14 is a flowchart for explaining a method for creating an oral model according to one embodiment of the present invention.

[0117] As illustrated in FIG. 14, in step S110, the computing device can receive a first image capturing a first area of ​​the subject from the oral scanner and a second image capturing a second area at least partially overlapping the first area.

[0118] Next, in step S120, the computing device can match the received first and second images to generate a first depth map for the first region and a third region overlapping the second region. At this time, the computing device can match pixels between the first and second images through stereo matching and generate a first depth map for the third region containing depth information based on triangulation.

[0119] Additionally, the computing device may generate a second depth map for the first area through the first image to supplement the first depth map, and may generate a third depth map for the second area through the second image. Here, the computing device may derive depth information for each of the second depth map and the third depth map through triangulation of a pattern of structured light that changes according to the depth of the subject in each of the first image and the second image.

[0120] Next, at step S130, the computing device can complement the first depth map generated in a stereo manner and the second depth map and third depth map generated in a single manner.

[0121] In one embodiment, the computing device can identify at least one pixel in the first depth map that does not include depth information, and extract depth information of a pixel corresponding to the identified at least one pixel from at least one of the second depth map and the third depth map to supplement the first depth map.

[0122] That is, the computing device can supplement pixels of the first depth map that do not contain depth information because they were not simultaneously acquired from the first image and the second image for shaded areas such as between adjacent teeth, between teeth and periodontal tissue, etc., based on depth information for the second depth map and the third depth map.

[0123] At this time, the computing device can calculate an average value of depth information extracted from each of the second depth map and the third depth map, and generate depth information for at least one identified pixel based on the calculated average value.

[0124] In another embodiment, the computing device can extract a depth map for a region of the first region and the second region that does not overlap with the third region from at least one of the second depth map and the third depth map and match the depth map to the first depth map.

[0125] At this time, the computing device can correct the depth information for the matched depth map based on the depth information of the first depth map. Here, the computing device can calculate the depth deviation for the same pixel between the first depth map and the second depth map or the third depth map, and correct the depth information for the matched depth map based on the average value of the calculated depth deviations.

[0126] Next, when scanning is terminated at step S140, the computing device can generate an oral cavity model based on the generated depth map (S150).

[0127] Specifically, the computing device can generate a three-dimensional oral cavity model by reconstructing the generated depth maps when the oral scanner moves and continuously receives images scanned continuously and a depth map is generated as described above.

[0128] In summary, the method for generating an oral model according to one embodiment of the present invention generates a first depth map, a second depth map, and a third depth map in real time based on images received in real time during a scanning process of a computing device, and then complements them, thereby minimizing unscanned areas and increasing the scanning speed.

[0129]

[0130] Figure 15 is a flowchart for explaining a method for creating an oral model according to another embodiment of the present invention.

[0131] As illustrated in FIG. 15, in step S210, the computing device can receive a first image capturing a first area of ​​the subject from the oral scanner and a second image capturing a second area at least partially overlapping the first area.

[0132] Next, in step S120, the computing device can match the received first image and second image to generate a first depth map for the first region and a third region overlapping the second region.

[0133] Specifically, the computing device can match the received first and second images to generate a first depth map for the first region and a third region overlapping the second region. At this time, the computing device can match pixels between the first and second images through stereo matching and generate a first depth map for the third region containing depth information based on triangulation.

[0134] Next, in step S230, when scanning is terminated, the computing device can generate an oral model based on the generated first depth map (S240).

[0135] Specifically, the computing device can generate a three-dimensional oral cavity model by reconstructing the generated depth maps when the oral scanner moves and continuously receives images scanned continuously and a depth map is generated as described above.

[0136] And, at step S250, the computing device can supplement the oral model generated based on the first depth map.

[0137] Specifically, the computing device may generate a second depth map for the first area through the first image to supplement the first depth map, and may generate a third depth map for the second area through the second image. Here, the computing device may derive depth information for each of the second depth map and the third depth map through triangulation of a pattern of structured light that changes according to the depth of the subject in each of the first image and the second image.

[0138] Thereafter, the computing device can identify at least one pixel in the first depth map that does not include depth information, extract depth information of a pixel corresponding to the identified at least one pixel from at least one of the second depth map and the third depth map, and supplement the generated oral model based on the extracted depth information.

[0139] In summary, a method for generating an oral model according to another embodiment of the present invention can generate a first depth map in stereo through an image received in real time during a scanning process by a computing device, and generate an oral model through the generated first depth map.

[0140] In addition, the computing device can generate an oral model with high precision by supplementing the depth information of the generated oral model with the second depth map and the third depth map acquired through a single method.

[0141]

[0142] As described above, although the present specification and drawings have disclosed preferred embodiments of the present invention, it will be apparent to those skilled in the art that other modifications based on the technical idea of ​​the present invention can be implemented in addition to the embodiments disclosed herein. In addition, although specific terms have been used in the present specification and drawings, these have been used only in a general sense to easily explain the technical contents of the present invention and to help the understanding of the invention, and are not intended to limit the scope of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all aspects, but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims

1. A step in which a computing device receives multiple images of a subject from an oral scanner; A step in which the computing device generates a depth map based on depth information obtained through visual disparity between the received plurality of images and depth information obtained through analysis of structured light included in each of the plurality of images; and A method for generating an oral cavity model, characterized in that the computing device comprises a step of generating an oral cavity model for the subject through the generated depth map.

2. In the first paragraph, the receiving step A method for generating an oral model, characterized by receiving a first image of a first area of ​​the subject and a second image of a second area of ​​the subject that at least partially overlaps the first area.

3. In the second paragraph, the generating step A method for generating an oral cavity model, characterized in that a first depth map is generated for a third region overlapping the first region and the second region by matching the first image and the second image.

4. In the third paragraph, the generating step A method for generating an oral cavity model, characterized in that a displacement image for the third area is generated through stereo matching between the first image and the second image, and the first depth map is generated through triangulation of the generated displacement image.

5. In the third paragraph, the generating step A method for generating an oral cavity model, characterized in that a second depth map for the first region is generated through the first image, and a third depth map for the second region is generated through the second image.

6. In the fifth paragraph, the generating step A method for generating an oral model, characterized in that depth information for each of the second depth map and the third depth map is derived through triangulation of a pattern of structured light that changes according to the depth of the subject in each of the first image and the second image.

7. In the third paragraph, the generating step A method for generating an oral model, characterized in that the method comprises identifying at least one pixel in the first depth map that does not include depth information, and extracting depth information of a pixel corresponding to the identified at least one pixel from at least one of the second depth map and the third depth map to supplement the first depth map.

8. In the 7th paragraph, the generating step A method for generating an oral model, characterized in that the method comprises calculating an average value of depth information extracted from the second depth map and the third depth map, and reflecting the average value as depth information for at least one identified pixel.

9. In the third paragraph, the generating step A method for generating an oral cavity model, characterized in that a depth map for an area among the first area and the second area that does not overlap with the third area is extracted from at least one of the second depth map and the third depth map and aligned with the first depth map.

10. In the 9th paragraph, the generating step A method for creating an oral model, characterized in that the depth information for the aligned depth map is corrected based on the depth information of the first depth map.

11. In the 10th paragraph, the generating step A method for generating an oral model, characterized in that depth values ​​for the same pixel between the first depth map and the second depth map or the third depth map are calculated, and depth information for the matched depth map is corrected based on an average value of the calculated depth values.

12. In the 11th paragraph, the generating step A method for creating an oral cavity model, characterized in that the reliability of the oral cavity model is evaluated based on the deviation of the depth values ​​calculated above.

13. Memory; transceiver; and Combined with a computing device comprising a processor for processing instructions residing in the above memory, The above processor receives a plurality of images of a subject from an oral scanner; The step of the processor generating a depth map based on depth information obtained through visual disparity between the plurality of received images and depth information obtained through structured light included in each of the plurality of images; and A computer program recorded on a recording medium, for causing the processor to execute a step of creating an oral model for the subject through the generated depth map.

Images