Method for generating three-dimensional oral cavity model, and electronic device for performing same

Abstract

This method by which an electronic device generates a three-dimensional oral cavity model comprises the steps of: acquiring a plurality of first depth frames on the basis of first scan data; generating, on the basis of the plurality of first depth frames, a three-dimensional model representing the inside of an oral cavity; acquiring a second depth frame on the basis of second scan data received after the first scan data is received; determining the similarity between the respective first depth frames and the second depth frame with respect to at least one characteristic; if the determined similarity satisfies a preset condition, processing the first depth frames and / or the second depth frame corresponding to the similarity satisfying the preset condition, so as to reduce the total number of depth frames; and updating the three-dimensional model on the basis of the reduced number of depth frames.

Description

Method for creating a three-dimensional oral model and an electronic device for performing the same

[0001] The present disclosure relates to a method for generating a three-dimensional oral cavity model and an electronic device for performing the same, and more particularly, to a method for processing depth data used for generating a three-dimensional oral cavity model.

[0002] Digital Impression Scanning (DIS) is the process of digitizing the interior of the oral cavity using a 3D scanner, replacing the traditional physical impression (which involves taking a mold of the oral cavity's interior). This method uses an intraoral scanner (usually an intraoral scanner) to precisely scan a patient's teeth, gums, and oral structure, then converts them into a digital 3D model.

[0003] Typically, intraoral scanners are connected to a PC installed in the dental clinic, transmitting scan data to the PC, which then creates a 3D model in real time. While dental PC performance has recently improved, generating high-resolution 3D models in real time still requires significant resources. In particular, as the amount of data acquired by the intraoral scanner increases, so does the resource consumption required to process it.

[0004] Because of this, many intraoral scanner manufacturers have implemented restrictions to prevent users from collecting more than a certain amount of scan data. For example, some software running on a PC provides a guidance message to prevent further scans once a certain amount of scan data has been collected.

[0005] However, if the user's skill level is low, these limitations can cause significant inconvenience. For example, if the user is unable to continue scanning before collecting sufficient scan data, this can be frustrating and ultimately make it difficult to complete a 3D model of the desired quality.

[0006] Therefore, a technology is required that can generate high-resolution 3D models in real time without limiting the collection of scan data.

[0007] The technical problem that the present invention seeks to solve is to create a high-resolution 3D model in real time without placing any restrictions on the collection of scan data.

[0008] Another technical problem that the present invention seeks to solve is to maintain the resources of an electronic device at a certain level while generating a 3D model.

[0009] According to one embodiment of the present disclosure, a method for generating a three-dimensional oral cavity model performed by an electronic device may be provided, the method including: obtaining a plurality of first depth frames including depth information of an inside of an oral cavity based on first scan data received from an oral cavity scanner; generating a three-dimensional model representing the inside of the oral cavity based on the plurality of first depth frames; obtaining a second depth frame based on second scan data received from the oral cavity scanner after receiving the first scan data; determining a similarity between at least one characteristic of each of the plurality of first depth frames and the second depth frame; reducing the total number of depth frames by processing at least one of the first depth frame and the second depth frame corresponding to the similarity satisfying the preset condition when the determined similarity satisfies a preset condition; and updating the three-dimensional model based on the depth frames of which the number is reduced.

[0010] The at least one characteristic may include pose information, and the preset condition may be a case where the similarity between the pose information is greater than a preset first threshold value.

[0011] The at least one characteristic may include pose information and depth information, and the preset condition may be a case where the similarity between the pose information is greater than a preset first threshold value and the similarity between the depth information is greater than a preset second threshold value.

[0012] The at least one characteristic may include pose information and a scan area, and the preset condition may be a case where the similarity between the pose information is less than or equal to a preset first threshold value and the similarity between the scan areas is greater than a preset third threshold value.

[0013] The step of reducing the number of total depth frames may include the step of identifying a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset condition, and the step of merging the identified first depth frame and the second depth frame to generate a new depth frame.

[0014] The above merging step may merge the identified first depth frame and the second depth frame based on a weight assigned according to the quality of each of the identified first depth frame and the second depth frame.

[0015] The step of reducing the number of total depth frames may include the step of identifying a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset condition, and the step of removing a depth frame having a lower quality among the identified first depth frames and the second depth frames.

[0016] The step of reducing the number of total depth frames may include the step of identifying a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset condition, and the step of removing a depth frame having an earlier acquisition time among the identified first depth frame and the second depth frame.

[0017] The step of reducing the number of total depth frames above allows a greater number of depth frames to be processed per unit time when scanning an area already scanned by the oral scanner compared to when scanning a new area.

[0018] The step of reducing the total number of depth frames may be performed so that the number of depth frames processed per unit time increases as the movement speed of the oral scanner increases.

[0019] The degree of change in computing resource usage required to perform the above 3D model generation method decreases as the scanning session progresses, and the computing resources may include at least one of CPU usage, GPU usage, memory usage, and processing time.

[0020] The steps of acquiring the plurality of first depth frames, generating the three-dimensional model, acquiring the second depth frame, determining the similarity, reducing the total number of depth frames, and updating the three-dimensional model can be performed within a single scan session.

[0021] According to one embodiment of the present disclosure, an electronic device may be provided, comprising: a communication interface; a memory including at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, obtains a plurality of first depth frames including depth information of an inside of an oral cavity based on first scan data received from an oral scanner, generates a three-dimensional model representing the inside of the oral cavity based on the plurality of first depth frames, obtains a second depth frame based on second scan data received from the oral scanner after receiving the first scan data, determines a similarity between at least one characteristic between each of the plurality of first depth frames and the second depth frame, and, when the determined similarity satisfies a preset condition, processes at least one of the first depth frame and the second depth frame corresponding to the similarity satisfying the preset condition to reduce the total number of depth frames, and updates the three-dimensional model based on the depth frames whose number is reduced.

[0022] The processor can identify a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset conditions, and merge the identified first depth frame and the second depth frame to generate a new depth frame.

[0023] The processor can merge the identified first depth frame and the second depth frame based on a weight assigned according to the quality of each of the identified first depth frame and the second depth frame.

[0024] The processor can identify a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset conditions, and remove a depth frame having a lower quality among the identified first depth frame and the second depth frame.

[0025] The processor can identify a first depth frame having the highest similarity to the second depth frame among a plurality of first depth frames corresponding to similarities that satisfy the preset conditions, and remove a depth frame having an earlier acquisition time among the identified first depth frame and the second depth frame.

[0026] According to one embodiment of the present disclosure, a data processing method performed by an electronic device may be provided, the data processing method including: obtaining depth data composed of a plurality of depth frames including depth information for an inside of an oral cavity scanned by an oral scanner based on scan data received from an oral scanner; obtaining pose information of the oral scanner corresponding to each of the plurality of depth frames by aligning the plurality of depth frames; determining whether a first similarity between first pose information corresponding to a first depth frame among the plurality of depth frames and second pose information corresponding to a second depth frame is greater than a preset first threshold; and reducing the number of the plurality of depth frames by deleting the first depth frame or the second depth frame or merging the first depth frame and the second depth frame based on a result of the determination.

[0027] The step of reducing the number of the plurality of depth frames may be performed when it is determined that the first similarity is greater than the preset first threshold value.

[0028] The above data processing method further includes a step of determining whether a second similarity between the first depth frame and the second depth frame is greater than a second preset threshold value when the first similarity is determined to be greater than the first preset threshold value; and the step of reducing the number of the plurality of depth frames may be performed when the first similarity is determined to be greater than the first preset threshold value and the second similarity is determined to be greater than the second preset threshold value.

[0029] The above data processing method further includes a step of determining whether a third similarity between a first scan area corresponding to the first depth frame and a second scan area corresponding to the second depth frame is greater than a third preset threshold value when the first similarity is determined to be less than or equal to the first preset threshold value; and the step of reducing the number of the plurality of depth frames may be performed when the third similarity is determined to be greater than the third preset threshold value.

[0030] The above data processing method may further include a step of comparing the quality of the first depth frame and the second depth frame; and a step of identifying a depth frame with low quality among the first depth frame and the second depth frame as a target for deletion.

[0031] The above data processing method may further include a step of comparing the acquisition times of the first depth frame and the second depth frame; and a step of identifying a depth frame having an earlier acquisition time among the first depth frame and the second depth frame as a deletion target.

[0032] When merging the first depth frame and the second depth frame, the first depth frame and the second depth frame may be merged based on a weight assigned according to the quality of each of the first depth frame and the second depth frame.

[0033] The step of reducing the number of said plurality of depth frames may delete or merge a greater number of depth frames per unit time when scanning an area already scanned by the oral scanner compared to when scanning a new area.

[0034] The step of reducing the number of the plurality of depth frames may be performed so that the number of depth frames deleted or merged per unit time increases as the movement speed of the oral scanner decreases.

[0035] The above data processing method further includes a step of generating a 3D model representing the inside of the oral cavity based on the plurality of depth frames; and a change in the usage of resources required while generating the 3D model may be greater in the early part of the scan session compared to the latter part in which the step of reducing the number of the plurality of depth frames is performed.

[0036] According to one embodiment of the present disclosure, an electronic device may be provided, comprising: a communication interface; a memory including at least one instruction; and a processor; wherein the processor, by executing the at least one instruction, obtains depth data composed of a plurality of depth frames including depth information for an inside of an oral cavity scanned by the oral scanner based on scan data received from an oral scanner through the communication interface, and aligns the plurality of depth frames to obtain pose information of the oral scanner corresponding to each of the plurality of depth frames; determines whether a first similarity between first pose information corresponding to a first depth frame among the plurality of depth frames and second pose information corresponding to a second depth frame is greater than a preset first threshold; and reduces the number of the plurality of depth frames by deleting the first depth frame or the second depth frame or merging the first depth frame and the second depth frame based on a result of the determination.

[0037] The processor may reduce the number of the plurality of depth frames when it is determined that the first similarity is greater than the preset first threshold value.

[0038] The processor may determine whether the second similarity between the first depth frame and the second depth frame is greater than the second preset threshold value when the first similarity is determined to be greater than the first preset threshold value and the second similarity is determined to be greater than the second preset threshold value, and may reduce the number of the plurality of depth frames when the first similarity is determined to be greater than the first preset threshold value and the second similarity is determined to be greater than the second preset threshold value.

[0039] The processor may determine whether a third similarity between a first scan area corresponding to the first depth frame and a second scan area corresponding to the second depth frame is greater than a third preset threshold value when the first similarity is determined to be less than the first preset threshold value, and may reduce the number of the plurality of depth frames when the third similarity is determined to be greater than the third preset threshold value.

[0040] The processor can compare the quality of the first depth frame and the second depth frame, and identify a depth frame with low quality among the first depth frame and the second depth frame as a target for deletion.

[0041] The processor can compare the acquisition times of the first depth frame and the second depth frame, and identify a depth frame having an earlier acquisition time among the first depth frame and the second depth frame as a deletion target.

[0042] The processor may merge the first depth frame and the second depth frame based on a weight assigned according to the quality of each of the first depth frame and the second depth frame when merging the first depth frame and the second depth frame.

[0043] The processor may be capable of deleting or merging a greater number of depth frames per unit time when scanning an area already scanned by the oral scanner compared to when scanning a new area.

[0044] The processor may reduce the number of the plurality of depth frames so that the number of depth frames deleted or merged per unit time increases as the movement speed of the oral scanner decreases.

[0045] The processor generates a 3D model representing the inside of the oral cavity based on the plurality of depth frames, and a change in the usage of resources required while generating the 3D model may be greater in the early part of the scan session compared to the latter part of the scan session in which the number of the plurality of depth frames is reduced.

[0046] The above exemplary embodiments and other exemplary embodiments will be explained or clarified by the detailed description set forth below of exemplary embodiments to be read in connection with the accompanying drawings.

[0047] The disclosed technology may have the following effects. However, this does not mean that a particular embodiment must include all or only the following effects, and thus the scope of the disclosed technology should not be construed as being limited thereby.

[0048] According to one embodiment of the present disclosure, a high-resolution 3D model can be generated in real time without limiting the collection of scan data. Furthermore, the resources of an electronic device can be maintained at a certain level while generating the 3D model.

[0049] The above description of the invention is not intended to be an exhaustive list of all aspects of the present invention. It should be understood that the present invention encompasses all methods, devices, and systems capable of being implemented from all appropriate combinations of the various aspects disclosed in the detailed description and claims below, as well as the matters summarized above. Furthermore, the detailed description of the embodiments of the present disclosure will directly or implicitly disclose any benefits that may be obtained or anticipated from the embodiments of the present disclosure. For example, various anticipated benefits of the embodiments of the present disclosure will be disclosed in the detailed description that follows.

[0050] Aspects, features and advantages of specific embodiments of the present disclosure will become more apparent from the following description taken in conjunction with the accompanying drawings.

[0051] FIG. 1 is a schematic diagram illustrating an oral scanning system according to one embodiment of the present disclosure.

[0052] FIG. 2 is a block diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0053] FIG. 3 is a flowchart illustrating a depth data processing method according to one embodiment of the present disclosure.

[0054] FIG. 4 is a flowchart illustrating a depth data processing method according to another embodiment of the present disclosure.

[0055] FIG. 5 is a flowchart illustrating a depth data processing method according to another embodiment of the present disclosure.

[0056] FIG. 6 is a flowchart illustrating a method for obtaining a 3D model according to one embodiment of the present disclosure.

[0057] FIG. 7 illustrates an oral scanner scanning a lower jaw according to one embodiment of the present disclosure and depth data and a 3D model obtained according to the operation of the oral scanner.

[0058] FIG. 8 is a graph showing computing resource usage of an electronic device in a scan session according to one embodiment of the present disclosure.

[0059] FIG. 9 is a block diagram showing the configuration of an oral scanning system according to one embodiment of the present disclosure.

[0060] The terms used in this specification will be briefly explained, and the present disclosure will be described in detail.

[0061] The terms used in the embodiments of this disclosure have been selected from widely used, current terms, taking into account the functions of this disclosure. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, and in such cases, their meanings will be described in detail in the description of the relevant disclosure. Therefore, the terms used in this disclosure should not be defined simply as names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0062] The embodiments of the present disclosure are capable of various modifications and multiple embodiments. Specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the scope of the present disclosure to specific embodiments, but rather to encompass all modifications, equivalents, and alternatives falling within the scope of the disclosed concepts and techniques. In describing the embodiments, detailed descriptions of related known technologies will be omitted if they are deemed to obscure the main point.

[0063] Terms such as "first" and "second" may be used to describe various components, but the components should not be limited by these terms. These terms are used solely to distinguish one component from another.

[0064] Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0065] Below, with reference to the attached drawings, embodiments of the present disclosure are described in detail so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description are omitted for clarity of description of the present disclosure, and similar parts are designated with similar reference numerals throughout the specification.

[0066]

[0067] FIG. 1 is a schematic diagram illustrating an oral scanning system according to one embodiment of the present disclosure.

[0068] Referring to FIG. 1, an oral scanning system (1000) may include an oral scanner (100) and an electronic device (200). A user (1) may scan the inside of a patient's (2) oral cavity using the oral scanner (100). The user (1) may be a dental staff member. For example, dental staff members may include dentists, dental hygienists, dental assistants, orthodontists, dental technicians, and oral surgeons.

[0069] The oral scanner (100) can transmit scan data acquired during the scanning process to an electronic device (200). The electronic device (200) can generate a 3D model representing the inside of the oral cavity based on the scan data. In addition, the electronic device (200) can display the 3D model on the display. The user (1) can continue the scanning process while checking the 3D model displayed on the electronic device (200). The scanning target of the oral scanner (100) is not limited to the inside of the patient's (2) oral cavity, and may also include structures that are scheduled to be installed or applied to the inside of the oral cavity in the future or are intermediate results therefor, even if they are not currently located in the oral cavity.

[0070] The electronic device (200) can be connected to the oral scanner (100) via a wired or wireless connection to exchange data. The electronic device (200) may be a desktop, laptop, tablet PC, or a computer designed as an embedded system specifically for the purpose of using the oral scanner (100). The electronic device (200) may be implemented by being physically or functionally integrated with the oral scanner (100).

[0071] FIG. 2 is a block diagram illustrating the operation of an electronic device according to one embodiment of the present disclosure.

[0072] Referring to FIG. 2, the electronic device (200) can obtain scan data (21). Specifically, the electronic device (200) can receive scan data (21) from an oral scanner (100).

[0073] Scan data (21) refers to raw data acquired by the oral scanner (100) during the oral scanning process. For example, the scan data (21) may include an RGB image photographing the inside of the oral cavity, an infrared image, an intensity image indicating the reflectivity of the tooth surface, a polarization image, etc. The scan data (21) may be a monocular image captured by a single camera, or a pair of stereo images captured from different angles. In addition, the scan data (21) may include a structured light pattern projected onto the tooth surface.

[0074] Scan data (21) is composed of a plurality of scan frames acquired in a time series manner. Here, the scan frame may be a unit data including scan information of the inside of the oral cavity captured by the oral scanner (100) at a specific point in time, and may be composed of one of the forms of the scan data (21) described above.

[0075] Depth data (22) refers to data containing depth information about the inside of the oral cavity. For example, the depth data (22) may take the form of a depth map that stores distance information to teeth and surrounding tissues in a two-dimensional array format, a point cloud composed of three-dimensional coordinates of teeth and surrounding tissues, a triangle mesh of the tooth surface, etc.

[0076] Depth data (22) is composed of multiple depth frames acquired in a time-series manner. Here, a depth frame may be unit data including depth information in a three-dimensional space acquired at a specific point in time, and may be composed of one of the forms of the depth data (22) described above.

[0077] Depth data (22) can be acquired in various ways. In one embodiment, the electronic device (200) can derive depth data (22) based on scan data (21). For example, if the scan data (21) is a stereo image, the electronic device (200) can analyze the disparity between the stereo images to generate a depth map. Additionally, the electronic device (200) can acquire a point cloud based on the internal parameters of the oral scanner (100) and the depth map.

[0078] In another embodiment, depth data (22) may be acquired by an oral scanner (100). For example, the oral scanner (100) may directly acquire depth data (22) using a Time-of-Flight (TOF) sensor or an active stereo method. That is, if the scan data (21) already includes three-dimensional shape information, the scan data (21) itself may be utilized as depth data (22).

[0079] The electronic device (200) can reconstruct an intraoral model (23) based on the depth data (22). Specifically, the electronic device (200) can register a plurality of depth frames included in the depth data (22). For example, the electronic device (200) can perform local registration between time-series adjacent depth frames. Here, each depth frame can be a 3D point cloud. The electronic device (200) can generate a transformation matrix including rotation and translation components for registering two depth frames through registration.

[0080] The electronic device (200) can obtain pose information of the oral scanner (100) based on a transformation matrix. The pose information can include position and posture information of the oral scanner (100). The electronic device (200) can calculate a relative pose change of the oral scanner (100) based on the transformation matrix. The electronic device (200) can use this pose information to align 3D point clouds of each depth frame to create an intraoral model.

[0081] In one embodiment, the electronic device (200) can generate a pose graph in which the pose information of the oral scanner (100) at the time of acquisition of each depth frame is used as nodes and the relative transformation relationship between the pose information is used as edges. The electronic device (200) can perform pose graph optimization to globally optimize the pose information of each node. The electronic device (200) can generate a more precise intraoral model by realigning the 3D point cloud of each depth frame using the optimized pose information. In addition, when the oral scanner (100) rescans an area that has been previously scanned, the electronic device (200) can detect a loop closure and add it as a new edge of the pose graph, thereby effectively correcting accumulated drift errors.

[0082] The intraoral model (23) refers to a three-dimensional model that digitally represents the oral structure of a patient. The intraoral model (23) may include a maxillary model, a mandibular model, and an occlusal model. The maxillary model is a digital model representing the three-dimensional shape of the patient's maxillary arch. The mandibular model is a digital model representing the three-dimensional shape of the patient's mandibular arch. The occlusal model is a digital model representing the three-dimensional shape of the patient's maxillary model and mandibular model in an occluded state. The occlusal model reflects the patient's actual occlusal state, which may include the contact relationship and occlusal gap between the maxillary and mandibular teeth.

[0083] The type of intraoral model (23) may vary depending on the scan mode. The electronic device (200) may operate in multiple scan modes. The multiple scan modes may include an upper jaw scan mode, a lower jaw scan mode, and an occlusion scan mode.

[0084] The maxillary scan mode is a scanning mode for scanning the maxilla. In the maxillary scan mode, the user (1) can scan the maxilla using the oral scanner (100). The oral scanner (100) can collect scan data for the maxilla. The electronic device (200) can receive scan data for the maxilla from the oral scanner (100) and create a maxillary model.

[0085] The mandibular scan mode is a scan mode for scanning the mandible. In the mandibular scan mode, the user (1) can scan the mandible using the oral scanner (100). The oral scanner (100) can collect scan data for the mandible. The electronic device (200) can receive scan data for the mandible from the oral scanner (100) and create a mandibular model.

[0086] The occlusion scan mode is a scan mode for scanning occlusion. In the occlusion scan mode, the user (1) can scan the upper and lower jaws in occlusion using the oral scanner (100). The oral scanner (100) can collect scan data on occlusion. The electronic device (200) can receive scan data on occlusion from the oral scanner (100) and create an occlusion model.

[0087] Meanwhile, in the present disclosure, a "scan session" may refer to a period of time during which a scanning operation is performed in a specific scan mode. For example, a scan session in the maxillary scan mode may be the period from the time a user (1) begins scanning the maxilla using an oral scanner (100) to the time the maxillary scan is completed.

[0088] FIG. 3 is a flowchart illustrating a depth data processing method according to one embodiment of the present disclosure.

[0089] Referring to FIG. 3, the electronic device (200) can acquire depth data including a plurality of depth frames (S310). The electronic device (200) can align the plurality of depth frames to acquire pose information corresponding to each depth frame (S320).

[0090] The electronic device (200) can determine the similarity of pose information corresponding to multiple depth frames (S330). Specifically, the electronic device (200) can calculate a first similarity between first pose information corresponding to a first depth frame and second pose information corresponding to a second depth frame, and determine whether the calculated first similarity is greater than a preset first threshold. Here, the first similarity between the pose information can be calculated by considering both positional differences and posture differences.

[0091] If the pose information is similar (S330: Yes), that is, if it is determined that the first similarity is greater than the preset first threshold value, the electronic device (200) can reduce the number of multiple depth frames (S340).

[0092] In one embodiment, the electronic device (200) can selectively delete either the first depth frame or the second depth frame. Here, the pose information of the depth frame being deleted can also be deleted. For example, the electronic device (200) can compare the quality of the first depth frame and the second depth frame and delete the depth frame with lower quality among the two. Alternatively, the electronic device (200) can compare the acquisition times of the first depth frame and the second depth frame and delete the depth frame with an earlier acquisition time among the two. Conversely, it is also possible to delete the depth frame with a later acquisition time among the two depth frames.

[0093] In another embodiment, the electronic device (200) may merge the first depth frame and the second depth frame to generate a new depth frame. After the new depth frame is generated through the merge, the electronic device (200) may delete the first depth frame and the second depth frame.

[0094] For example, the electronic device (200) may determine the depth value of a new depth frame by taking the arithmetic mean of the depth values ​​at each pixel location of the first depth frame and the depth values ​​at the corresponding pixel locations of the second depth frame. As another example, the electronic device (200) may generate a new depth frame by applying quality-based weights to the first depth frame and the second depth frame. Specifically, the electronic device (200) may calculate a first quality value of the first depth frame and a second quality value of the second depth frame, and determine the first weight and the second weight based on the calculated first quality value and the second quality value. For example, when the first quality value is greater than the second quality value, the first weight may be determined to be greater than the second weight. The electronic device (200) may generate a new depth frame by applying the determined weights to the first depth frame and the second depth frame, respectively. Here, the quality value of the depth frame can be determined based on the reliability, noise level, or reflectivity of the depth values.

[0095] When a new depth frame is generated by merging the first depth frame and the second depth frame, the electronic device (200) can determine new pose information corresponding to the new depth frame based on the first pose information and the second pose information.

[0096] In one embodiment, the electronic device (200) can simply merge the first pose information and the second pose information. Specifically, the location of the new pose information can be determined as the arithmetic mean of the locations of the first pose information and the locations of the second pose information, and the pose can be determined by interpolating the pose of the first pose information and the pose of the second pose information.

[0097] In another embodiment, the electronic device (200) may determine a first weight and a second weight based on a first quality value of a first depth frame and a second quality value of a second depth frame, and apply the first and second weights to determine new pose information. Specifically, the position of the new pose information may be determined as a weighted average of the position of the first pose information to which the first weight is applied and the position of the second pose information to which the second weight is applied, and the pose may be determined by interpolating the pose of the first pose information and the pose of the second pose information by applying the first and second weights.

[0098] Meanwhile, when an area already scanned by the oral scanner (100) is scanned, the electronic device (200) may delete or merge a greater number of depth frames per unit time compared to when a new area is scanned. This is because similar depth frames may be acquired when the same area is continuously scanned. In addition, as the movement speed of the oral scanner (100) decreases, the number of depth frames deleted or merged per unit time may increase. This is because the slower the movement speed of the oral scanner (100), the more likely it is that depth frames similar to the already acquired depth frames will be acquired as an area overlapping with an already scanned area is scanned.

[0099] The electronic device (200) can generate a 3D model representing the oral cavity based on a plurality of depth frames. The change in resource usage required to generate the 3D model may be greater in the early part of the scan session compared to the latter part, when the number of depth frames is reduced.

[0100] If the pose information is not similar (S330: No), that is, if it is determined that the first similarity is less than or equal to the preset first threshold value, the electronic device (200) may not reduce the number of multiple depth frames.

[0101] FIG. 4 is a flowchart illustrating a depth data processing method according to another embodiment of the present disclosure.

[0102] Referring to FIG. 4, the electronic device (200) can obtain depth data including a plurality of depth frames (S410). The electronic device (200) can align the plurality of depth frames to obtain pose information corresponding to each depth frame (S420). The electronic device (200) can determine the similarity of the pose information corresponding to the plurality of depth frames (S430). S410, S420, and S430 are substantially the same as S310, S320, and S330 of FIG. 3, respectively, and therefore, redundant descriptions are omitted.

[0103] If the pose information is similar (S430: Yes), the electronic device (200) can determine the similarity between the depth information of the multiple depth frames (S440). Specifically, the electronic device (200) can calculate the second similarity between the first depth information of the first depth frame and the second depth information of the second depth frame, and can determine whether the similarity exceeds a preset second threshold. In the present disclosure, the second similarity may refer to the similarity between the depth information included in each depth frame.

[0104] If the depth information is similar, i.e., if it is determined that the second similarity is greater than the preset second threshold value (S440: Yes), the electronic device (200) may reduce the number of multiple depth frames (S450). S450 is substantially the same as S340 of FIG. 3, and therefore, a duplicate description will be omitted.

[0105] If the depth frames are not similar, that is, if it is determined that the second similarity is less than or equal to the preset second threshold value (S440: No), the electronic device (200) may not reduce the number of multiple depth frames.

[0106] FIG. 5 is a flowchart illustrating a depth data processing method according to another embodiment of the present disclosure.

[0107] Referring to FIG. 5, the electronic device (200) can obtain depth data including a plurality of depth frames (S510). The electronic device (200) can align the plurality of depth frames to obtain pose information corresponding to each depth frame (S520). The electronic device (200) can determine the similarity of the pose information corresponding to the plurality of depth frames (S530). S510, S520, and S530 are substantially the same as S310, S320, and S330 of FIG. 3, respectively, and therefore, redundant descriptions are omitted.

[0108] If the pose information is similar (S530: Yes), the electronic device (200) can reduce the number of depth frames (S540). S540 is substantially the same as S340 of FIG. 3, and thus, a duplicate description is omitted.

[0109] If the pose information is not similar (S530: No), the electronic device (200) can determine the similarity of scan areas corresponding to multiple depth frames (S550). Specifically, the electronic device (200) can calculate a third similarity between a first scan area corresponding to a first depth frame and a second scan area corresponding to a second depth frame. Here, the scan area may refer to a specific area inside the oral cavity, and the depth frame may be generated based on scan data acquired from the oral scanner (100) at the time when the corresponding scan area was scanned. The electronic device (200) can determine whether the calculated third similarity is greater than a preset third threshold. The third similarity between the scan areas can be calculated by comparing the three-dimensional coordinate information of each scan area.

[0110] If the scan areas are similar, i.e., if it is determined that the third similarity is greater than the preset second threshold value (S550: Yes), the electronic device (200) may reduce the number of multiple depth frames (S540). Even if the pose information is not similar, if the scan areas are similar, the electronic device (200) may reduce the number of multiple depth frames.

[0111] If the scan areas are not similar, that is, if it is determined that the third similarity is less than or equal to the preset second threshold value (S550: No), the electronic device (200) may not reduce the number of multiple depth frames.

[0112] FIG. 6 is a flowchart illustrating a method for obtaining a 3D model according to one embodiment of the present disclosure.

[0113] Referring to FIG. 6, the electronic device (200) can acquire a plurality of first depth frames based on first scan data (S610). The electronic device (200) can receive the first scan data from the oral scanner (100). Each first depth frame can include depth information about the inside of the oral cavity.

[0114] The electronic device (200) can generate a three-dimensional model representing the inside of the oral cavity based on a plurality of first depth frames (S620).

[0115] The electronic device (200) can acquire a second depth frame based on the second scan data (S630). The electronic device (200) can receive the second scan data from the oral scanner (100) after receiving the first scan data.

[0116] The electronic device (200) can determine the similarity of at least one characteristic between a plurality of first depth frames and a second depth frame (S640). The at least one characteristic may include pose information, depth information, and a scan area.

[0117] The pose information of a depth frame may refer to pose information that temporally corresponds to the corresponding depth frame. Specifically, the pose information of the oral scanner (100) at the time when the scan frame that serves as the basis of the depth frame is acquired may be defined as the pose information of the depth frame. The electronic device (200) may compare the pose information of each of a plurality of first depth frames with the pose information of a second depth frame to determine the similarity between the pose information. Here, the similarity between the pose information may refer to the first similarity of FIG. 3.

[0118] The depth information of a depth frame may refer to the depth information contained in the depth frame, i.e., the depth information inside the oral cavity. The electronic device (200) may compare the depth information of each of a plurality of first depth frames with the depth information of a second depth frame to determine the similarity between the depth information. Here, the similarity between the depth information may refer to the second similarity of FIG. 4.

[0119] The scan area of ​​a depth frame may refer to a scan area that temporally corresponds to the corresponding depth frame. Specifically, the scan area of ​​the oral scanner (100) at the time when the scan frame that serves as the basis of the depth frame is acquired may be defined as the scan area of ​​the depth frame. The electronic device (200) may compare the scan area of ​​each of a plurality of first depth frames with the scan area of ​​a second depth frame to determine the similarity between the scan areas. Here, the similarity between the scan areas may refer to the third similarity of FIG. 5.

[0120] If the determined similarity satisfies a preset condition, the electronic device (200) can process at least one of the first depth frame and the second depth frame corresponding to the similarity that satisfies the preset condition to reduce the total number of depth frames (S650).

[0121] In one embodiment, the preset first condition may be a case where the similarity between pose information is greater than a preset first threshold, and where at least one pose information among the plurality of first depth frames is similar to the pose information of the second depth frame.

[0122] In another embodiment, the preset second condition may be a case where the similarity between pose information is greater than a preset first threshold and the similarity between depth information is greater than a preset second threshold, and where at least one of the pose information and depth information among the plurality of first depth frames is similar to the second depth frame.

[0123] In another embodiment, the third predetermined condition may be a case where the similarity between pose information is less than or equal to a first predetermined threshold and the similarity between scan areas is greater than a third predetermined threshold, and at least one of the plurality of first depth frames has pose information that is not similar to the second depth frame but has a similar scan area.

[0124] The electronic device (200) can process at least one of a first depth frame and a second depth frame corresponding to a similarity satisfying a preset condition. In one embodiment, the electronic device (200) can merge the first depth frame and the second depth frame to generate a new depth frame. At this time, the electronic device (200) can merge the two depth frames based on a weight assigned according to the quality of each of the two depth frames. If there are multiple first depth frames corresponding to a similarity satisfying the preset condition, the electronic device (200) can identify the first depth frame having the highest similarity with the second depth frame. In addition, the electronic device (200) can merge the identified first depth frame and the second depth frame.

[0125] In another embodiment, the electronic device (200) may delete one of the two depth frames. For example, the electronic device (200) may delete a depth frame with relatively lower quality or a depth frame acquired earlier. Alternatively, the electronic device (200) may also delete a depth frame acquired later. If there are multiple first depth frames corresponding to similarities that satisfy a preset condition, the electronic device (200) may identify the first depth frame with the highest similarity to the second depth frame. Then, the electronic device (200) may delete the identified first or second depth frame.

[0126] The electronic device (200) can update the 3D model based on the reduced number of depth frames (S660). Updating the 3D model may include optimization operations for the 3D model. For example, the optimization operations may include iterative graph-based data alignment and alignment processes to minimize data alignment errors.

[0127] Typically, the most resource-intensive tasks in intraoral scanning systems are the creation and updating of 3D models. Therefore, to prevent resource overload, existing intraoral scanning systems limit the user's scanning activity to a certain number of scan frames. For example, existing intraoral scanning systems provide users with a scan allowance and limit scanning activity to prevent exceeding this limit.

[0128] In contrast, the oral scanning system (1000) according to the present disclosure can provide a higher degree of scanning freedom compared to existing oral scanning systems by not restricting the scanning operation of the user (1). This may be possible because overlapping similar depth frames are deleted and not used for 3D model updates. Since overlapping depth frames are not used for 3D model updates, resource overload may not occur even if the user (1) continuously scans the same area.

[0129] Meanwhile, the shape of a 3D model may change during the 3D model update process. Even if a depth frame is deleted, the quality of the 3D model can be maintained because a similar depth frame was previously used to create or update the 3D model.

[0130] Although overlapping depth frames are described as being deleted in FIG. 6, in other embodiments, overlapping depth frames can be excluded from 3D model updates without being deleted. In this case, the electronic device (200) may store overlapping depth frames but not use them for 3D model updates. Accordingly, even if the user (1) continuously scans the same area, the resources consumed for 3D model updates can be maintained within a certain range.

[0131] Meanwhile, the number of depth frames deleted may vary depending on the scan mode of the oral scanning system (1000). For example, in the occlusion scan mode, the number of depth frames deleted may be greater than in the maxillary or mandibular scan mode. This is because in the occlusion scan mode, depth data already secured in the maxillary or mandibular scan mode exists, which may result in more duplicated data.

[0132] Meanwhile, S610, S620, S630, S640, S650 and S660 can be performed within a single scan session.

[0133] Fig. 7(A) illustrates an oral scanner scanning a lower jaw according to one embodiment. Fig. 7(B) illustrates depth data and a 3D model acquired according to the operation of the oral scanner of Fig. 7(A).

[0134] Referring to (A) and (B) of FIG. 7, the oral scanner (100) can scan the lower jaw (30) inside the oral cavity of a patient (2).

[0135] When the first region (R1) is scanned by the oral scanner (100), the electronic device (200) can obtain first depth data (31) including depth information of the first region (R1). The first depth data (31) can include a first depth frame (311), a second depth frame (312), and a third depth frame (313). The electronic device (200) can generate a 3D model (M) including a 3D representation of the first region (R1) based on the first depth data (31).

[0136] An electronic device (200) may acquire a fourth depth frame (32) by performing an additional scan on the oral scanner (100). The fourth depth frame (32) may include depth information of a first region (R1) already reflected in the 3D model (M). Alternatively, the fourth depth frame (32) may include depth information of another region (e.g., a second region (R2)) that is not reflected in the 3D model (M).

[0137] The electronic device (200) can determine the similarity between each of the depth frames (311, 312, 313) reflected in the creation of the 3D model (M) and at least one characteristic of the fourth depth frame (32) acquired after the creation of the 3D model (M) but not yet reflected. This is described in S640 of FIG. 6, and therefore, a duplicate description is omitted.

[0138] The electronic device (200) can determine whether the determined similarity satisfies a preset condition. If the preset condition is satisfied, the electronic device (200) can process depth frames corresponding to the similarity that satisfies the preset condition.

[0139] In one embodiment, the preset first condition may be when the similarity between pose information exceeds a preset first threshold. For example, the similarity between the pose information of the first depth frame (311) and the fourth depth frame (32) may exceed the preset first threshold. In this case, the electronic device (200) may merge the first depth frame (311) and the fourth depth frame (32) to generate a new frame. Alternatively, the electronic device (200) may delete the first depth frame (311) or the fourth depth frame (32).

[0140] Meanwhile, there may be multiple depth frames corresponding to similarities that satisfy the preset first condition. For example, both the first depth frame (311) and the second depth frame (312) may have pose information similar to that of the fourth depth frame (32). In this case, the electronic device (200) may select one of the two depth frames (311, 312) and merge it with the fourth depth frame (32), or delete the selected depth frame or the fourth depth frame (32).

[0141] The electronic device (200) may select a depth frame to process based on various criteria. For example, the electronic device (200) may select a depth frame having the highest similarity between pose information and the fourth depth frame (32). As another example, the electronic device (200) may select a depth frame having the highest similarity between depth information and the fourth depth frame (32). As yet another example, the electronic device (200) may select a depth frame having the highest similarity between scan areas and the fourth depth frame (32). Alternatively, the electronic device (200) may select a depth frame having a lower depth quality among the first depth frame (311) and the second depth frame (312).

[0142] In another embodiment, the preset second condition may be a case where the similarity between pose information exceeds a preset first threshold value and the similarity between depth information exceeds a preset second threshold value. At this time, the electronic device (200) may identify a depth frame that is similar to the fourth depth frame (32) in both pose information and depth information as a processing target by comparing the first depth frame (311) with the fourth depth frame (32). For example, the first depth frame (311) may be similar to the fourth depth frame (32) in both pose information and depth information, and the second depth frame (312) may be similar to the fourth depth frame (32) in pose information but not in depth information. At this time, the electronic device (200) may process at least one of the first depth frame (311) and the fourth depth frame (32), and may not process the second depth frame (312).

[0143] Meanwhile, there may be multiple depth frames corresponding to similarities that satisfy the preset second condition. For example, not only the first depth frame (311) but also the second depth frame (312) may be similar to the fourth depth frame (32) in both pose information and depth information. In this case, the electronic device (200) may select one of the two depth frames (311, 312) and merge it with the fourth depth frame (32), or delete the selected depth frame or the fourth depth frame (32).

[0144] The electronic device (200) may select a depth frame to process based on various criteria. For example, the electronic device (200) may select a depth frame having the highest similarity between pose information and the fourth depth frame (32). As another example, the electronic device (200) may select a depth frame having the highest similarity between depth information and the fourth depth frame (32). As yet another example, the electronic device (200) may select a depth frame having the highest similarity between scan areas and the fourth depth frame (32). Alternatively, the electronic device (200) may select a depth frame having a lower depth quality among the first depth frame (311) and the second depth frame (312).

[0145] In another embodiment, the preset third condition may be when the similarity between pose information is below a preset first threshold and the similarity between scan areas exceeds a preset third threshold. For example, the pose information of the third depth frame (313) may not be similar to that of the fourth depth frame (32), but the scan areas may be similar. In this case, the electronic device (200) may process at least one of the third depth frame (313) and the fourth depth frame (32).

[0146] Meanwhile, there may be multiple depth frames corresponding to similarities that satisfy the preset third condition. For example, not only the third depth frame (313) but also the second depth frame (312) may not have similar pose information to the fourth depth frame (32) but may have similar scan areas. In this case, the electronic device (200) may select one of the two depth frames (312, 313) and merge it with the fourth depth frame (32), or delete the selected depth frame or the fourth depth frame (32).

[0147] The electronic device (200) may select a depth frame to process based on various criteria. For example, the electronic device (200) may select a depth frame having the highest similarity between pose information and the fourth depth frame (32). As another example, the electronic device (200) may select a depth frame having the highest similarity between depth information and the fourth depth frame (32). As yet another example, the electronic device (200) may select a depth frame having the highest similarity between scan areas and the fourth depth frame (32). Alternatively, the electronic device (200) may select a depth frame having a lower depth quality among the second depth frame (312) and the third depth frame (313).

[0148] Meanwhile, a priority may be determined between the preset first condition and the preset third condition. For example, the priority of the preset first condition may be higher than the priority of the preset third condition. Accordingly, the electronic device (200) may determine whether the preset third condition is satisfied only when there is no depth frame corresponding to the similarity that satisfies the preset first condition. For example, if at least one of the depth frames (311, 312, 313) corresponds to the similarity that satisfies the preset first condition, i.e., if the pose information is similar to that of the fourth depth frame (32), the electronic device (200) may not determine whether the preset third condition is satisfied.

[0149] The electronic device (200) can update the 3D model (M) based on the remaining depth data that has not been deleted. For example, some depth frames similar to the fourth depth frame (32) in the first depth data (31) can be deleted, and the fourth depth frame (32) can be reflected in the 3D model (M).

[0150] Meanwhile, the number of depth frames processed per unit time (e.g., 1 second) may vary depending on the scan area of ​​the oral scanner (100). When an area that has already been scanned by the oral scanner (100) is scanned, a greater number of depth frames may be processed per unit time compared to when a new area is scanned. For example, when both the first depth data (31) and the fourth depth frame (32) are acquired based on a scan of the first area (R1), a first number of depth frames may be deleted per unit time (e.g., 1 second). On the other hand, when the first depth data (31) and the fourth depth frame (32) are acquired based on scans of different areas, for example, when the fourth depth frame (32) is acquired based on a scan of the second area (R2), a second number of depth frames may be deleted per unit time. In this case, the second number may be smaller than the first number. That is, when scanning the same area continuously, the number of depth frames dropped per unit time may be greater than when scanning while moving to a new area. This is because when scanning the same area, depth frames similar to those already acquired are acquired.

[0151] Depending on the scan area of ​​the oral scanner (100), the number of depth frames of the depth data used to update the 3D model (M) may also vary. When the oral scanner (100) scans an area that has already been scanned, that is, when the fourth depth frame (32) is acquired based on a scan of the first area (R1), the number of depth frames used to update the 3D model (M) may increase by a first increment per unit time. On the other hand, when the oral scanner (100) scans an area that has not yet been scanned, that is, when the fourth depth frame (32) is acquired based on a scan of the second area (R2), the number of depth frames used to update the 3D model (M) may increase by a second increment per unit time. In this case, the second increment may be greater than the first increment.

[0152] Meanwhile, the number of depth frames processed may vary depending on the movement speed of the oral scanner (100). Even when moving along the same path, the faster the average movement speed of the oral scanner (100), the smaller the number of depth frames deleted per unit time. For example, if the oral scanner (100) moves at a first average speed when scanning the first region (R1), a first number of depth frames may be deleted per unit time. On the other hand, if the oral scanner (100) moves at a second average speed higher than the first average speed when scanning the first region (R1), a second number of depth frames may be deleted per unit time. In this case, the first number may be greater than the second number.

[0153] FIG. 8 is a graph illustrating the computing resource usage of an electronic device in a scan session according to one embodiment of the present disclosure. Here, the computing resource may be a processor or memory of the electronic device (200).

[0154] Referring to Fig. 8, resource usage may gradually increase from a first time point (t1) to a second time point (t2), and may be maintained within a certain range from a second time point (t2) to a third time point (t3). Here, the first time point (t1) may indicate the start time of scanning, the second time point (t2) may indicate the time when the scan completion level reaches a predetermined level, and the third time point (t3) may indicate the time when the scanning session ends. The predetermined level may refer to the level at which depth data for the entire area of ​​the scan target is secured. Meanwhile, Fig. 8 is merely an example briefly showing the trend of resource usage, and is not limited thereto. Even if an increase or decrease in resource usage occurs locally, the following description may be equally applicable.

[0155] During the first period from the first time point (t1) to the second time point (t2), the oral scanner (100) moves and scans a new area that has not yet been scanned, so the amount of depth frames that the electronic device (200) processes (e.g., creates or updates a 3D model) may increase. While some of the overlapping depth frames may be deleted during the scanning process, the overall amount of depth frames may increase because more depth frames are additionally acquired. Therefore, resource usage may gradually increase during the first period.

[0156] During the second period, from the second time point (t2) to the third time point (t3), the oral scanner (100) scans an area that has already been scanned, so the number of depth frames deleted due to data duplication may be greater than in the first period. Therefore, during the second period, resource usage can be maintained without further increase, and the user (1) can perform scanning operations without restriction.

[0157] As shown, the change in resource usage may be greater in the first period than in the second period. The change in resource usage may refer to the absolute difference between the start and end points of each period. Resource usage may increase in the early stages of a scan session due to the acquisition of new data, while resource usage may remain constant in the latter stages due to the acquisition of duplicate data.

[0158] That is, the degree of change in computing resource usage required to perform a method for generating a three-dimensional model of the oral cavity may decrease as the scanning session progresses. Here, the computing resources may include at least one of CPU usage, GPU usage, memory usage, and processing time. The degree of change in computing resource usage may be the difference in computing resource usage between two consecutive updates, the standard deviation of computing resource usage within a given time interval, or the difference between a moving average of computing resource usage for a given number of consecutive updates and the actual measured value.

[0159] FIG. 9 is a block diagram showing the configuration of an oral scanning system according to one embodiment of the present disclosure.

[0160] Referring to FIG. 9, an oral scanning system (1000) may include an oral scanner (100) and an electronic device (200). The oral scanner (100) may acquire scan data of the inside of a patient's oral cavity and transmit it to the electronic device (200). The oral scanner (100) may include an optical sensor (e.g., a structured light projector, a stereo camera, or other optical depth sensor) for scanning the inside of the oral cavity, an inertial sensor (IMU) for estimating the movement of the oral scanner, or a separate imaging sensor.

[0161] The oral scanner (100) can acquire scan data using structured light, Time-of-Flight (ToF), LiDAR, Computed Tomography (CT), or ultrasound. The oral scanner (100) can include an IMU sensor. IMU information acquired through the IMU sensor can be used to acquire pose information or movement speed of the oral scanner (100).

[0162] The electronic device (200) may include a communication interface (210), memory (220), processor (230), and display (240). The electronic device (200) may be a personal computer (PC), tablet, workstation, or smartphone.

[0163] The communication interface (210) may include at least one communication circuit. The communication interface (210) may receive scan data from the oral scanner (100). The communication interface (210) may include a wired interface and a wireless interface. The wired interface may include USB, Ethernet, HDMI, and Thunderbolt. The wireless interface may include Wi-Fi, Bluetooth, Zigbee, and NFC.

[0164] The memory (220) may store an operating system (OS) for controlling the overall operation of components of the electronic device (200) and commands or data related to components of the electronic device (200). In particular, the memory (220) may include instructions for controlling oral scanning software. The memory (220) may be implemented as a non-volatile memory (e.g., a hard disk, a solid state drive (SSD), a flash memory) or a volatile memory.

[0165] The processor (230) is electrically connected to the memory (220) and can control the overall functions and operations of the electronic device (200). The processor (230) can control the overall functions and operations of the electronic device (200) by executing instructions stored in the memory (220).

[0166] The processor (230) can receive scan data from the oral scanner (100) via the communication interface (210).

[0167] The processor (230) can obtain a plurality of first depth frames including depth information inside the oral cavity based on first scan data received from the oral scanner (100).

[0168] The processor (230) can generate a three-dimensional model representing the inside of the oral cavity based on a plurality of first depth frames.

[0169] The processor (230) can obtain a second depth frame based on second scan data received from the oral scanner after receiving the first scan data.

[0170] The processor (230) can determine the similarity between at least one characteristic of each of the plurality of first depth frames and the second depth frame.

[0171] If the determined similarity satisfies a preset condition, the processor (230) can process at least one of the first depth frame and the second depth frame corresponding to the similarity satisfying the preset condition to reduce the total number of depth frames.

[0172] The processor (230) can generate a new depth frame by merging the first depth frame and the second depth frame corresponding to the similarity that satisfies the preset condition.

[0173] The processor (230) can merge the identified second depth frame and the second depth frame based on the weight assigned according to the quality of each of the first depth frame and the second depth frame corresponding to the similarity that satisfies the preset condition.

[0174] The processor (230) can remove a depth frame with lower quality among the first depth frame and the second depth frame corresponding to a similarity that satisfies a preset condition.

[0175] The processor (230) can remove a depth frame that is acquired earlier among the first depth frame and the second depth frame corresponding to a similarity that satisfies a preset condition.

[0176] The processor (230) can update the three-dimensional model based on the reduced number of depth frames.

[0177] The processor (230) can obtain depth data composed of a plurality of depth frames including depth information about the inside of the oral cavity scanned by the oral scanner (100) based on scan data.

[0178] The processor (230) can align a plurality of depth frames to obtain pose information of the oral scanner (100) corresponding to each of the plurality of depth frames.

[0179] The processor (230) can determine whether the first similarity between the first pose information corresponding to the first depth frame among the plurality of depth frames and the second pose information corresponding to the second depth frame is greater than a preset first threshold value.

[0180] The processor (230) can reduce the number of depth frames by deleting the first depth frame or the second depth frame or merging the first depth frame and the second depth frame based on the judgment result.

[0181] In one embodiment, if the first similarity is determined to be greater than a preset first threshold value, the processor (230) may reduce the number of multiple depth frames.

[0182] In another embodiment, if the first similarity is determined to be greater than a preset first threshold, the processor (230) may determine whether the second similarity between the first depth frame and the second depth frame is greater than a preset second threshold. If the first similarity is determined to be greater than the preset first threshold and the second similarity is determined to be greater than the preset second threshold, the processor (230) may reduce the number of multiple depth frames.

[0183] In another embodiment, if the first similarity is determined to be less than or equal to a preset first threshold, the processor (230) may determine whether a third similarity between a first scan area corresponding to the first depth frame and a second scan area corresponding to the second depth frame is greater than a preset third threshold. If the third similarity is determined to be greater than the preset third threshold, the processor (230) may reduce the number of multiple depth frames.

[0184] The processor (230) can compare the quality of the first depth frame and the second depth frame. The processor (230) can identify a depth frame with low quality among the first depth frame and the second depth frame as a target for deletion.

[0185] The processor (230) can compare the acquisition times of the first depth frame and the second depth frame. The processor (230) can identify the depth frame with the earlier acquisition time among the first depth frame and the second depth frame as a deletion target.

[0186] When merging the first depth frame and the second depth frame, the processor (230) can merge the first depth frame and the second depth frame based on a weight assigned according to the quality of each of the first depth frame and the second depth frame.

[0187] When an area that has already been scanned by the oral scanner (100) is scanned, the processor (230) can delete or merge a greater number of depth frames per unit time compared to when a new area is scanned.

[0188] The processor (230) can reduce the number of depth frames so that the number of depth frames deleted or merged per unit time increases as the movement speed of the oral scanner (100) decreases.

[0189] The processor (230) can generate a 3D model representing the oral cavity based on a plurality of depth frames. The change in resource usage required to generate the 3D model may be greater in the early part of the scan session compared to the latter part, where the number of depth frames is reduced.

[0190] The display (240) can output a 3D model under the control of the processor (230). In addition, the display (240) can output a UI element that can rotate or move the 3D model.

[0191] Although not shown, the electronic device (200) may include an input / output interface (I / O Interface) that is connected to an input / output device. For example, the I / O interface may include a USB and Bluetooth that are connected to a mouse. The processor (230) may receive user input through the I / O interface. A user may input a command to start or switch a scan mode or a command to start an optimization task through the I / O interface.

[0192] Although the preferred embodiments of the present disclosure have been illustrated and described above, the present disclosure is not limited to the specific embodiments described above, and various modifications may be made by a person skilled in the art to which the present disclosure pertains without departing from the gist of the present disclosure as claimed in the claims, and such modifications should not be understood individually from the technical idea or prospect of the present disclosure.

[0193] The various embodiments described above may be implemented in a computer-readable recording medium using software, hardware, or a combination thereof, or a computer or similar device. In some cases, the embodiments described herein may be implemented in the processor itself. When implemented in software, the embodiments, such as the procedures and functions described herein, may be implemented as separate software modules. Each of the software modules may perform one or more functions and operations described herein.

[0194] Computer instructions for performing processing operations according to the various embodiments of the present disclosure described above may be stored on a non-transitory computer-readable medium. When executed by a processor, the computer instructions stored on the non-transitory computer-readable medium may cause a specific device to perform processing operations according to the various embodiments described above.

[0195] A non-transitory computer-readable medium refers to a medium that permanently stores data and can be read by a device, rather than a medium that stores data for a short period of time, such as a register, cache, or memory. Specific examples of non-transitory computer-readable media include CDs, DVDs, hard disks, Blu-ray discs, USBs, memory cards, and ROMs.

[0196] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0197] The methods according to various embodiments disclosed in this document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) via an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. A method for generating a three-dimensional oral model performed by an electronic device, A step of acquiring a plurality of first depth frames containing depth information inside the oral cavity based on first scan data received from an oral scanner; A step of generating a three-dimensional model representing the inside of the oral cavity based on the plurality of first depth frames; A step of acquiring a second depth frame based on second scan data received from the oral scanner after receiving the first scan data; A step of determining the similarity of at least one characteristic between each of the plurality of first depth frames and the second depth frame; If the above similarity satisfies a preset condition, a step of processing at least one of the first depth frame and the second depth frame corresponding to the similarity satisfying the preset condition to reduce the total number of depth frames; and A step of updating the 3D model based on the depth frames with the number reduced; Method for generating a 3D oral cavity model.

2. In paragraph 1, The above at least one characteristic includes pose information, and The above-mentioned preset conditions are, When the similarity between the above pose information is greater than a preset first threshold, Method for generating a 3D oral cavity model.

3. In paragraph 1, The above at least one characteristic includes pose information and depth information, and The above-mentioned preset conditions are, The similarity between the above pose information is greater than a preset first threshold, and In the case where the similarity between the above depth information is greater than a preset second threshold, Method for generating a 3D oral cavity model.

4. In paragraph 1, The above at least one characteristic includes pose information and a scan area, and The above-mentioned preset conditions are, The similarity between the above pose information is below a preset first threshold value, and When the similarity between the above scan areas is greater than a preset third threshold, Method for generating a 3D oral cavity model.

5. In Paragraph 1, The step of reducing the number of the total depth frames above is, A step of generating a new depth frame by merging the first depth frame and the second depth frame corresponding to the similarity satisfying the above-described condition, Method for generating a 3D oral cavity model.

6. In paragraph 5, The above merging steps are: Assigning weights to each of the first depth frame and the second depth frame according to the quality corresponding to the similarity satisfying the above-described condition, and merging the identified first depth frame and the second depth frame based on the assigned weights. Method for generating a 3D oral cavity model.

7. In paragraph 1, The step of reducing the number of the total depth frames above is, A step of removing a depth frame having a lower quality among the first depth frame and the second depth frame corresponding to the similarity satisfying the above-described condition, Method for generating a 3D oral cavity model.

8. In paragraph 1, The step of reducing the number of the total depth frames above is, A step of removing a depth frame whose acquisition time is earlier among the first depth frame and the second depth frame corresponding to the similarity satisfying the above-described condition, Method for generating a 3D oral cavity model.

9. In paragraph 1, The step of reducing the number of the total depth frames above is, If an area that has already been scanned by the above oral scanner is scanned, Processing a greater number of depth frames per unit time compared to when a new area is scanned, Method for generating a 3D oral cavity model.

10. In paragraph 1, The step of reducing the number of the total depth frames above is, The number of depth frames processed per unit time is increased as the movement speed of the oral scanner decreases. Method for generating a 3D oral cavity model.

11. In paragraph 1, The degree of change in computing resource usage required to perform the above 3D model generation method decreases as the scanning session progresses, The above computing resources are, Including at least one of CPU usage, GPU usage, memory usage, and processing time. Method for generating a 3D oral cavity model.

12. In paragraph 1, The steps of acquiring the plurality of first depth frames, generating the three-dimensional model, acquiring the second depth frame, determining the similarity, decreasing the total number of depth frames, and updating the three-dimensional model are: performed within a single scan session, Method for generating a 3D oral cavity model.

13. In electronic devices, communication interface; Memory containing at least one instruction; and Processor; including; The above processor, by executing the above at least one instruction, Based on first scan data received from an oral scanner, a plurality of first depth frames including depth information inside the oral cavity are obtained, and A three-dimensional model representing the inside of the oral cavity is generated based on the plurality of first depth frames mentioned above, and After receiving the first scan data, a second depth frame is obtained based on the second scan data received from the oral scanner, and Determining the similarity of at least one characteristic between each of the plurality of first depth frames and the second depth frame, and If the similarity determined above satisfies a preset condition, at least one of the first depth frame and the second depth frame corresponding to the similarity satisfying the preset condition is processed to reduce the total number of depth frames, and Updating the 3D model based on the depth frames with the aforementioned reduced number, Electronic devices.

14. In paragraph 13, The above at least one characteristic includes pose information, and The above-mentioned preset conditions are, When the similarity between the above pose information is greater than a preset first threshold, Electronic devices.

15. In paragraph 13, The above at least one characteristic includes pose information and depth information, and The above-mentioned preset conditions are, The similarity between the above pose information is greater than a preset first threshold, and In the case where the similarity between the above depth information is greater than a preset second threshold, Electronic devices.

Images