Data processing method and data processing device using the same

Abstract

The present invention relates to a data processing method and a data processing device using the same, the data processing method comprising the following steps: acquiring a scanned image representing a target body; aligning the scanned image to at least one pre-formed inactivated cluster; and selectively merging an activated cluster containing the scanned image and the inactivated cluster according to whether alignment is achieved in the step of aligning to the inactivated cluster.

Description

Technical Field

[0001] The present invention relates to a data processing method and a data processing device using the same. Background Art

[0002] Recently, development of three-dimensional scanners for use in measuring and analyzing narrow areas such as the inside of the oral cavity has been actively carried out. The three-dimensional scanner is constructed in a handheld form so that a user (typically a dentist) can easily scan a narrow area.

[0003] When using a three-dimensional scanner, the user continuously scans the patient's oral cavity and finally obtains a three-dimensional model representing the target body (the inside of the patient's oral cavity, or a plaster model that imitates the inside of the oral cavity). The three-dimensional model generated by real-time modeling during the scanning process of the user using the three-dimensional scanner can be aligned by continuously acquired image data. For example, when three image data (first image data, second image data, and third image data) are continuously acquired, the second image data can be aligned with the first image data, and the third image data can be aligned with the second image data, thereby generating a three-dimensional model.

[0004] However, in the prior art, if the alignment between the image data fails once, the 3D modeling process will not be performed after the alignment process fails. The user should be careful to re-scan the target object from the part where the alignment failed to ensure that the alignment is performed normally. The scanning proficiency of the users varies. The user may have dozens (for example, 10) or more alignment failures during the process of scanning a palate. The user will experience inconvenience in re-scanning the target object from the part where the alignment failed and the scanning time will also increase.

[0005] Furthermore, the user usually looks at the target object, and even if the alignment failure is fed back to the display screen, it is actually impossible to alternately check the target object and the display screen. Furthermore, in the process of the user alternately checking the target object and the display screen, the following problem occurs, that is, the user's concentration may be dispersed, which may increase the scanning time required to complete the three-dimensional model, and the purpose of developing a three-dimensional scanner to quickly obtain a highly completed three-dimensional model is forgotten.

[0006] Prior art literature

[0007] Patent Literature

[0008] Patent document 1: Korean Patent Publication No. 10-2020-0115580 (published on October 7, 2020). Summary of the invention

[0009] In order to solve the above problems, the present invention provides a data processing method, which aligns a scanned image obtained during scanning of a target object with a pre-formed inactivated cluster, and merges multiple clusters through the above alignment, thereby finally generating a three-dimensional model.

[0010] Furthermore, the present invention provides a data processing device that executes the data processing method of the present invention under the action of a plurality of organically connected structural elements, thereby ultimately generating a three-dimensional model.

[0011] The technical problems of the present invention are not limited to the technical problems described above, and ordinary technicians in the technical field to which the present invention belongs can clearly understand other technical problems not mentioned through the following description.

[0012] In order to solve the above problems, the data processing method of the present invention includes the following steps: obtaining a scanned image representing a target body; aligning the above scanned image to at least one pre-formed inactivated cluster; and selectively merging the activated cluster containing the above scanned image and the above inactivated cluster according to whether alignment is achieved in the step of aligning to the above inactivated cluster.

[0013] Furthermore, the present invention may further include the following step, namely, aligning the scanned image with the previous scanned image included in the activated cluster, and the step of aligning with the inactivated cluster and the step of aligning with the previous scanned image may be performed simultaneously or sequentially.

[0014] Furthermore, in the step of aligning with the inactive cluster, feature points of the scanned image and feature points of the inactive cluster may be aligned.

[0015] Furthermore, in the above-mentioned selective merging step, when the above-mentioned scanned image is aligned with the above-mentioned previous scanned image and the above-mentioned scanned image is aligned with the above-mentioned inactivated cluster, the above-mentioned inactivated cluster and the above-mentioned activated cluster can be merged, and multiple scanned images included in the above-mentioned inactivated cluster can be included in the above-mentioned activated cluster.

[0016] Furthermore, when the scanned image is aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the scanned image may be included in the active cluster.

[0017] Furthermore, when the scanned image is not aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the active cluster may be inactive and a new active cluster may be generated and the scanned image may be included in the new active cluster.

[0018] Furthermore, when the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the scanned image may be included in the inactivated cluster.

[0019] Furthermore, when the scanned image is included in the inactivated cluster, the inactivated cluster may be activated, so that the activated cluster not including the scanned image is inactivated.

[0020] Furthermore, the above-mentioned scanned image, the above-mentioned activated cluster and the above-mentioned inactivated cluster can be displayed on a user interface screen. The above-mentioned user interface screen has multiple areas, including a main area occupying a specified area and a sub-area occupying an area smaller than the above-mentioned main area. The above-mentioned scanned image and the above-mentioned activated cluster are displayed in the above-mentioned main area. When the above-mentioned scanned image is included in the above-mentioned inactivated cluster, the above-mentioned inactivated cluster is activated and displayed in the above-mentioned main area, so that the above-mentioned activated cluster that does not include the above-mentioned scanned image is inactivated and displayed in the above-mentioned sub-area.

[0021] On the other hand, the data processing device of the present invention includes: a scanner for scanning a target object; a control unit for aligning a scanned image obtained by scanning the above-mentioned target object to at least one pre-formed inactivated cluster, and selectively merging an activated cluster including the above-mentioned scanned image and the above-mentioned inactivated cluster according to whether the above-mentioned alignment is performed; and a display unit for displaying the above-mentioned scanned image, the above-mentioned inactivated cluster and at least one of the above-mentioned activated clusters.

[0022] Furthermore, the control unit may include: a scan image generating unit, which generates the scan image based on the target object scanned by the scanner; an alignment unit, which aligns the scan image with a previous scan image included in the activated cluster, or aligns the scan image with the inactivated cluster; and a cluster management unit, which generates and manages the inactivated cluster and the activated cluster according to the alignment result of the alignment unit.

[0023] Furthermore, the alignment unit may simultaneously or sequentially perform a process of aligning the scanned image with the previously scanned image and a process of aligning the scanned image with the inactivated cluster.

[0024] Furthermore, when the scanned image is aligned with the previous scanned image and the scanned image is aligned with the inactive cluster, the cluster management unit may merge the inactive cluster into the active cluster and include the plurality of scanned images included in the inactive cluster into the active cluster.

[0025] Furthermore, when the scanned image is aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the cluster management unit may include the scanned image in the active cluster.

[0026] Furthermore, when the scanned image is not aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the cluster management unit may make the active cluster inactive and generate a new active cluster, and include the scanned image in the new active cluster.

[0027] Furthermore, when the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactive cluster, the cluster management unit may include the scanned image in the inactive cluster.

[0028] Furthermore, the cluster management unit may convert an active cluster that is not aligned with the scanned image into the inactive cluster, and convert an inactive cluster that is aligned with the scanned image into the active cluster.

[0029] Furthermore, the display unit can display a user interface having multiple areas, including a main area occupying a specified area and a sub-area occupying an area smaller than the main area, the scanned image and the activated cluster are displayed in the main area, and when the scanned image is included in the inactivated cluster, the inactivated cluster is activated and displayed in the main area, making the activated cluster inactivated and displayed in the sub-area.

[0030] The data processing method and the data processing device using the same according to the present invention have the following advantages, namely, the number of generated clusters is minimized and the scanning process is performed, thereby saving resources used for generating clusters, simplifying the calculation process, and maximizing the scanning speed.

[0031] Furthermore, the present invention has the following advantages, namely, even if the alignment between scanned images fails, data can be distinguished in cluster units and the scanning process can be continued, and the user can perform scanning regardless of the success or failure of the alignment, thereby improving the user's scanning freedom. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 The data processing method of the present invention is a flowchart.

[0033] Figure 2 Used to illustrate the process of obtaining a scanned image representing a target object.

[0034] Figure 3 Used to illustrate the alignment process between scanned images and clusters.

[0035] Figure 4 Used to illustrate feature points used as a medium for aligning scanned images with clusters.

[0036] Figure 5 It is used to explain the detailed process of step S130.

[0037] Figures 6 to 10 It is used to illustrate the process of processing data based on the success or failure of scanned image alignment and cluster alignment in the data processing method of the present invention.

[0038] Fig.11 It is used to illustrate the undo process taking the third cluster as an example.

[0039] Fig.12 It is used to illustrate the process of aligning the scanned image and the inactivated cluster and converting the image in the data processing method of the present invention.

[0040] Fig.13 and Fig.14 Used to illustrate the screen conversion process between the activated cluster and the inactivated cluster.

[0041] Fig.15 It is a structural diagram of the data processing device of the present invention.

[0042] Description of reference numerals:

[0043] 100: Scan image 200: Cluster

[0044] 500: User interface screen 510: Main area

[0045] 520: Sub-area

[0046] 900: Data processing device DETAILED DESCRIPTION

[0047] Hereinafter, some embodiments of the present invention will be described in detail with reference to the exemplary drawings. In the process of assigning reference numerals to the structural elements of each drawing, it should be noted that the same reference numerals are assigned to the same structural elements as much as possible even if they are shown in different drawings. In addition, in the process of describing the embodiments of the present invention, if it is determined that the specific description of the relevant known structure or function may hinder the understanding of the embodiments of the present invention, the detailed description thereof will be omitted.

[0048] In the process of describing the structural elements of the embodiments of the present invention, terms such as "first", "second", "A", "B", "(a)", "(b)", etc. may be used. Such terms are only used to distinguish one structural element from another structural element, and the nature, order or sequence of the corresponding structural elements are not limited by the above terms. In addition, unless otherwise defined, the meanings of all terms used herein, including technical terms or scientific terms, are the same as those generally understood by ordinary technicians in the technical field to which the present invention belongs. Common terms such as terms defined in commonly used dictionaries should be interpreted as being consistent with the contextual meaning of the relevant technology, and should not be interpreted as idealized or overly formalized meanings unless clearly defined in the present invention.

[0049] Figure 1 The data processing method of the present invention is a flowchart.

[0050] Reference Figure 1 , the data processing method of the present invention includes step S110, that is, obtaining a scanned image representing the target body. The step S110 of obtaining the scanned image may refer to obtaining image data representing the target body by receiving light reflected from the surface of the target body through the scanner. The camera built into the scanner receives light entering the scanner, and obtains the scanned image based on the received light. In this case, the scanned image may be two-dimensional image data, or may refer to three-dimensional data fragments. Multiple scanned images obtained by the scanner may be stored according to the order of acquisition. The structure for storing the scanned images may use a known storage device, and the above-mentioned storage device may be a hard disk drive, a solid state drive (SSD: Solid State Drive) and a mobile storage device (Universal Serial Bus Flash Drive (USB Flash drive) etc.), and the type of storage device is not limited to the example disclosed in the present invention.

[0051] Figure 2 1 is used to illustrate the process of obtaining a scanned image 100 representing the target object O.

[0052] Reference Figure 2 , the scanned image 100 can obtain a plurality of images according to the direction in which the scanner moves and the angle at which the scanner points to the target object O. For example, Figure 2 As shown, the first scanning image 101, the second scanning image 102, the third scanning image 103, the fourth scanning image 104, the fifth scanning image 105, the sixth scanning image 106, the seventh scanning image 107, the eighth scanning image 108 and the ninth scanning image 109 can be obtained by scanning the target object O. In this case, the user does not need to scan the target object O in sequence from one direction to another, and can scan the target object O in a way that is convenient for the user.

[0053] Reference Figure 1 and Figure 2 , the multiple scanned images 100 acquired in the data processing method of the present invention perform an alignment process for connecting to each other. In this case, the alignment performed between the scanned images 100 is referred to as "scanned image alignment". Scanned image alignment may refer to aligning a scanned image acquired at a specific time point with a scanned image acquired at a previous time point (i.e., a previous scanned image). The process of performing alignment between the scanned images 100 may use the Iteration Closest Point (ICP) method, but is not limited thereto.

[0054] When performing alignment between multiple scanned images 100, if the overlapping overlap intervals between the multiple scanned images 100 do not exist or are insufficient (do not reach the standard value), there may be a portion where the scanned images 100 fail to align. Figure 2 As shown, there is no overlapping superposition interval between the third scan picture 103 and the fourth scan picture 104, and between the sixth scan picture 106 and the seventh scan picture 107, so the scan pictures cannot be aligned with each other. In this case, the first scan picture 101, the second scan picture 102, and the third scan picture 103 can be classified into one group and stored, the fourth scan picture 104, the fifth scan picture 105, and the sixth scan picture 106 can be classified into another group and stored, and the seventh scan picture 107, the eighth scan picture 108, and the ninth scan picture 109 can be classified into another group and stored. In this case, the group that stores multiple scan pictures 100 respectively is called a cluster 200, and the cluster 200 can be a collection of scan pictures 100 including at least one scan picture 100. That is, as Figure 2 As shown, when the concept of cluster 200 is applied, the first scanned image 101, the second scanned image 102, and the third scanned image 103 can be classified into the first cluster and stored, the fourth scanned image 104, the fifth scanned image 105, and the sixth scanned image 106 can be classified into the second cluster and stored, and the seventh scanned image 107, the eighth scanned image 108, and the ninth scanned image 109 can be classified into the third cluster and stored. A new cluster 200 can be generated or different clusters 200 can be merged according to whether the scanned images 100 obtained by scanning the target body are aligned or not.

[0055] The following describes an inactive cluster and an active cluster.

[0056] Figure 3 Used to illustrate the alignment process between scanned images and clusters.

[0057] Reference Figure 1 and Figure 3 , the acquired multiple scanned images 100 may be stored in at least one cluster 200. In this case, during the scanning process displayed in real time by a display unit such as a display screen, the currently acquired scanned image 100 and the cluster 200 including the scanned image 100 may be mainly displayed, and the previously acquired multiple scanned images 100 and / or the multiple clusters 200 including the same may be additionally displayed. Figure 3As shown, the currently acquired scan picture 100 and the third cluster 203 including it are mainly displayed in real time in the center of the user interface screen 500, and the pre-formed first cluster 201 and second cluster 202 are additionally displayed on one side of the user interface screen 500. In this case, the mainly displayed cluster 200 can be called an activated cluster, and the additionally displayed cluster 200 can be called an inactivated cluster. The activated cluster includes the previous scan pictures, and the previous scan pictures will become the objects of scanning picture alignment for the new scan pictures 100 acquired during the scanning process. Exemplarily, in the case of the latest acquisition of the eighth scan picture included in the third cluster 203, the above-mentioned eighth scan picture can become the previous scan picture, the third cluster 203 becomes the activated cluster, and the user can acquire a new ninth scan picture during the scanning process. The inactivated cluster will become the object of cluster alignment for the scan pictures 100 acquired during the scanning process.

[0058] Figure 4 It is used to illustrate the feature points used as a medium for aligning the scanned image with the cluster. In more detail, Figure 4 Part (a) of Figure 4 Part (b) of Figure 4 Part (c) is used to explain the Figure 4 Part (a) to Figure 4 The process of comparing the scanned image feature points SP of the scanned image 100 acquired during the scanning process with the cluster feature points CP of the cluster 200 is performed up to part (c).

[0059] Reference Figure 1 , Figure 3 and Figure 4 The data processing method of the present invention includes step S121, aligning the scanned image 100 to at least one pre-formed inactive cluster. The step of aligning the scanned image 100 to the pre-formed inactive cluster is called cluster alignment.

[0060] When cluster alignment is performed, a medium called a feature point may be used to align the inactive cluster and the scanned image 100. The scanned image 100 and the cluster 200 each have at least one feature point, and the feature point may have color information, curvature information, shape information, etc., respectively displayed at a specific position of each data. The feature point may be obtained by sampling each data (the cluster and the scanned image formed in the form of a point cloud).

[0061] Exemplarily, the cluster 200 may have a plurality of cluster feature points CP, and the scanned image 100 may have a plurality of scanned image feature points SP. When aligning the inactivated cluster and the scanned image 100, the information possessed by the cluster feature points CP and the information possessed by the scanned image feature points SP may be compared with each other, thereby performing alignment between the inactivated cluster and the scanned image 100. As described above, the method of performing alignment using feature points as a medium may be included in the category of initial alignment. Additionally, the initial alignment is slightly different from the above method and may also be performed by aligning the center point of the scanned image 100 to a specified position of the cluster 200.

[0062] On the other hand, the cluster alignment method may be slightly different from the scanned image alignment method. For example, the cluster alignment may be performed by performing the above-mentioned initial alignment, using the plurality of feature points SP and CP respectively possessed by the scanned image 100 and the cluster 200 as a medium to perform the alignment between the scanned image 100 and the cluster 200, and then, the precise alignment between the scanned image 100 and the cluster 200 may be performed by using the nearest point algorithm. In contrast, the scanned image alignment may only use the nearest point algorithm to perform the alignment between the plurality of scanned images 100.

[0063] exist Figure 4 Part (a) of Figure 4 Part (b) of Figure 4 The cluster 200 formed on the left side of part (c) is an inactive cluster. Figure 4 Part (a) of Figure 4 Part (b) of Figure 4 The scanned image 100 formed on the right side of part (c) is acquired as the scanning process proceeds. As the scanned image 100 is acquired, the scanned image feature points SP can be acquired together by sampling the scanned image 100, and cluster alignment can be performed by comparing multiple scanned image feature points SP with multiple cluster feature points CP. In the process of performing cluster alignment, cluster alignment is no longer performed on the scanned image 100 that is acquired during the scanning process and aligned with the pre-formed inactive cluster and attempted cluster.

[0064] In the prior art, in order to attempt to align clusters 200 with clusters 200, it is necessary to continuously detect the overlapping parts (overlapping areas) between clusters 200. In more detail, in the prior art, in order to compare one cluster with another cluster, the characteristics of the cluster as a whole are compared with those of the other cluster. In this process, as the number of scanned images gathered in each cluster increases, the size of the cluster also increases, and in the process of performing inter-cluster comparisons, the problem of excessive increase in computational load arises. In contrast, in the present invention, after performing alignment between scanned images 100 and clusters 200, scanned images 100 included in any cluster 200 are not cluster aligned with other clusters 200. As a result, the time required to obtain a three-dimensional model can be ultimately shortened, and the load required in the calculation can also be reduced.

[0065] like Figure 4 As shown, the cluster 200 has 12 cluster feature points CP, and the scanned image 100 has 4 scanned image feature points SP, but it is not limited to this number. The cluster feature points CP and the scanned image feature points SP can be configured to an appropriate number to achieve fast alignment speed and accurate data alignment.

[0066] According to the above content, the data processing method of the present invention may include two alignment steps S120. One may be a step S121 of aligning the scanned image 100 obtained during the scanning process with a pre-formed inactive cluster, and the other may be a step S122 of aligning the scanned image 100 obtained during the scanning process with a previous scanned image contained in an active cluster. That is, step S121 may be a step of a process of performing cluster alignment, and step S122 may be a step of a process of performing scanned image alignment. Step S121 of aligning with an inactive cluster and step S122 of aligning with a previous scanned image may be performed simultaneously or sequentially. Exemplarily, as Figure 1 As shown, the acquired scanned image 100 may attempt to perform cluster alignment with multiple inactive clusters by attempting to perform scanned image alignment with the previous scanned image. Alternatively, the acquired scanned image 100 may first attempt to perform scanned image alignment with the previous scanned image, and then attempt to perform cluster alignment with multiple inactive clusters. Conversely, the acquired scanned image 100 may also first attempt to perform cluster alignment with the inactive clusters, and then attempt to perform scanned image alignment with the previous scanned image. That is, as Figure 1 As shown, although step S121 and step S122 are executed in parallel in the figure, it is not limited to this. Step S121 can be executed first and then step S122, or step S122 can be executed first and then step S121.

[0067] The selective merging step S130 in the data processing method of the present invention is described in detail below.

[0068] Figure 5 For explaining the detailed process of step S130, Figures 6 to 10 It is used to illustrate the process of processing data based on the success or failure of scan image alignment and cluster alignment in the data processing method of the present invention. Figures 6 to 10 When describing the present invention, the clusters that are processed by shading may be referred to as activated clusters, and the clusters that are not processed by shading may be referred to as inactivated clusters.

[0069] Reference Figure 1 and Figure 5 The data processing method of the present invention includes step S130, that is, selectively merging the activated cluster and the inactivated cluster containing the scanned image 100 according to whether the alignment is achieved in step S121 of aligning to the inactivated cluster. Thus, the activated cluster and the inactivated cluster can form a merged cluster, and when the process of merging multiple clusters 200 is repeated, a complete three-dimensional model can be finally obtained. Selectively merging the activated cluster and the inactivated cluster can refer to merging the activated cluster and the inactivated cluster under specific conditions according to the success or failure of the cluster alignment and the scanned image alignment.

[0070] The selectively merged step S130 is described in detail. The selectively merged step S130 may include at least two judgment steps. In the selectively merged step S130, it can be judged whether the cluster alignment and the scanned image alignment are successful respectively. Exemplarily, in the selectively merged step S130, it can be judged whether the scanned image alignment is successful, and then it can be judged whether the cluster alignment is successful. However, it is not limited to this order, and the selectively merged step S130 can also judge whether the scanned image alignment is successful after judging whether the cluster alignment is successful. In addition, the selectively merged step S130 can judge the success or failure of the scanned image alignment and the cluster alignment in parallel.

[0071] Reference Figure 6, the above judgment steps are described in more detail by an example. Exemplarily, eight scanned images 100 are first obtained, and each scanned image 100 is respectively included in three clusters 200. In this case, the first cluster 201 may include the first scanned image 101, the second scanned image 102, and the third scanned image 103, the second cluster 202 may include the fourth scanned image 104, the fifth scanned image 105, and the sixth scanned image 106, and the third cluster 203 may include the seventh scanned image 107 and the eighth scanned image 108. The user can obtain the ninth scanned image 109 after the eighth scanned image 108 by scanning the target body. In this case, the previous scanned image is the eighth scanned image 108, the third cluster 203 including the previous scanned image is an activated cluster, and the first cluster 201 and the second cluster 202 not including the previous scanned image are inactivated clusters.

[0072] The ninth scan picture 109 may attempt to perform scan picture alignment with the eighth scan picture 108 as the previous scan picture, and may attempt to perform cluster alignment with the first cluster 201 and the second cluster 202 as the inactive clusters. In this case, the process of performing scan picture alignment and cluster alignment is as described above.

[0073] Reference Figure 5 and Figure 7 , in the selective merging step S130, when the acquired scan picture (the ninth scan picture) is aligned with the previous scan picture (the eighth scan picture) and the scan picture (the ninth scan picture) is aligned with the inactivated cluster (any one of the first cluster and the second cluster), the scan picture and the cluster-aligned inactivated cluster can be merged with the activated cluster. Exemplarily, it is assumed that the ninth scan picture 109 is successfully cluster-aligned in the first cluster 201. In this case, the first cluster 201, which is an inactivated cluster, can be merged with the third cluster 203, which is an activated cluster, to form a cluster 210. And, the merged cluster 210 may include all scan pictures 100 included in the first cluster 201 and the third cluster 203. That is, a plurality of scan pictures 101, 102, 103 included in the first cluster 201, which is an inactivated cluster, may be included in the third cluster 203, which is an activated cluster. The merged cluster 210 may include the first scanned picture 101 , the second scanned picture 102 , the third scanned picture 103 , the seventh scanned picture 107 , the eighth scanned picture 108 , and the ninth scanned picture 109 , and the merged cluster 210 may serve as an active cluster.

[0074] And, refer to Figure 5 and Figure 8, when the ninth scan picture 109 performs scan picture alignment with the eighth scan picture 108 as the previous scan picture but does not perform cluster alignment with the first cluster 201 and the second cluster 202 as the inactive cluster, the ninth scan picture 109 may be included in the third cluster 203 as the active cluster. Then, when the subsequent tenth scan picture (not shown) is acquired by performing the scanning process, the ninth scan picture 109 may be used as the previous scan picture, and the third cluster 203 including the ninth scan picture 109 may be used as the active cluster. The tenth scan picture may attempt to perform scan picture alignment with the ninth scan picture 109, and attempt to align with the clusters of the first cluster 201 and the second cluster 202.

[0075] And, refer to Figure 5 and Fig. 9 , when the ninth scan picture 109 is not aligned with the eighth scan picture 108 as the previous scan picture and the ninth scan picture 109 is not cluster aligned with any of the first cluster 201 and the second cluster 202 as the inactive clusters, it can be determined that the ninth scan picture 109 is not aligned with any pre-formed inactive clusters and active clusters 201, 202, 203. In this case, the ninth scan picture 109 can be included in the newly generated new active cluster, which is not included in the pre-formed inactive clusters and active clusters 201, 202, 203. Exemplarily, the fourth cluster 204 as a new active cluster is generated and the ninth scan picture 109 will be included in the fourth cluster 204. Then, when the subsequent tenth scan picture is obtained by performing a scanning process, the ninth scan picture 109 can be used as the previous scan picture, and the fourth cluster 204 as a new active cluster including the ninth scan picture 109 can be used as an active cluster. In this case, the third cluster 203 as an active cluster can be converted to an inactive cluster. The tenth scanned image may attempt to perform scanned image alignment with the ninth scanned image 109 , and may attempt to perform cluster alignment with the first cluster 201 , the second cluster 202 , and the third cluster 203 .

[0076] And, refer to Figure 5 and Fig.10, when the ninth scan picture 109 is not scanned image aligned with the eighth scan picture 108 as the previous scan picture and the ninth scan picture 109 is aligned with any one of the first cluster 201 and the second cluster 202 as the inactivated cluster, the ninth scan picture 109 can be included in the inactivated cluster capable of cluster alignment. Exemplarily, when the ninth scan picture 109 is cluster aligned with the first cluster 201 as the inactivated cluster, the ninth scan picture 109 is included in the first cluster 201. And, when the subsequent tenth scan picture is acquired by performing the subsequent scanning process, the ninth scan picture 109 can be used as the previous scan picture, and therefore, the first cluster 201 including the ninth scan picture 109 can be used as the activated cluster. That is, when the ninth scan picture 109 is included in the first cluster 201 as the inactivated cluster, the activated cluster and the inactivated cluster can be converted so that the first cluster 201 as the inactivated cluster is activated and the third cluster 203 as the activated cluster will not be activated. This has the following advantages, namely, the user can easily confirm the cluster 200 including the currently acquired scan image 100, thereby being able to grasp the position of the target body being scanned by the user. Furthermore, it has the following advantages, namely, scanning can be performed freely regardless of the success or failure of alignment. Furthermore, it has the following advantages, namely, even if the scan image alignment fails, as long as the cluster alignment is successful, unnecessary new clusters will not be generated, but inactivated clusters will be activated for use, thereby minimizing the number of clusters considered during the alignment process to improve the operation speed.

[0077] Fig.11 It is used to illustrate the undo process taking the third cluster 203 as an example.

[0078] Reference Fig.11 , the user can cancel the acquisition of the scanned image 100. For example, referring to Fig.11 In part (a) of FIG. 1 , it is assumed that the third cluster 203 as the active cluster includes the seventh scan image 107, the eighth scan image 108, and the ninth scan image 109. The user can delete the scan images in reverse chronological order starting from the most recently acquired scan image 100 by selecting an undo execution button (not shown) formed on the user interface screen. Fig.11 In part (b) of FIG. 1 , when the user selects the cancel execution button, the ninth scan image 109 obtained most recently can be deleted. Fig.11 In part (c), when the user reselects the cancel execution button, the eighth scan image 108 acquired before the deleted ninth scan image 109 can be deleted. Fig.11In part (d), when the user reselects the undo execution button, the seventh scan image 107 acquired before the deleted eighth scan image 108 can be deleted. In this case, since the third cluster 203 does not include any scan images 100, the third cluster 203 can also be deleted. When the third cluster 203 is deleted, the cluster that was the active cluster before the third cluster 203 became the active cluster can be reactivated and displayed on the user interface screen. Then, when the undo execution process is further performed, the scan images 100 acquired most recently can be deleted in the activated cluster in reverse order of the time of acquisition.

[0079] On the other hand, as described above, the process of deleting the acquired scanned image 100 may be performed in units of a single image, but a plurality of scanned images 100 may also be deleted at the same time as required.

[0080] The following describes in more detail the process of the user interface screen changing when the scanned image 100 is aligned with the inactive cluster so that the inactive cluster and the active cluster are switched.

[0081] Fig.12 It is used to illustrate the process of aligning the scanned image 100 and the inactivated cluster and converting the image in the data processing method of the present invention. Fig.13 and Fig.14 Used to illustrate the screen conversion process between the activated cluster and the inactivated cluster.

[0082] Reference Fig.12 and Fig.13 , by scanning the target O, a plurality of scanned images 100 representing the target O are obtained. Exemplarily, the user can obtain a first scanned image 101, a second scanned image 102, a third scanned image 103, a fourth scanned image 104, a fifth scanned image 105, a sixth scanned image 106, and a seventh scanned image 107 by scanning the target O. Fig.12 As shown in the content, the first scanning picture 101, the second scanning picture 102 and the third scanning picture 103 may be included in the first cluster 201, and the fourth scanning picture 104, the fifth scanning picture 105 and the sixth scanning picture 106 may be included in the second cluster 202. Before the user obtains the seventh scanning picture 107 through the scanning process, the second cluster 202 including the sixth scanning picture 106 as the previous scanning picture may be used as an active cluster.

[0083] In this case, the scanned image 100, the activated cluster, and the inactivated cluster may be displayed on a user interface screen 500 having a plurality of regions. Exemplarily, the acquired scanned image 100 and the activated cluster are mainly displayed on the user interface screen 500, and the inactivated cluster may be additionally displayed on the user interface screen 500. In more detail, the user interface screen 500 may include a main area 510 occupying a prescribed area and a sub-area 520 occupying an area smaller than the main area 510. In this case, the main area 510 may be formed to include the center of the user interface screen 500 so as to be visually conveniently displayed to the user, and the sub-area 520 may be formed on one side of the user interface screen 500. In the user interface screen 500, in order to display the scanning process in real time together with the scanned image 100 currently acquired by the user, the acquired scanned image 100 and the second cluster 202 as the activated cluster may be displayed in the main area 510. The first cluster 201 as the inactivated cluster may be displayed in the sub-area 520, and the user may obtain scanning information such as the number of clusters 200 generated by the main area 510 and the sub-area 520, and the position of the target body O currently being scanned.

[0084] On the other hand, refer to Figure 12 to Figure 14 , scan picture alignment can be performed by acquiring the seventh scan picture 107 and the sixth scan picture 106 as the previous scan picture, and cluster alignment can be performed with the first cluster 201 as the inactivated cluster. In this case, when the scan picture alignment fails and the cluster alignment succeeds, the seventh scan picture 107 can be included in the first cluster 201 as the inactivated cluster. And, the first cluster 201 can be converted to an activated cluster (the inactivated cluster is activated), and the second cluster 202 can be converted to an inactivated cluster (the activated cluster is not activated). Fig.14 , the first cluster 201, which is an inactive cluster including a scanned image (more specifically, the seventh scanned image), is activated and displayed in the main area 510 of the user interface screen 500. At the same time, the second cluster 202, which is an active cluster not including a scanned image, is not activated and is displayed in a thumbnail form in the sub-area 520 of the user interface screen 500. As described above, there is an advantage that, by switching the area displayed according to the switching between the active cluster and the inactive cluster, the user can easily confirm the position of the currently scanned target body O, and the convenience of the user is increased.

[0085] On the other hand, as described above, scan image alignment and cluster alignment can be limited to specific areas for execution. Exemplarily, when scan image alignment and cluster alignment are performed, the area where the alignment is performed can be limited to the scan image representing the target body and the area representing the teeth in the cluster, and the gums, tongue, soft tissue, noise, etc. can be excluded from the aligned objects. The process of distinguishing the area representing the teeth from the remaining areas can be performed by using artificial intelligence (AI) based on image learning, and the process of eliminating noise can be performed by using known noise filtering techniques. The data processing method of the present invention has the following advantages, namely, by limiting the teeth as the objects of alignment, the scan image alignment and / or cluster alignment can be performed quickly and accurately. In addition, the merging speed between clusters can be increased, and therefore, the time required to finally obtain a three-dimensional model with a high degree of completion can be shortened.

[0086] Hereinafter, a data processing device of the present invention using the above-mentioned data processing method will be described. In the process of describing the data processing device, the contents overlapping with the above-mentioned contents will be briefly described or omitted.

[0087] Fig.15 It is a structural diagram of the data processing device 900 of the present invention.

[0088] Reference Fig.15 The data processing device 900 of the present invention includes a scanner 910, a control unit 920 and a display unit 930. The scanner 910 can scan a target object. The scanner 910 can have at least one camera (not shown) built in to receive light therein, and light reflected from the surface of the target object can be introduced into the interior of the scanner 910 and received by the lens of the camera. Based on the light received by the camera, a scanned image representing the target object can be generated. The scanner 910 can be a desktop scanner that places the target object on a tray and rotates or moves the target object, or it can be a handheld scanner that a user holds and points at the target object and performs a scanning process.

[0089] The control unit 920 may be electrically connected to the scanner 910 and generate a scanned image based on the light received by the scanner 910. Furthermore, the scanned image is aligned with a pre-formed cluster (more specifically, an inactive cluster), and the active cluster and the inactive cluster may be selectively merged according to the alignment.

[0090] The detailed structure of the control unit 920 is described in more detail below. The control unit 920 includes a scanned image generation unit 921 to generate a scanned image based on the target object scanned by the scanner 910. The scanned image generation unit 921 can generate a scanned image by receiving light received by the camera of the scanner 910. The scanned image can be two-dimensional image data or three-dimensional data fragments. Although the scanned image generation unit 921 is shown as a structure of the control unit 920 in the drawings, it is not limited thereto and can be formed as a structure of the scanner 910 as needed.

[0091] Furthermore, the control unit 920 includes an alignment unit 922. The alignment unit 922 can align the scanned image generated based on the scanned image generation unit 921 with the previous scanned image included in the activated cluster (scanned image alignment), and can align the scanned image with the inactivated cluster (cluster alignment). Exemplarily, the process of aligning the scanned image generated and acquired by the scanned image generation unit 921 with the previous scanned image included in the activated cluster can use the nearest point algorithm alignment method, and the process of aligning the scanned image with the inactivated cluster can use the initial alignment method. Scanned image alignment, cluster alignment, multiple scanned image feature points possessed by the scanned image and multiple cluster feature points possessed by the cluster can be performed by comparing with each other, and the process of performing alignment using feature points is the same as described above.

[0092] Furthermore, the alignment unit 922 may simultaneously or sequentially perform the process of aligning the scanned image with the previously scanned image and the process of aligning the scanned image with the inactivated cluster. That is, the alignment unit 922 may perform the above-mentioned steps S121 and S122. In this case, the execution order of steps S121 and S122 may be simultaneous or sequential, and step S121 may be performed first, followed by step S122, or step S122 may be performed first, followed by step S121.

[0093] Furthermore, the control unit 920 includes a cluster management unit 923. The cluster management unit 923 performs the function of generating and managing clusters. Exemplarily, the cluster management unit 923 may merge multiple clusters according to the alignment result of the alignment unit 922. Furthermore, the cluster management unit 923 may generate a new activation cluster according to the alignment result of the alignment unit 922, and manage the inclusion of the scanned image in the above-mentioned new activation cluster. Furthermore, the cluster management unit 923 may activate an inactive cluster, or make an active cluster inactive.

[0094] The following describes the detailed operation of the cluster management unit 923 .

[0095] In the alignment unit 922, when the scanned image is aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the cluster management unit 923 can merge the activated cluster and the inactivated cluster including the previous scanned image. In this case, the cluster management unit 923 can perform inter-cluster merging in a manner of merging the inactivated cluster with the activated cluster, and the merged cluster includes a plurality of scanned images included in the activated cluster and a plurality of scanned images included in the inactivated cluster. This has the following advantages, that is, according to the scanned image alignment and cluster alignment of the scanned images, the number of clusters constituting the three-dimensional model can be reduced during the scanning process, and a complete three-dimensional model can be obtained more quickly.

[0096] Furthermore, in the alignment unit 922, when the scanned image is aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the cluster management unit 923 may include the scanned image in the active cluster including the previous scanned image. In the subsequent scanning process, the active cluster including the scanned image continues to be activated.

[0097] Furthermore, in the alignment unit 922, when the scanned image is not aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the cluster management unit 923 may control the generation of a new group including the scanned image. In more detail, the cluster management unit 923 inactivates the active cluster and generates a new active cluster to include the scanned image in the new active cluster. Therefore, in the subsequent scanning process, the new active cluster can be used as the active cluster.

[0098] Furthermore, in the alignment unit 922, when the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the cluster management unit 923 controls the scanned image to be included in the inactivated cluster. In this case, the cluster management unit 923 can include the acquired scanned image in the inactivated cluster, and activate the inactivated cluster containing the scanned image (convert the inactivated cluster into the activated cluster), and can make the activated cluster containing the previous scanned image (convert the activated cluster into the inactivated cluster) inactivated. The inactivated cluster becomes the object of cluster alignment of the newly acquired scanned image, and the activated cluster becomes the object of scanned image alignment, and the detailed description related thereto is the same as the above. According to the above process, the following advantages will be generated, namely, even if the scanned image alignment fails, as long as the cluster alignment is successful, the cluster management unit 923 will not generate unnecessary new clusters and can use them by activating the inactivated cluster, and the operation speed will be improved by minimizing the number of clusters considered in the alignment process.

[0099] On the other hand, the display unit 930 can visually display at least a part of the scanned image and the clusters in the above content to the user. In more detail, the display unit 930 can display at least one of the scanned image, the inactivated cluster, and the activated cluster. The user can confirm the position of the currently scanned target, the number of generated clusters, etc. by confirming the display unit 930. The display unit 930 can use a known visual display device. For example, the display unit 930 can be a display screen, a tablet, a touch screen, etc.

[0100] On the other hand, the display unit 930 displays a user interface having multiple regions by including a main region occupying a specified area and a sub-region occupying an area smaller than the main region. The display unit 930 can display an activated cluster and a scanned image in the main region of the user interface, and can display an inactivated cluster in the sub-region of the user interface.

[0101] Furthermore, when the display unit 930 includes the scanned image in the inactive cluster due to the control of the cluster management unit 923, the inactive cluster can be activated and displayed in the main area, and the active cluster can be inactivated, so that it can be displayed in the sub-area in the form of a thumbnail. This has the advantage that the user can easily confirm the position of the target object currently being scanned by cluster switching.

[0102] The above description is merely an exemplary description of the technical concept of the present invention. Anyone skilled in the art in the technical field to which the present invention belongs can make various modifications and variations without departing from the essential features of the present invention.

[0103] Therefore, the multiple embodiments disclosed in the present invention are used to illustrate the technical ideas of the present invention, rather than to limit the technical ideas of the present invention, and the scope of the technical ideas of the present invention is not limited to these embodiments. The protection scope of the present invention should be interpreted by the protection scope of the attached invention claims, and all technical ideas within the same scope should be interpreted as included in the protection scope of the present invention.

Claims

1. A data processing method, characterized in that: The steps include: Obtain a scanned image representing a target object; Aligning the scanned image to the at least one pre-formed inactive cluster by cluster alignment between the scanned image and the at least one pre-formed inactive cluster; Aligning the scanned image with a previous scanned image included in the activated cluster; and According to whether alignment is achieved in the step of aligning to the inactive cluster, selectively merging the active cluster including the scanned image aligned by scanned image alignment between the scanned image and the previous scanned image and the inactive cluster aligned by cluster alignment of the scanned image, The cluster is a group storing a plurality of the scanned images. The step of aligning with the inactive cluster comprises the following steps: aligning the scanned image feature points of the scanned image with the cluster feature points of the inactive cluster, The step of aligning the scanned image with the previously scanned image comprises the following steps: aligning the scanned image with the previously scanned image through an overlapping area, Wherein, when the scanned image is not aligned with the previously scanned image and the scanned image is aligned with the inactivated cluster, the inactivated cluster is activated as the activated cluster.

2. The data processing method according to claim 1, characterized in that: In the selective merging step, when the scanned image is aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the inactivated cluster is merged with the activated cluster, and a plurality of scanned images included in the inactivated cluster are included in the activated cluster.

3. The data processing method according to claim 1, characterized in that: When the scanned image is aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the scanned image is included in the active cluster.

4. The data processing method according to claim 1, characterized in that: When the scanned image is not aligned with the previous scanned image and the scanned image is not aligned with the inactivated cluster, the activated cluster is made inactivated and a new activated cluster is generated and the scanned image is included in the new activated cluster.

5. The data processing method according to claim 1, characterized in that: When the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the scanned image is included in the inactivated cluster.

6. The data processing method according to claim 5, characterized in that: When the scanned image is included in the inactivated cluster, the inactivated cluster is activated, and the activated cluster that does not include the scanned image is inactivated.

7. The data processing method according to claim 5, characterized in that: The scanned image, the activated cluster, and the inactivated cluster are displayed on a user interface screen, and the user interface screen has multiple areas, including a main area occupying a specified area and a sub-area occupying an area smaller than the main area. The scanned image and the activated cluster are displayed in the main area. When the scanned image is included in the inactivated cluster, the inactivated cluster is activated and displayed in the main area, and the activated cluster not including the scanned image is inactivated and displayed in the sub-area.

8. A data processing device, characterized in that: include: A scanner, used for scanning a target object; a control unit, aligning a scanned image obtained by scanning the target body with at least one pre-formed inactive cluster by cluster alignment between the scanned image and at least one pre-formed inactive cluster, and selectively merging an activated cluster including the scanned image aligned by the scanned image alignment between the scanned image and a previous scanned image and the inactive cluster aligned by the cluster alignment of the scanned image, depending on whether the alignment is performed; as well as A display unit displays at least one of the scanned image, the inactivated cluster, and the activated cluster. The control unit includes: an aligning unit configured to align the scanned image with a previous scanned image included in the activated cluster, or to align the scanned image with the inactivated cluster; and The cluster management unit generates and manages the inactivated cluster and the activated cluster according to the alignment result of the alignment unit. The cluster is a group storing a plurality of the scanned images. The alignment unit simultaneously or sequentially performs a process of aligning the scanned image with the previously scanned image and a process of aligning the scanned image with the inactivated cluster. The cluster alignment is performed by aligning the scanned image feature points of the scanned image with the cluster feature points of the inactivated cluster by the control unit. The scanned image alignment is performed by the control unit aligning the scanned image with the previously scanned image through the overlapping area. When the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactivated cluster, the control unit activates the inactivated cluster to the activated cluster.

9. The data processing device according to claim 8, characterized in that: The control unit also includes: The scanned image generating unit generates the scanned image based on the target object scanned by the scanner.

10. The data processing device according to claim 8, characterized in that: When the scanned image is aligned with the previous scanned image and the scanned image is aligned with the inactive cluster, the cluster management unit merges the inactive cluster into the active cluster and includes the plurality of scanned images included in the inactive cluster into the active cluster.

11. The data processing device according to claim 8, characterized in that: When the scanned image is aligned with the previous scanned image and the scanned image is not aligned with the inactive cluster, the cluster management unit includes the scanned image in the active cluster.

12. The data processing device according to claim 8, characterized in that: When the scan picture is not aligned with the previous scan picture and the scan picture is not aligned with the inactive cluster, the cluster management unit makes the active cluster inactive and generates a new active cluster, and includes the scan picture in the new active cluster.

13. The data processing device according to claim 8, characterized in that: When the scanned image is not aligned with the previous scanned image and the scanned image is aligned with the inactive cluster, the cluster management unit includes the scanned image in the inactive cluster.

14. The data processing device according to claim 13, characterized in that: The cluster management unit converts the active cluster that is not aligned with the scanned image into the inactive cluster, and converts the inactive cluster that is aligned with the scanned image into the active cluster.

15. The data processing device according to claim 13, characterized in that: The display unit displays a user interface, wherein the user interface has a plurality of regions, including a main region occupying a predetermined area and a sub-region occupying an area smaller than the main region. The scanned image and the activated cluster are displayed in the main area. When the scanned image is included in the inactivated cluster, the inactivated cluster is activated and displayed in the main area, and the activated cluster is inactivated and displayed in the sub-area.

Images