Image processing method and image processing apparatus using the same

Abstract

The image processing method according to the present invention includes a scanning step of acquiring image data by scanning an object including teeth, a step of determining caries in the teeth from the image data, and a step of displaying the image data of the determined caries.

Description

Technical Field

[0001] This disclosure relates to an image processing method and an image processing apparatus using the same. Background Technology

[0002] Tooth decay accounts for a significant proportion of dental care. Treatment for caries involves removing the decayed portion and filling it with amalgam, resin, or gold.

[0003] A dentist examines a patient's teeth and proposes a treatment plan appropriate for the patient's condition. However, dentists generally examine a patient's teeth with the naked eye, thus relying heavily on their subjective judgment. These judgments may vary depending on the treatment situation and may result in inaccurate assessments.

[0004] 3D scanning technology is applied across various industries, including measurement, inspection, reverse engineering, content generation, CAD / CAM, and medical devices. Its practicality is further expanding as scanning performance improves with advancements in computing technology. Particularly in the dental field, where precise measurements are required, the frequency of 3D scanning technology usage is increasing. Within the dental industry, there is a demand for methods that utilize 3D scanning technology to diagnose dental conditions, including the presence or absence of cavities.

[0005] In Korean Patent Registration No. 1574376, a light-emitting device is used to detect tooth decay. Light-emitting devices typically utilize fluorescence, and scanners, etc., should include a light-emitting device to emit fluorescence onto the teeth. Scanners that include a light-emitting device are relatively heavier and more expensive than those that do not. Therefore, there is a need for a method and system capable of expressing whether a tooth has tooth decay and the state of tooth decay without having a separate light-emitting device. Summary of the Invention

[0006] This disclosure provides an image processing method that prevents dentists from performing unnecessary procedures to examine a patient's teeth and enables the visual representation of carious parts of teeth through scanning.

[0007] Furthermore, this disclosure provides an image processing apparatus capable of visually representing a carious portion of a tooth by performing a series of processes using the image processing method described above.

[0008] The technical objectives to be achieved by this disclosure are not limited to those described above, and other technical objectives not described above can be clearly understood by those skilled in the art from the following description.

[0009] To achieve the above aspects, the image processing method according to this disclosure may include: a scanning step to acquire image data by scanning an object including teeth; a data analysis step to determine dental caries from the image data acquired in the scanning step; and a display step to display the dental caries analysis results in three dimensions.

[0010] The image processing apparatus according to this disclosure may have various elements, including: a scanning unit configured to acquire image data by scanning an object including teeth; a control unit configured to determine caries in the image data for data analysis; and a display unit configured to display the image data of the determined caries in three dimensions.

[0011] Its advantage is that, by using the image processing method and image processing apparatus according to the present disclosure, the process for determining the caries state of each tooth can be performed quickly.

[0012] Furthermore, by using the image processing method and apparatus according to this disclosure, the user (therapist) can easily identify carious teeth without using a separate light-emitting device. This results in the advantages of reducing device weight and lowering costs.

[0013] Furthermore, because the image data is generated as a 3D model, and the voxels corresponding to the decayed areas are visually displayed, users can easily identify the number of cavities in a clear location within the patient's mouth. Therefore, users can suggest treatment methods suitable for a predetermined tooth condition without needing to re-examine the actual teeth. Attached Figure Description

[0014] Figure 1 This is a flowchart of the image processing method according to the present disclosure.

[0015] Figures 2 to 4 This is a detailed flowchart of the image processing method according to the present disclosure.

[0016] Figures 5 to 8 The process of generating and adjusting a large region in the image processing method according to this disclosure is illustrated.

[0017] Figure 9 Exemplary image data of a large area is shown by adjusting the image processing method according to this disclosure.

[0018] Figure 10 The image data acquired by the scanning step in the image processing method according to this disclosure is illustrated schematically.

[0019] Figure 11 Image data for caries determination is schematically shown in the image processing method according to this disclosure.

[0020] Figure 12This is a flowchart of an image data processing method according to another embodiment of the present disclosure.

[0021] Figure 13 This is a detailed flowchart of an image data processing method according to another embodiment of the present disclosure.

[0022] Figure 14 The configuration of the image processing apparatus according to this disclosure is shown.

[0023] [Symbol Explanation]

[0024] S110: Scanning step; S130: Data analysis step

[0025] S131: Steps for dividing the area into large regions S132: Steps for determining dental caries

[0026] S133: Steps for Generating a 3D Model

[0027] S1311: Steps for generating multiple large regions

[0028] S1312: Steps for comparing eigenvalues

[0029] S1313: Procedures for Adjusting Large Area Boundaries

[0030] S1321: Caries Grading Procedures

[0031] S1322: Mask generation steps

[0032] S1323: Caries expression value mapping steps

[0033] S150: Display steps

[0034] S210: Scanning step; S230: Data analysis step

[0035] S231: 3D model generation step; S232: Tooth-specific segmentation step

[0036] S233: Steps for analyzing image data of selected teeth

[0037] S250: Display step 10: Image processing device

[0038] 100: Scanning unit; 200: Control unit

[0039] 210: Large area generation / adjustment unit; 220: Tooth decay determination unit.

[0040] 230: Mask generation unit; 240: 3D model generation unit

[0041] 300: Display unit. Detailed Implementation

[0042] Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. When adding reference numerals to elements in each drawing, the same elements will be represented by the same reference numerals, even if they are shown in different drawings. Furthermore, in the following description of embodiments of the present disclosure, detailed descriptions of known functions and configurations incorporated herein will be omitted where it is determined that such detailed descriptions might hinder understanding of the embodiments of the present disclosure.

[0043] When describing elements of embodiments of this disclosure, terms such as first, second, A, B, (a), (b), etc., may be used herein. These terms are used only to distinguish one element from other elements, and the nature, order, sequence, etc., of the corresponding elements are not limited by these terms. Unless otherwise defined, all terms used herein, including technical or scientific terms, shall have the meaning commonly understood by one of ordinary skill in the art to which this disclosure pertains. Terms defined in generally used dictionaries shall be interpreted as having a meaning consistent with the context of the relevant technical field, and shall not be construed as having an ideal or overly formal meaning unless expressly defined in this application.

[0044] In describing this disclosure, a pixel, which is the smallest unit constituting image data, is referred to as a small region, while a superpixel, which is a predetermined set of pixels, is referred to as a large region.

[0045] Figure 1 This is a flowchart of the image processing method according to the present disclosure. Figures 2 to 4 This is a detailed flowchart of the image processing method according to the present disclosure.

[0046] refer to Figure 1 The image processing method according to this disclosure may include: a scanning step S110, acquiring image data by scanning an object including teeth; a data analysis step S130, identifying caries in the image data; and a display step S150, displaying the image data of the identified caries in three dimensions. Each step will be described in detail below.

[0047] Scanning step S110

[0048] The image processing method according to this disclosure includes a scanning step S110. Scanning step S110 may be a step of acquiring image data by scanning an object. The object may be various objects, but for the purposes of this disclosure, it may be the interior of a patient's oral cavity, including teeth, gums, dental arch, etc. Scanning step S110 is performed by a scanning unit of the image processing apparatus, which will be described later, and the scanning unit may be a handheld scanner capable of freely adjusting the scanning distance to the object and the scanning angle to the object.

[0049] Image data can be two-dimensional or three-dimensional image data (shots). The image data acquired in scanning step S110 can be used in analysis step S130, which will be described later.

[0050] In one implementation, scanning step S110 may mean taking a picture of the object, and in this case, the image data may be two-dimensional image data. More specifically, in this disclosure, when tooth decay is determined, the two-dimensional image is divided into large regions with similar feature values, and the tooth decay is determined from the large regions. Compared to obtaining and analyzing feature values ​​from three-dimensional data, this has the advantages of saving storage space, enabling rapid analysis, and preventing resource waste.

[0051] A scanning unit can emit light toward an object to generate a 3D model from the acquired image data. The light emitted toward the object can take various forms. For example, the light can be light with wavelengths in the visible light region. Furthermore, the light can be structured light with a predetermined pattern. Structured light allows image data to contain depth information, and for example, a stripe pattern in which straight lines of different colors appear consecutively can be used as structured light. For example, structured light can be light with vertical or horizontal stripe patterns, where bright portions with relatively high brightness and dark portions with relatively low brightness are alternated. The pattern of structured light can be generated using pattern generation devices such as pattern masks or digital micromirror devices (DMDs). However, it is not always necessary to use structured light to acquire a 3D model, and any method capable of acquiring depth information of the object can be used to obtain a 3D model.

[0052] Data analysis step S130

[0053] Subsequently, a data analysis step S130 can be performed to determine dental caries using the image data acquired in the scanning step S110 described above. (Refer to...) Figure 2The data analysis step S130 may include step S131 of dividing the image data into predetermined large regions, step S132 of identifying cavities from the image data, and step S133 of generating a three-dimensional model from the image data of the identified cavities. In other words, the data analysis step S130 includes the process of interpreting and processing the image data acquired in the scanning step S110.

[0054] The detailed steps of data analysis step S130 will be described below.

[0055] refer to Figure 2 The data analysis step S130 may include a step S131 of dividing the image data into at least one large region. In fact, there may be multiple large regions, and boundaries may exist between them. In this case, each large region may be a superpixel comprising at least one small region.

[0056] Dividing the image data into at least one large region is for the purpose of identifying caries in the large region, and thus the computational load of the apparatus performing the image processing method according to the present disclosure can be reduced.

[0057] Figure 3 and Figures 5 to 8 The process of generating and adjusting a large area in the image processing method according to this disclosure is illustrated.

[0058] refer to Figure 3 and Figure 5 The step S131 of dividing the image data into large regions includes the step S1311 of generating multiple large regions for dividing the image data. For example, the image data is acquired in the form of rectangles, and the image data is formed by a set of small regions that represent the smallest units of the image.

[0059] Image data can be formed from a set of regularly arranged M×N (M and N are natural numbers, where M×N can represent the number of pixels arranged horizontally × the number of small regions arranged vertically) small regions. In step S1311, which generates multiple large regions, the image data can be divided to generate large regions of the same size and containing the same number of small regions. For example, when the image data has a resolution of 1600×1200 small regions, 16 large regions, each with 400×300 small regions, can be generated to divide the image data. When the image data is divided into multiple large regions, steps S1312 (comparing feature values) and S1313 (adjusting boundaries), which will be described later, can be executed quickly.

[0060] In step S1311, which generates multiple large regions, the number of large regions can be appropriately pre-configured based on the resolution of the image data. For example, if the image data as a whole has a resolution of 1600×1200 small regions, the image data can be divided into 10 pre-configured large regions. In this case, based on the number of large regions, the image data can be divided into large regions, each with a resolution of 320×600 small regions.

[0061] As an example describing this disclosure, image data having 6×6 small regions is presented. The image data can be divided into six large regions, each with the same size and 2×3 small regions, so as to be divided into multiple large regions 710, 720, 730, 740, 750, and 760. The boundaries BD between the large regions 710, 720, 730, 740, 750, and 760 can be optionally configured, and the boundaries BD can be changed by the boundary adjustment step S1313, which will be described later.

[0062] The number and size of the large regions mentioned above are examples and can be variably configured according to the user's needs or conditions regarding image data.

[0063] To adjust the boundaries, step S131, which divides the large regions, includes step S1312, which compares the feature values ​​between multiple large regions. Step S1312, comparing feature values, is a process for grouping small regions with the same or similar features into large regions. Feature values ​​can be at least one of multiple pieces of information obtained from image data. For example, feature values ​​can include color information, size information, texture information, and fill information, which indicate differences between the bounding boxes of data already input into the small regions constituting the image data. In this case, for the purpose of determining dental caries, color information can be used as a feature value. In step S1312, the feature values ​​of small regions adjacent to the boundary BD of the large region can be compared. By comparing the feature values ​​of the small regions adjacent to the boundary BD, the boundary BD can be changed (S1313) so that small regions with similar feature values ​​form large regions.

[0064] refer to Figures 6 to 8Image data divided into large regions yields color information for pixels 711 to 716 and 721 to 726 corresponding to each smaller region. Various color models, such as the Gray model, RGB model, HSV model, CMYK model, and YCbCr model, can be used to obtain this color information. However, the Gray model represents achromatic colors from 0 to 255, and it is not easy to clearly compare feature values ​​and determine actual cavities. Therefore, a chromatic color model can be used. To determine the similarity between smaller regions, the color histogram of the corresponding color model can be used as feature values.

[0065] The similarity determination based on color histograms will be described in more detail. A color histogram for each small region can be generated, and similarity can be determined by overlapping the color histograms of adjacent small regions (which have a boundary BD between them). For example, when generating color histograms for each of two random small regions (not shown) that are adjacent to each other and have a boundary between them, and when the generated color histograms overlap, the overlap area can be calculated. When the overlap area has a value equal to or greater than a threshold, it can be determined that the first and second small regions have similar feature values.

[0066] In the following text, for ease of description, a small area will be described as having a representative color. However, this should be interpreted as describing a small area with a color histogram that includes the representative color.

[0067] refer to Figure 6 The feature values ​​are assigned to each small region. At this point, the feature values ​​of small regions adjacent to the boundaries between large regions can be compared with each other. For example, the feature value of the second small region 712 of the first large region 710 and the feature value of the first small region 721 of the second large region 720 can be compared with each other. As shown in the figure, the feature value of the second small region 712 of the first large region 710 and the feature value of the first small region 721 of the second large region 720 can both be W (white).

[0068] Furthermore, the feature values ​​of the fourth sub-region 714 of the first large region 710 and the third sub-region 723 of the second large region 720 can be compared with each other. As shown in the figure, the feature values ​​of the fourth sub-region 714 of the first large region 710 and the third sub-region 723 of the second large region 720 can both be B (black).

[0069] Similarly, the feature values ​​of the sixth sub-region 716 of the first large region 710 and the fifth sub-region 725 of the second large region 720 can be compared. As shown in the figure, the feature value of the sixth sub-region 716 of the first large region 710 can be W (white), and the feature value of the fifth sub-region 725 of the second large region 720 can be Y (yellow).

[0070] Based on a comparison of the characteristic values ​​of the small regions adjacent to boundary BD, boundary BD can be adjusted to a new boundary BD' (S1313). (Refer to...) Figure 7 A portion of the first sub-region 721, which is part of the second large region 720, can be incorporated into the first large region 710, corresponding to the seventh sub-region 721' of the first large region 710. Furthermore, a portion of the fourth sub-region 714, which is part of the first large region 710, can be incorporated into the second large region 720, corresponding to the seventh sub-region 714' of the second large region 720. Therefore, the boundary BD' can be adjusted based on changes in the large region to which the sub-region belongs. Sub-region adjustment can be performed by referencing the feature values ​​of other sub-regions adjacent to the sub-regions adjacent to the existing boundary BD. Based on the continuous renewal of the boundary BD in the image data, sub-regions with similar feature values ​​can form large regions.

[0071] refer to Figure 8 The number of large regions can be varied based on the image data, and therefore, the number of small regions included in the initial large regions can also be varied. More specifically, the number of large regions can be greater than or less than the number of initially generated large regions, depending on the distribution of the feature values ​​of the small regions. For example, a fourth small region 714 included in the first large region 710 and a third small region 723 included in the second large region 720 can form a new large region. The new large region 770” can include small regions 714” and 723” with a feature value of B (black). In this way, small regions with similar feature values ​​can form large regions, and thus the image data can be divided in the form of superpixels. Therefore, it is possible to determine whether a tooth has caries for each large region, and the user can quickly identify teeth with caries. Furthermore, caries can be determined for each large region, thereby reducing the computational load on the apparatus performing the image processing method according to this disclosure.

[0072] Figure 9 An exemplary image data of a large area is shown by adjusting the image processing method according to this disclosure.

[0073] like Figure 9 As shown in figures a through 9c, the regions of image data are adjusted based on feature values ​​(e.g., color information). First, in... Figure 9 In step a, large regions of the same size are generated to divide the image data. Figure 9In step b, eigenvalue comparisons are performed near the boundaries between adjacent large regions, and boundary adjustments are made to form larger regions from smaller regions with similar eigenvalues. Large regions can be created or merged by performing boundary adjustments. Figure 9 In C, the boundary adjustment is complete, and the milestone, the first cloud, the second cloud, and the background are accurately distinguished and can be identified.

[0074] The caries identification step S132 will be described in detail below with reference to the relevant accompanying drawings.

[0075] refer to Figure 4 The caries determination step S132 includes a caries grading step S1321, which determines whether caries are present in each large region, or determines the caries grade for regions where caries are identified. Specifically, in the caries grading step S1321, artificial intelligence (AI) can be used to extract caries features and / or caries severity from image data divided into large regions.

[0076] Artificial intelligence can be deep learning, specifically at least one of the following: Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Deep Neural Network (DNN), and Recurrent Neural Network (RNN).

[0077] In this disclosure, for example, a convolutional neural network (CNN) can be used. In a CNN, convolutional layers and pooling layers are executed sequentially, or in addition to convolutional layers and pooling layers, fully connected layers may also be executed.

[0078] More specifically, the CNN obtains feature maps through convolutional layers (which perform convolutional steps using kernel filters of size n×n). Subsequently, pooling layers sample the values ​​of the feature maps using kernel filters of size n×n to highlight caries features. Caries features can be extracted by repeating the above process, and optionally, fully connected layers can classify caries levels (i.e., indicators of caries severity) according to pre-configured criteria.

[0079] Pre-configured standards can be, for example, the International Caries Detection and Assessment System (ICDAS). ICDAS is a standard for classifying the severity of dental caries, with severity divided into seven levels from integers 0 to 6. The level can be determined by characteristic values ​​of a large area.

[0080] After classifying the caries grade, in the caries determination step S132, a mask including at least one mask region can be generated based on the caries grade and / or the presence or absence of the determined caries (S1322). The mask can be a set of parts determined to have caries based on the presence or absence of caries and / or the caries grade, and these parts can be large areas. That is, large areas determined to have caries can constitute the mask region included in the mask.

[0081] Subsequently, caries expression values ​​corresponding to the presence or absence of caries and / or caries grade can be mapped to a mask area (S1323). Mapping caries expression values ​​can mean assigning a specific expression value to a mask area. The caries expression values ​​can be a means of visually displaying the presence or absence of caries to the user during the display step. For example, the caries expression values ​​can be at least one of a predetermined color and a predetermined pattern. The mask to which the caries expression values ​​are mapped can overlap with image data and can be displayed together with the image data or a 3D model.

[0082] The caries expression value can be mapped based on the presence or absence of caries and / or caries grade. For example, in the caries grading step S1321, the portion determined to be without caries is not formed as a mask, while the portion determined to have caries is formed as a mask. Therefore, by mapping at least one of a predetermined color and a predetermined pattern relative to the entire mask, the presence or absence of caries can be visually displayed. That is, the caries expression value can be the same value, regardless of the caries grade. For example, regarding the caries expression value, based on the result of determining only the presence or absence of caries without classifying the caries grade, the same caries expression value can be applied to the portion where caries has occurred, or the same caries expression value can be applied to the mask area with caries grades 3 to 6. The caries expression value can be fluorescent, red, or black. When the caries expression value is displayed, the user can easily identify teeth that have caries.

[0083] Alternatively, the caries expression value can be mapped to different values ​​based on the caries grade. More specifically, the mask can include mask areas formed for each caries grade, and different colors can be mapped to the caries grade of the mask area.

[0084] The classification of caries grades and the mapping of caries expression values ​​will be described in more detail. For example, a large region can have various feature values, such as white, yellow, brown, and black. A large region can include smaller regions with the same or similar feature values. Caries features and / or caries grades can be extracted from the feature values ​​of the large region using artificial intelligence, and a mask can be generated based on the extracted values. The generated mask can include at least one mask region, and caries expression values ​​can be mapped to the mask region.

[0085] For example, when a feature value is white in a random large area according to pre-configured criteria, that area is identified as a cavity-free region (Grade 0) and is not formed into a mask. Furthermore, when a region with a yellow feature value is located on dry enamel, that region is identified as having mild caries (Grade 1), and the caries expression value can be mapped to a mask area corresponding to the caries region. Additionally, when a region with a yellow feature value is located on wet enamel, that region is identified as having moderate caries (Grade 2), and the caries expression value can be mapped to a mask area corresponding to the caries region.

[0086] Furthermore, when cavities are present in teeth, the area is determined to be part of a tooth with grades 3 to 6 caries based on the area and / or ratio of the cavity, and the caries expression value can be mapped to a mask area corresponding to the carious area. More specifically, the grade 3 to 6 of the cavity can be determined based on the occupancy of large areas with black characteristic values ​​or the area of ​​large areas with black characteristic values. Alternatively, the caries grade can be classified by the relationship between the mixing ratio and the area.

[0087] For example, in a large area with a black feature value and a size less than 0.5 mm within the portion corresponding to tooth enamel, the caries expression value can be mapped to a mask area corresponding to the caries region. Similarly, when a large area with a black feature value occupies more than half of a predetermined image data, it is determined that very severe caries has occurred, and the caries expression value can be mapped to a mask area corresponding to the caries region. According to the above process, the portion where caries has occurred can be indicated. However, the reference values ​​used for classifying caries grades have been used as examples to describe this disclosure. Therefore, the values ​​can be changed as needed.

[0088] This section details how caries expression values ​​are colored differently depending on the caries grade. For example, at caries grade 0, no caries expression value is mapped because no mask is generated. At caries grade 1, green is mapped to the corresponding mask area; at caries grade 2, cyan is mapped; at caries grade 3, magenta is mapped; at caries grade 4, blue is mapped; at caries grade 5, red is mapped; and at caries grade 6, black is mapped.

[0089] As described above, the mask to which the caries expression values ​​are mapped can be displayed along with the actual color of the image data, while also overlapping the image data. The mask can have a predetermined transparency. Because the mask has a predetermined transparency, its advantage lies in allowing users to identify the actual shape of the teeth and whether they are decayed.

[0090] According to an optional embodiment, the caries expression value can be a 6-grade caries classification standard including integers from 0 to 5. This standard can be a simplified version of the ICDAS standard. For example, in the standard, when the feature value is white in a random large area, that area is identified as a caries-free region (grade 0) and is not formed as a mask. Furthermore, when a region with a yellow feature value is present, that region is identified as a region with moderate caries (grade 1), and the caries expression value can be mapped to a mask region corresponding to the caries region.

[0091] Furthermore, when cavities have already formed in a tooth, the corresponding area is identified as the portion of the tooth with grades 2 to 5 caries based on the area and / or ratio of the cavity, and the caries expression value can be mapped to a mask area corresponding to the caries region. More specifically, the grade 2 to 5 of the cavity can be determined based on the occupancy of a large area with a black characteristic value or the area of ​​a large area with a black characteristic value. Alternatively, the caries grade can be classified by using the relationship between ratio and area. As mentioned above, the same or different caries expression values ​​can be mapped according to the caries grade.

[0092] When the caries determination step S132 is completed and a mask is generated, and when caries expression values ​​are mapped according to the presence or absence of caries and / or caries grade (S1323), step S133 can be performed to generate a three-dimensional model including at least one voxel from the image data. In this case, the three-dimensional model can be generated by aligning multiple pieces of image data, and the two-dimensional data can be generated into a three-dimensional model using the depth information of the image data. Simultaneously, the mask overlaps with the image data. When generating the three-dimensional model from the image data, the voxels constituting the three-dimensional model can display caries expression values ​​only in the overlapping portions of the caries expression values. Therefore, the three-dimensional model can display the caries region together with the caries expression values.

[0093] Figure 10 The image data acquired through the scanning step in the image processing method according to this disclosure is illustrated schematically, and Figure 11 Image data for caries determination is schematically shown in the image processing method according to this disclosure.

[0094] refer to Figure 10 The image data 910 obtained through scanning step S110 is shown. Image data 910 may include teeth 911 and gingiva 912, and may identify portions of tooth 911 where caries is developing. For example, a first carious portion L1, a second carious portion L2, a third carious portion L3, a fourth carious portion L4, and a fifth carious portion L6 may be shown. Carious portions L1, L2, L3, L4, and L6 may be separated into at least one large region divided according to feature values ​​in step S131 (S131).

[0095] refer to Figure 11 Once the caries is identified, caries expression values ​​can be displayed on the image data. These values ​​allow users to clearly identify the locations of caries (L1', L2', L3', L4', and L6') on the image data 920. The advantage of displaying these caries expression values ​​is that users can accurately identify whether caries has occurred and its severity, and can then tailor treatment accordingly.

[0096] Show step S150

[0097] When the data analysis step S130 is completed, the display step S150, which displays the image data of the identified caries in three dimensions, can be executed. In the display step S150, the data analysis results can be displayed through an output device, which can be a visual display device such as a monitor.

[0098] The information displayed in step S150 is either two-dimensional image data of an object including teeth or a three-dimensional model generated in step S133 via three-dimensional model generation. While displaying caries expression values ​​on two-dimensional image data may be intuitive, accurately identifying the location of caries within the teeth is not easy. In contrast, a three-dimensional model expresses the color and shape of the object in a three-dimensional manner. A three-dimensional model can have the same color, shape, and size as the actual object, and when caries expression values ​​are displayed on a three-dimensional model, the user can easily identify the location, size, and angle of the caries, thus providing optimal treatment for the patient. Therefore, displaying the data analyzed in step S130 (especially caries identification) on a three-dimensional model is effective.

[0099] The image processing method described above can be performed only on the portion corresponding to the tooth region, and the image processing method according to this disclosure cannot be applied to portions corresponding to the gingiva and dental arch. For example, when the color obtained from each small region constituting the image data corresponds to the red series in the color histogram, the object included in the image data is likely to correspond to the gingiva or dental arch. Therefore, when the object is determined to be the gingiva or dental arch, a mask may not be generated at the corresponding location based on the color obtained from the small region. Through the above process, the teeth and other parts of the oral cavity (gingiva and dental arch) are distinguished, and a mask is selectively generated only for the tooth region, and caries expression values ​​can be mapped. Therefore, the presence or absence of caries and / or caries grade can be determined only for the tooth region, and user confusion can be prevented.

[0100] Furthermore, the image processing method described above not only displays caries expression values ​​across the entire 3D model, but also performs segmentation processing on each tooth and classifies the caries grade of each tooth. Based on the caries grade determined for each tooth, an additional process can be performed to propose appropriate treatment methods based on the location and area of ​​caries occurring in a pre-defined tooth. For example, when caries occurs at the cusp, an onlay treatment method can be displayed along with the caries expression values, and when caries occurs outside the cusp, an inlay treatment method can be displayed along with the caries expression values.

[0101] In the above text, externally exposed caries can be identified using feature values ​​from image data. However, it may be necessary to identify internal caries that are not externally exposed, and extraction may be required depending on the progression of the internal caries. Therefore, when selecting each tooth that has undergone segmentation processing in the generated 3D model, the extraction treatment method based on the internal caries type can be displayed along with caries expression values. For example, when a predetermined tooth is selected, an additional step to determine internal caries can be performed in addition to the previously performed process for determining surface caries. The step to determine internal caries can be performed based on feature values ​​obtained from image data, and these feature values ​​can be, for example, color information. If a predetermined tooth has a low-saturation color overall, the tooth can be determined to have internal caries even when the caries grade is low. For example, when a low-saturation portion in the image data of a predetermined tooth is greater than a pre-configured ratio, that low-saturation portion can be identified as internal caries, and the "extraction" treatment method can be displayed. Internal caries can be determined using color saturation, but it can also be determined using acoustic scanning information, infrared scanning information, etc. Because it also includes a step to identify internal caries, it can treat not only surface caries but also underlying caries.

[0102] In the following, an image processing method according to another embodiment of the present disclosure will be described. When describing this other embodiment, content identical to that of the image processing method according to the embodiments of the present disclosure will be omitted.

[0103] Figure 12 This is a flowchart of an image data processing method according to another embodiment of the present disclosure. Figure 13 This is a detailed flowchart of an image data processing method according to another embodiment of the present disclosure.

[0104] refer to Figure 12 and 13 An image processing method according to another embodiment of the present disclosure includes a scanning step S210 to acquire image data by scanning an object including teeth; a data analysis step S230 to determine caries in the image data; and a step S250 to display the image data of the determined caries in three dimensions. The scanning step S210 and the display step S250 according to another embodiment of the present disclosure are as described above.

[0105] In the data analysis step S230, unlike the above, a three-dimensional model can be generated without determining the caries based on the image data obtained in the scanning step S210 (S231).

[0106] Subsequently, the image processing method according to another embodiment of this disclosure may include step S232 of classifying the three-dimensional model of each tooth. The three-dimensional model can be classified for each tooth, allowing the user to select only the teeth for which caries determination is needed. More specifically, when the user selects the tooth for which caries determination is desired, only two-dimensional image data including that tooth can be extracted, and caries determination can be performed from the extracted image data (S233). Therefore, its advantage is that only image data including the tooth to be analyzed can be analyzed, thereby saving system resources required for analysis and enabling rapid analysis.

[0107] The method described above can be used to identify caries using image data, so the relevant description will be omitted.

[0108] Alternatively, when the image data is three-dimensional, it can be divided into large three-dimensional regions. At the boundaries between these large regions, the feature values ​​of adjacent smaller three-dimensional regions can be compared, and the boundaries between the large regions can be adjusted based on this comparison. Therefore, smaller three-dimensional regions with similar feature values ​​can form larger three-dimensional regions.

[0109] Subsequently, the presence or absence of caries and / or caries grade can be determined for each three-dimensional large region, and a three-dimensional mask with at least one mask region can be generated. Then, caries expression values ​​can be mapped to the mask regions based on the presence or absence of caries and / or caries grade. This is a transformation from a two-dimensional planar mask to a three-dimensional stereo mask, and the overall caries determination steps are as described above.

[0110] Furthermore, for regions identified as corresponding to the gingiva or dental arch in the 3D model, no masking or mapping of caries expression values ​​is required. This means that when the feature values ​​of a small 3D region (voxel) are in the red series of colors on a color histogram, the corresponding region can be identified as the gingiva or dental arch. This prevents caries expression values ​​from being mapped to regions identified as the gingiva or dental arch, thus preventing user confusion.

[0111] The image processing apparatus according to this disclosure will be described in detail below. However, content that overlaps with the image processing method described above according to this disclosure will be omitted.

[0112] Figure 14 The configuration of the image processing apparatus according to this disclosure is shown.

[0113] refer to Figure 14The image processing apparatus 10 according to the present disclosure includes: a scanning unit 100 configured to acquire image data by scanning an object including teeth; a control unit 200 for determining dental caries from the image data by analyzing the image data; and a display unit 300 for displaying the image data (from which dental caries are determined) in three dimensions.

[0114] The scanning unit 100 scans objects including teeth. For the purposes of this disclosure, the object may be the inside of a patient's oral cavity, used to determine whether dental caries have occurred. The scanning unit 100 may be a three-dimensional scanner capable of scanning the inside of a patient's oral cavity at various scanning distances and angles as it moves in and out of the patient's oral cavity. The scanning unit 100 may include at least one camera and an imaging sensor connected thereto to acquire image data of the object. The imaging sensor may be a monochrome image sensor or a color image sensor, such as a CCD sensor or a CMOS sensor.

[0115] The scanning unit 100 can emit structured light with a predetermined pattern, enabling the control unit 200 to generate a three-dimensional model from the acquired image data. The pattern of the structured light can be generated using a pattern generation device such as a pattern mask or a digital micromirror device (DMD). However, to generate image data as a three-dimensional model, structured light is not required, and at least one of known methods such as marking, lasers, and Time-of-Flight (TOF) can be used.

[0116] The configuration of the control unit 200 will be described in detail below.

[0117] The control unit 200 can analyze the image data acquired by the scanning unit 100. More specifically, the control unit 200 may include a large region generation / adjustment unit 210 that divides the image data into at least one large region. In this case, the large region may be a superpixel comprising at least one small region, and the small region may be pixels constituting the image data. The large region generation / adjustment unit 210 can generate large regions such that the image data is divided into at least one large region and identical small regions of the same size. Boundaries may be formed between the large regions.

[0118] The large region generation / adjustment unit 210 can compare feature values ​​of small regions adjacent to the boundary. Here, the feature values ​​to be compared are specific partial features of the image data possessed by the small region, and may include, for example, color information, texture information, size information, fill information, etc. Furthermore, the feature values ​​used for the purposes of this disclosure may be color information obtained from the image data.

[0119] When performing feature value comparison, the large region generation / adjustment unit 210 can adjust the boundaries so that small regions with similar feature values ​​are formed into the same large region. The process by which the large region generation / adjustment unit 210 generates large regions, compares feature values, and adjusts boundaries is the same as the process described above in the image processing method according to this disclosure.

[0120] Control unit 200 may include a caries determination unit 220 for determining caries based on large regions with adjusted boundaries. The caries determination unit 220 may determine the presence or absence of caries using artificial intelligence (AI) techniques including deep learning, and the deep learning techniques used to determine caries may be selected from ANN, CNN, DNN, and RNN already described above. The caries determination unit 220 may determine caries by determining the presence or absence of caries and / or the caries grade in each large region, and the standard may be the ICDAS standard. However, this disclosure is not limited to the ICDAS standard, and appropriate classification standards may be applied in some cases.

[0121] The control unit 200 may also include a mask generation unit 230 for generating a mask having at least one mask region based on the presence or absence of caries and / or caries level determined by the caries determination unit 220.

[0122] Subsequently, the mask generation unit 230 can map caries expression values ​​corresponding to the presence or absence of caries and / or caries grade to each mask region of the mask. Mapping caries expression values ​​can mean assigning specific expression values ​​to mask regions. Caries expression values ​​can be a means of visually displaying the presence or absence of caries to the user during the display step. For example, caries expression values ​​can be at least one of a predetermined color and a predetermined pattern. The mask to which the caries expression values ​​are mapped overlaps with the image data and can be displayed together with the image data or the 3D model described later.

[0123] The process of identifying dental caries, generating a mask, and mapping dental caries expression values ​​to the mask are the same as those described above with respect to the image processing method according to this disclosure, and therefore redundant descriptions will be omitted.

[0124] The control unit 200 may further include a 3D model generation unit 240 for generating a 3D model from the image data acquired by the scanning unit 100. To generate the 3D model, image data can be acquired using structured light or similar methods to include depth information of the object. The 3D model may include at least one voxel. When generating the 3D model from image data, the voxels constituting the 3D model can display caries expression values ​​only in areas where caries expression values ​​overlap. Therefore, its advantage is that users can easily identify the location, angle, size, etc., of caries in the 3D model, and can provide appropriate treatment for patients.

[0125] The image processing apparatus according to this disclosure may include a display unit 300 for three-dimensionally displaying the analysis results of the control unit 200. The display unit 300 may display image data, or image data that has been overlaid with a mask. Alternatively, the display unit 300 may display the caries expression values ​​of the three-dimensional model and its voxels, or may display the caries expression values ​​of the three-dimensional model and the three-dimensional mask mapped onto the three-dimensional model together. Therefore, the user does not need to subjectively analyze the patient's oral cavity or the generated three-dimensional model with the naked eye. As described above, regarding the patient's oral cavity, artificial intelligence can be used to objectively determine the presence or absence of caries and / or the caries grade, and display it visually, thereby enabling the user to provide rapid and objective treatment to the patient.

[0126] The above description is only for illustrating the technical ideas of this disclosure. Those skilled in the art will understand that various modifications and changes can be made without departing from the basic characteristics of this disclosure.

[0127] Therefore, the embodiments described in this disclosure are not intended to limit the technical concept of this disclosure, but rather to describe it, and the scope of the technical concept of this disclosure is not limited by these embodiments. The scope of protection of this disclosure should be interpreted based on the appended claims, such that all technical concepts included within the scope of the claims are included in this disclosure.

[0128] Industrial applicability

[0129] This disclosure provides an image processing method and an image processing apparatus, wherein, in order to easily identify a patient's dental caries, image data is acquired by scanning an object, dental caries are determined from the image data, and the image data of the identified dental caries is displayed in a three-dimensional manner.

Claims

1. An image processing method, comprising: The scanning step involves acquiring two-dimensional image data by scanning objects including teeth; The data analysis step involves using artificial intelligence to extract dental caries expression values ​​from the two-dimensional image data, mapping the dental caries expression values ​​corresponding to the dental caries regions in the two-dimensional image data, and generating a three-dimensional model from the two-dimensional image data in which the dental caries expression values ​​are mapped. The three-dimensional model expresses the color and shape of the object in a three-dimensional manner. as well as The steps involve displaying the caries expression values ​​on a three-dimensional model.

2. The image processing method according to claim 1, wherein, The data analysis steps include: The two-dimensional image data is divided into at least one large region; The steps for identifying dental caries include determining caries in each major area; and Generate a 3D model from 2D image data of teeth with known caries.

3. The image processing method according to claim 2, wherein, The large region is a superpixel that includes at least one small region.

4. The image processing method according to claim 2, wherein, The steps of dividing the large region include: Multiple large regions are generated by dividing the two-dimensional image data; Compare the feature values ​​among multiple large regions; and The boundaries between the large regions are adjusted based on the comparison of the feature values.

5. The image processing method according to claim 4, wherein, In the comparison of the feature values, the feature values ​​are color information obtained from the two-dimensional image data.

6. The image processing method according to claim 2, wherein, The steps for determining dental caries include: Determine the presence or absence of dental caries in each large region of the two-dimensional image data; Based on the determined presence or absence of caries, a mask including at least one mask region is generated; and The caries expression values ​​are mapped to the mask region.

7. The image processing method according to claim 2, wherein the tooth decay determination step includes: The caries grade is determined for each large region of the two-dimensional image data according to pre-configured standards; Generate a mask including at least one mask area based on the determined caries grade; as well as The caries expression value corresponding to the caries grade is mapped to the mask region.

8. The image processing method according to any one of claims 6 or 7, wherein, The caries expression value is at least one of a predetermined color and a predetermined pattern.

9. The image processing method according to any one of claims 6 or 7, wherein, The mask mapped by the caries expression value overlaps with the two-dimensional image data.

10. The image processing method according to claim 7, wherein the caries expression value has a different color or pattern for each caries grade.

11. The image processing method according to claim 9, wherein, The two-dimensional image data overlapping with the mask is used to generate the three-dimensional model, and the dental caries expression value is displayed on the three-dimensional model.

12. An image processing apparatus, comprising: The scanning unit is configured to acquire two-dimensional image data by scanning an object including teeth; The control unit is configured to use artificial intelligence to extract caries expression values ​​from the two-dimensional image data, map caries expression values ​​corresponding to caries regions in the two-dimensional image data, and generate a three-dimensional model from the two-dimensional image data in which the caries expression values ​​are mapped, the three-dimensional model representing the color and shape of the object in a three-dimensional manner; as well as The display unit is configured to display the caries expression value on a three-dimensional model.

13. The image processing apparatus according to claim 12, wherein, The control unit includes: A large region generation / adjustment unit is configured to divide the two-dimensional image data into at least one large region; and The caries identification unit is configured to identify caries in each large region.

14. The image processing apparatus according to claim 13, wherein, The large region is a superpixel that includes at least one small region.

15. The image processing apparatus according to claim 13, wherein, The large-area generation / adjustment unit is configured as follows: (1) Multiple large regions are generated by dividing the two-dimensional image data; (2) Compare the feature values ​​of multiple large regions; and (3) Adjust the boundaries between the large regions based on the comparison of the feature values.

16. The image processing apparatus according to claim 15, wherein, The feature value is color information obtained from the two-dimensional image data.

17. The image processing apparatus according to claim 13, wherein, The caries determination unit is configured to determine the presence or absence of caries in each large region of the two-dimensional image data.

18. The image processing apparatus according to claim 13, wherein, The caries determination unit is configured to determine the caries level of each large region of the two-dimensional image data.

19. The image processing apparatus according to claim 13, wherein, The control unit further includes a mask generation unit configured to generate a mask comprising at least one mask region based on the presence or absence of caries or caries grade in each large region determined by the caries determination unit. The mask generation unit is configured to map the caries expression value, which is determined by the caries determination unit and corresponds to the presence or absence of caries or the caries grade, to each mask region, such that the mask overlaps with the two-dimensional image data.

20. The image processing apparatus according to claim 19, wherein, The control unit further includes a 3D model generation unit, which is configured to generate a 3D model from the 2D image data superimposed by the mask. The display unit is configured to display the three-dimensional model and the caries expression values ​​superimposed on the three-dimensional model.

Images