Artificial intelligence-based tooth shade diagnosis system and operating method therefor
The AI-based tooth shade diagnosis system addresses the challenge of determining tooth whitening grades by using an SVM model to process and classify tooth photo information, ensuring accurate and efficient assessments across different illuminance conditions.
Patent Information
- Application Number
- PCT/KR2024/005013
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-03
- Filing Date
- 2024-04-15
- Publication Date
- 2024-11-07
AI Technical Summary
Accurately determining tooth whitening grade is challenging due to the difficulty in assessing tooth color brightness and hue with the naked eye, and existing methods face limitations in data collection and processing, particularly in varying illuminance conditions.
An artificial intelligence-based tooth shade diagnosis system utilizing a Support Vector Machine (SVM) learning model that processes tooth photo information, adjusts for illuminance, and generates prediction information on tooth whitening grade by converting color data and applying it to a pre-trained model, ensuring accurate classification across different illuminance ranges.
The system provides improved accuracy and speed in predicting tooth whitening grades by standardizing color assessment and adapting to varying lighting conditions, enhancing the precision of tooth shade evaluation.
Smart Images

Figure KR2024005013_07112024_PF_FP_ABST
Abstract
Description
Artificial intelligence-based tooth shade diagnosis system and operating method therefor
[0001] The present specification relates to an artificial intelligence-based tooth shade diagnosis system and an operating method therefor, and more specifically, to a tooth shade diagnosis system implemented to predict a tooth whitening grade based on an artificial intelligence learning model learned using a support vector machine (hereinafter referred to as 'SVM'), which is one of artificial intelligence supervised learning algorithms, and an operating method therefor.
[0002] A tooth shade is a standardized color chart used to compare and evaluate tooth color. Specifically, a tooth shade is used in dentistry to determine the color of artificial teeth or to record a patient's tooth color.
[0003] Older shades are divided into four series: A, B, C, and D, each numbered from 1 to 4. For example, A2 represents the second shade in the A series. Because older shades simultaneously express both the brightness and color of teeth, it can be difficult to accurately determine the teeth whitening grade with the naked eye.
[0004] For a previous proposal, reference may be made to Korean Patent No. 10-2056008 regarding ‘Method for selecting tooth shade and system thereof.’
[0005] The purpose of this specification is to provide an artificial intelligence-based tooth shade diagnosis system and an operating method therefor, which are implemented to predict a tooth whitening grade based on an artificial intelligence learning model learned using a support vector machine (SVM), which is one of the artificial intelligence supervised learning algorithms.
[0006] An operating method for an artificial intelligence-based tooth shade diagnosis device performed by an operating server device according to an embodiment of the present invention comprises the steps of: individually acquiring first tooth photograph information photographed in a predetermined illuminance range from at least one user terminal, wherein the first tooth photograph information includes an image of a gray card on which a plurality of reference colors are displayed and a tooth part image; performing preprocessing on the first tooth photograph information to generate second tooth photograph information; generating color conversion data based on first color data acquired from a plurality of reference colors included in the second tooth photograph information and preset reference color data for the plurality of reference colors; generating second color data by applying the color conversion data based on the second tooth photograph information; generating prediction information indicating a tooth whitening grade for at least one terminal by inputting the second color data into a pre-learned tooth shade learning model; and transmitting the prediction information to at least one user terminal.
[0007] According to one embodiment of the present invention, the first color data is generated based on first cropped image information obtained from at least one first location associated with a plurality of reference colors.
[0008] According to the present embodiment, the second tooth photograph information corresponds to second cropped image information obtained from at least one second location determined in advance to determine a tooth color for at least one user terminal.
[0009] According to the present embodiment, the tooth shade learning model is implemented based on a support vector machine based on a plurality of predetermined hyperplanes.
[0010] According to one embodiment of the present invention, the tooth whitening grade is associated with any one of a plurality of class information for representing a tooth shade, and the plurality of class information is implemented to be individually associated with a plurality of hyperplanes.
[0011] According to one embodiment of the present invention, the step of generating prediction information indicating a tooth whitening grade includes the step of specifying one of a plurality of class information based on a correlation between second color data and a plurality of hyperplanes; and the step of including the specified class information in the prediction information.
[0012] According to the present embodiment, when a shooting button for first tooth photo information is activated on at least one user terminal, the method further includes a step of determining whether the light intensity falls within a predetermined range by at least one user terminal.
[0013] According to the present embodiment, when it is determined by at least one user terminal that the illumination range is outside a predetermined range, the generation of the first tooth photo information is skipped and a message is generated instructing at least one user terminal to adjust the shooting illumination or take the photo at a different location.
[0014] According to the present embodiment, when it is determined that the illumination range is within a predetermined range, first tooth photo information is generated by at least one user terminal.
[0015] According to the present embodiment, a tooth whitening grade is predicted based on an artificial intelligence learning model learned using a support vector machine (SVM), which is one of artificial intelligence supervised learning algorithms, thereby providing improved accuracy and speed compared to the conventional method when determining a tooth whitening grade.
[0016] Figure 1 is a drawing for explaining multiple class information according to tooth shade.
[0017] FIG. 2 is a conceptual diagram for explaining an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0018] FIG. 3 is a block diagram showing an operation server device of an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0019] FIG. 4 is a drawing exemplarily showing first tooth photograph information including a tooth image and a gray card image according to an embodiment of the present invention.
[0020] FIG. 5 is a drawing for exemplarily explaining reference color data and second cropped image information according to the present embodiment.
[0021] FIG. 6 is a conceptual diagram for explaining a process of obtaining first cropped image information and second cropped image information according to an embodiment of the present invention.
[0022] FIG. 7 is a conceptual diagram illustrating a method for generating prediction information indicating a tooth whitening grade based on a support vector machine according to an embodiment of the present invention.
[0023] FIG. 8 is an operation diagram for explaining a process of acquiring first tooth photograph information in a predetermined illuminance range according to an embodiment of the present invention.
[0024] FIG. 9 is a flowchart for explaining the operation method of the artificial intelligence-based tooth shade diagnosis system according to the present embodiment.
[0025] FIG. 10 and FIG. 11 are conceptual diagrams for explaining the learning and prediction process of a tooth shade learning model for an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0026] FIG. 12 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.
[0027] The aforementioned characteristics and the detailed description below are all exemplary matters intended to aid in the explanation and understanding of this specification. In other words, this specification is not limited to these embodiments and may be embodied in other forms. The following embodiments are merely examples intended to fully disclose this specification and are intended to convey the disclosure of this specification to those skilled in the art. Therefore, if there are multiple methods for implementing the components of this specification, it is necessary to clearly state that this specification can be implemented using any of these methods, either a specific one or an equivalent one.
[0028] When this specification refers to a configuration as including specific elements, or a process as including specific steps, it implies that other elements or steps may also be included. In other words, the terminology used herein is intended only to describe specific embodiments and is not intended to limit the concepts of this specification. Furthermore, examples provided to aid understanding of the invention also include complementary embodiments.
[0029] The terms used in this specification have the meanings commonly understood by those skilled in the art to which this specification pertains. Commonly used terms should be interpreted consistently within the context of this specification. Furthermore, terms used in this specification should not be interpreted in an overly idealistic or formal sense unless their meanings are clearly defined. Embodiments of this specification are described below with reference to the accompanying drawings.
[0030] Figure 1 is a drawing for explaining multiple class information according to tooth shade.
[0031] Referring to Fig. 1 (a), the Vita Shade Guide, which is widely known for indicating tooth whitening grades, can be defined into 16 types using 4 types (i.e., A to D) by hue (HUE) and 4 types (i.e., 1 to 4) by chroma.
[0032] Referring to FIG. 1 (b), the tooth whitening grade mentioned in the present specification can be differentially expressed according to tooth class information corresponding to 16 vitamin color classifications.
[0033] For example, when the tooth class information is set to '1', the brightest vita color distinction (B1) in the vita shade guide may be indicated. Additionally, when the tooth class information is set to '2', the vita color distinction (A1) in the vita shade guide may be indicated.
[0034] For example, when the tooth class information is set to '3', the Vita color distinction (B2) may be indicated in the Vita shade guide. Additionally, when the tooth class information is set to '4', the Vita color distinction (D2) may be indicated in the Vita shade guide.
[0035] For example, when the tooth class information is set to '5', the Vita color distinction (A2) may be indicated in the Vita shade guide. Additionally, when the tooth class information is set to '6', the Vita color distinction (C1) may be indicated in the Vita shade guide.
[0036] For example, when the tooth class information is set to '7', the Vita color distinction (C2) may be indicated in the Vita shade guide. Additionally, when the tooth class information is set to '8', the Vita color distinction (D4) may be indicated in the Vita shade guide.
[0037] For example, when the tooth class information is set to '9', the Vita color distinction (A3) may be indicated in the Vita shade guide. Additionally, when the tooth class information is set to '10', the Vita color distinction (D3) may be indicated in the Vita shade guide.
[0038] For example, when the tooth class information is set to '11', the Vita color classification (B3) may be indicated in the Vita Shade Guide. Additionally, when the tooth class information is set to '12', the Vita color classification (A3.5) may be indicated in the Vita Shade Guide.
[0039] For example, when the tooth class information is set to '13', the Vita color classification (B4) may be indicated in the Vita Shade Guide. Additionally, when the tooth class information is set to '14', the Vita color classification (C3) may be indicated in the Vita Shade Guide.
[0040] For example, when the tooth class information is set to '15', the Vita color classification (A4) may be indicated in the Vita Shade Guide. Additionally, when the tooth class information is set to '16', the darkest Vita color classification (C4) may be indicated in the Vita Shade Guide.
[0041] FIG. 2 is a conceptual diagram for explaining an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0042] Referring to FIGS. 1 and 2, the artificial intelligence-based tooth shade diagnosis system (200) of FIG. 2 may include at least one user terminal (210) and an operation server device (220).
[0043] The artificial intelligence-based tooth shade diagnosis system (200) according to the present specification can be implemented so that first tooth photo information is collected from users within a predetermined illuminance range.
[0044] For example, at least one user terminal (210) can generate first tooth photograph information captured in a predetermined illuminance range. Here, the predetermined illuminance range can be set to 250 to 300 lux.
[0045] In this case, it will be understood that at least one user terminal (210) may be a wireless communication device for at least one user on which an application associated with the artificial intelligence-based tooth shade diagnosis system (200) according to the present specification is pre-installed.
[0046] Specifically, when a shooting button for generating first tooth photo information is activated in at least one user terminal (210), a light sensor (not shown) of at least one user terminal (210) can be implemented to determine whether the light around the user falls within a predetermined light range.
[0047] For example, the first tooth photograph information may include a tooth image photographed so that multiple teeth of the face of at least one user corresponding to at least one user terminal (210) are visible.
[0048] Additionally, the first tooth photograph information may include an image of a gray card on which multiple reference colors are displayed.
[0049] In this case, the first tooth photo information can be transmitted to the operation server device (220) via a pre-established wireless connection (e.g., Wi-Fi, LTE) with at least one user terminal (210).
[0050] For example, the first tooth photo information can be transmitted to the operation server device (220) through an image upload function implemented in an application associated with the artificial intelligence-based tooth shade diagnosis system (200).
[0051] The action server device (220) of FIG. 2 may be understood as a cloud computing-based device. In addition, the action server device (220) may be linked with at least one user terminal (210) based on a pre-established wireless connection (e.g., Wi-Fi, LTE).
[0052] In this case, the action server device (220) can individually receive (or acquire) first tooth photo information from at least one user terminal (210).
[0053] For example, the action server device (220) can generate prediction information indicating a tooth whitening grade for at least one terminal (210) based on first tooth photo information individually received (or acquired) from at least one user terminal (210).
[0054] In this case, the prediction information indicating the tooth whitening grade for at least one terminal can be transmitted to at least one user terminal (210) via a pre-established wireless connection (e.g., Wi-Fi, LTE).
[0055] The process of generating prediction information by the action server device (220) is described in more detail with reference to the drawings described below.
[0056] FIG. 3 is a block diagram showing an operation server device of an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0057] Referring to FIGS. 1 to 3, it will be understood that the operation server device (300) of FIG. 3 has a configuration corresponding to the operation server device (220) of FIG. 2 described above.
[0058] The action server device (300) of FIG. 3 may include a preprocessing module (310), an area extraction module (320), a correction module (330), and a prediction module (340).
[0059] The preprocessing module (310) of FIG. 3 can be implemented based on image editing software to perform preprocessing on the first tooth photo information.
[0060] For example, the preprocessing performed by the preprocessing module (310) may include a white balance operation that processes the image to make the color tone of the image captured by a camera module (not shown) equipped in at least one user terminal (e.g., 210 of FIG. 2) more similar to the actual color tone.
[0061] For example, image editing software for performing white balance work of the preprocessing module (310) may be implemented based on at least one of a gray world algorithm, a color temperature algorithm, and a selective color algorithm.
[0062] For reference, an Auto White Balance (hereinafter referred to as 'AWB') function may be provided that automatically detects the color temperature of a white light source in a scene captured by a camera module (not shown) equipped in at least one user terminal (e.g., 210 of FIG. 2) and adjusts the color temperature of the image.
[0063] However, the automatic white balance (AWB) function provided in at least one user terminal (e.g., 210 in FIG. 2) may not function properly when shooting in an environment where various lighting conditions exist in one location, when shooting against a background that contains a lot of a specific color (e.g., green grass, blue waves), or when the intensity of light changes suddenly during shooting (e.g., sunset).
[0064] Accordingly, even if preprocessing is not properly performed in at least one user terminal (e.g., 210 of FIG. 2), the preprocessing module (310) according to the present embodiment can be implemented to perform preprocessing, including white balance work, so that an image similar to an actual color tone is generated in the operation server device (300).
[0065] The region extraction module (320) of FIG. 3 can be implemented to generate (or extract) first cropped image information from at least one first location and second cropped image information from at least one second location.
[0066] For example, the region extraction module (320) can be implemented to recognize at least one first location and at least one second location based on a predetermined object recognition algorithm.
[0067] Here, the object recognition algorithm can correspond to the Haar Cascades algorithm of the OpenCV library or the deep learning-based YOLO (You Only Look Once) algorithm.
[0068] For example, the first cropped image information (e.g., IMG_c1 in FIG. 6) can be obtained from at least one first location (e.g., P1 in FIG. 6) associated with a plurality of reference colors (e.g., 600 in FIG. 6) of gray cards (e.g., 60 in FIG. 6) included in the second tooth photograph information (e.g., 6000 in FIG. 6).
[0069] For example, the second cropped image information (e.g., IMG_c2 in FIG. 6) can be obtained from at least one second location (e.g., P2 in FIG. 6) for determining the tooth color based on the second tooth photo information (e.g., 6000 in FIG. 6).
[0070] For reference, the process of obtaining the first cropped image information and the second cropped image information according to the present embodiment is described in more detail with reference to FIG. 6 described below.
[0071] The correction module (330) of FIG. 3 can be implemented to generate second color data based on CIE L*a*b data and color conversion data obtained from second cropped image information corresponding to second tooth photograph information.
[0072] For example, the color conversion data can be generated based on the difference between CIE L*a*b data obtained from first cropped image information (e.g., IMG_c1 of FIG. 6) associated with at least one first location (e.g., P1 of FIG. 6) and preset reference color data (e.g., [100, 0.0, 0.0]) for one of a plurality of reference colors (e.g., 600 of FIG. 6) (e.g., White).
[0073] As another example, when first cropped image information associated with two or more (B1, White) among a plurality of reference colors (e.g., 600 in FIG. 6)) is acquired, color conversion data can be generated based on an average value of the difference between CIE L*a*b data acquired from the first cropped image information and preset reference color data.
[0074] For example, a first difference between CIE L*a*b data obtained from the first cropped image information associated with the vita color classification (B1) and reference color data preset for the vita color classification (B1) (e.g., [78.90, -1.76, 12.33]) can be calculated.
[0075] Similarly, a second difference between the CIE L*a*b data obtained from the first cropped image information associated with the vita color distinction (White) and the preset reference color data (e.g., [100, 0.0, 0.0]) for the vita color distinction (White) can be calculated.
[0076] In this case, when first cut-out image information associated with two or more (B1, White) among a plurality of reference colors (e.g., 600 in FIG. 6) is acquired, the color conversion data may be determined as the average value of the first difference of the vita color distinction (B1) and the second difference of the vita color distinction (White).
[0077] The prediction module (340) of FIG. 3 is an input variable corresponding to the CIE a value and the CIE b value included in the second color data associated with the second tooth photograph information. , ) can be implemented to generate predictive information indicating a tooth whitening grade.
[0078] The learning process of the tooth shade learning model for the prediction module (340) of the present specification and the prediction process by the tooth shade learning model are described in more detail with reference to FIGS. 10 and 11 described below.
[0079] FIG. 4 is a drawing exemplarily showing first tooth photograph information including a tooth image and a gray card image according to an embodiment of the present invention.
[0080] Referring to FIGS. 1 to 4, the first tooth photograph information (4000) according to the present embodiment may include a tooth image photographed to show multiple teeth of at least one user's face and an image of a gray card (40) on which multiple reference colors (410 to 450) are displayed.
[0081] For example, a plurality of reference colors (410 to 450) displayed in a certain shape (e.g., octagon) on a gray card (40) can be implemented in five colors as shown in FIG. 4, but it will be understood that the present specification is not limited thereto.
[0082] Meanwhile, by performing preprocessing (e.g., white balance) on the first tooth photo information (4000) according to the present embodiment, the second tooth photo information of FIG. 5 described below can be generated.
[0083] FIG. 5 is a drawing for exemplarily explaining reference color data and second cropped image information according to the present embodiment.
[0084] Referring to FIGS. 4 and 5, the second tooth photo information (5000) of FIG. 5 may correspond to the result of performing preprocessing on the first tooth photo information (4000) of FIG. 4, and the gray card (50) of FIG. 5 and the plurality of reference colors (510 to 550) included therein may be understood to correspond to the gray card (40) of FIG. 4 and the plurality of reference colors (410 to 450) included therein.
[0085] Referring to FIGS. 1 to 5, first color data can be generated based on a plurality of reference colors (510 to 550) displayed on an image of a gray card (50) included in second tooth photograph information (5000).
[0086] In other words, RGB data or CIE L*a*b data can be obtained through image processing for at least one of the plurality of reference colors (510 to 550) displayed in the image of the gray card (50) included in the second tooth photograph information (5000).
[0087] In this case, at least one first position associated with a plurality of reference colors (510 to 550) displayed on the image of the gray card (50) included in the second tooth photograph information (5000) can be determined as an arbitrary internal position (e.g., central portion) of a certain shape (e.g., octagon).
[0088] Specifically, a predetermined object recognition algorithm may be applied to the artificial intelligence-based tooth shade diagnosis system (e.g., 200 of FIG. 2) according to the present embodiment so as to recognize at least one first location included in the second tooth photo information (5000).
[0089] Here, the object recognition algorithm can correspond to the Haar Cascades algorithm of the OpenCV library or the deep learning-based YOLO (You Only Look Once) algorithm.
[0090] For example, it will be understood that image processing can be implemented in various ways, such as using the DCT algorithm or the Open CV library, depending on the format of the image file associated with the second tooth photograph information (5000).
[0091] Additionally, reference color data for a plurality of reference colors (510 to 550) displayed in the image of the gray card (50) included in the second tooth photograph information (5000) can be preset.
[0092] The reference color data according to the present embodiment can be preset based on CIE L*a*b data defined in the CIE L*a*b color space.
[0093] For reference, in this specification, the conversion from RGB data to CIE L*a*b data defined in the CIE L*a*b color space may include a process of converting RGB data to an sRGB color space corresponding to an absolute color space, a gamma correction process, a process of converting sRGB data to a CIEXYZ color space based on a linear transformation, and a process of converting CIEXYZ data to a CIE L*a*b color space based on a nonlinear transformation.
[0094] In the CIE L*a*b data according to the present embodiment, 'L' may be implemented to represent brightness in a color space, 'a' to represent the degree of red and green, and 'b' to represent the degree of yellow and blue.
[0095] For example, the first reference color (510) of the reference color data may be associated with the Vita color classification (B1). In this case, the RGB data associated with the first reference color (510) may be set to [200, 195, 171].
[0096] Here, RGB data set to [200, 195, 171] can be associated with CIE L*a*b data defined as [78.90, -1.76, 12.33].
[0097] For example, the second reference color (520) of the reference color data may be associated with the Vita color classification (A3). In this case, the RGB data associated with the second reference color (520) may be set to [200, 184, 148].
[0098] Here, RGB data set to [200, 184, 148] can be associated with CIE L*a*b data defined as [75.36, 1.36, 19.61].
[0099] For example, RGB data associated with the third reference color (530) of the reference color data may be set to [255, 255, 255] associated with white corresponding to the brightest color.
[0100] Here, RGB data set to [255, 255, 255] can be associated with CIE L*a*b data defined as [100, 0.0, 0.0].
[0101] For example, the fourth reference color (540) of the reference color data may be associated with the Vita color classification (A3.5). In this case, the RGB data associated with the fourth reference color (540) may be set to [193, 175, 136].
[0102] Here, RGB data set to [193, 175, 136] can be associated with CIE L*a*b data defined as [72.31, 1.48, 21.81].
[0103] For example, the fifth reference color (550) of the reference color data may be associated with the Vita color classification (C4). In this case, the RGB data associated with the fifth reference color (550) may be set to [170, 156, 123].
[0104] Here, RGB data set to [170, 156, 123] can be associated with CIE L*a*b data defined as [64.78, 1.59, 18.66].
[0105] It will be understood that the method of implementing the reference color data according to the present embodiment according to the CIE L*a*b color space is only an example, and that the present specification is not limited thereto. In other words, it will be understood that the reference color data according to the present embodiment may also be implemented in the YIQ color space or the YUV color space.
[0106] FIG. 6 is a conceptual diagram for explaining a process of obtaining first cropped image information and second cropped image information according to an embodiment of the present invention.
[0107] For a clear and concise understanding of FIG. 6, it can be assumed that FIG. 6 is associated with second tooth photograph information (6000) to which preprocessing (i.e., white balance) has been applied as described above.
[0108] In addition, it can be assumed that the first cropped image information (IMG_c1) and the second cropped image information (IMG_c2) are defined (or determined) on the second tooth photograph information (6000) of FIG. 6.
[0109] Referring to FIGS. 1 to 6, first cropped image information (IMG_c1) can be obtained in a predetermined size and shape from at least one first location (P1) associated with a plurality of reference colors (600) included in the gray card (60) of FIG. 6.
[0110] For example, at least one first position (P1) may be determined based on an object recognition algorithm (e.g., YOLO algorithm) pre-applied to the artificial intelligence-based tooth shade diagnosis system (e.g., 200 of FIG. 2) according to the present embodiment.
[0111] For example, the first cropped image information (IMG_c1) of FIG. 6 can be obtained from the first position (P1) associated with the brightest color distinction (White) among the plurality of reference colors (600).
[0112] In FIG. 6, at least one first position (P1) is shown as being displayed for the brightest color distinction (White) among the plurality of reference colors (600) of the gray card (60), but it will be understood that the present specification is not limited to this embodiment.
[0113] In other words, it will be understood that, depending on the implementation method, first cropped image information (IMG_c1) associated with two or more of the plurality of reference colors (600) included in the gray card (60) can be acquired.
[0114] According to this embodiment, color conversion data can be generated based on first color data obtained from the first cut image information (IMG_c1) of FIG. 6 and preset reference color data for a plurality of reference colors (600).
[0115] For example, color conversion data can be generated based on the difference between first color data (i.e., CIE L*a*b data) obtained from the first cropped image information (IMG_c1) and preset reference color data (e.g., [100, 0.0, 0.0]) for one of a plurality of reference colors (600) (e.g., White).
[0116] As another example, when first cropped image information (IMG_c1) associated with two or more (B1, White) of a plurality of reference colors (600) is acquired, color conversion data may be generated as an average value of the difference between first color data (i.e., CIE L*a*b data) acquired from the first cropped image information (IMG_c1) and preset reference color data.
[0117] For example, a first difference (d1) between CIE L*a*b data obtained from the first cropped image information (IMG_c1) associated with the vita color classification (B1) and reference color data preset for the vita color classification (B1) (e.g., [78.90, -1.76, 12.33]) can be calculated.
[0118] Similarly, a second difference (d2) between the CIE L*a*b data obtained from the first cropped image information (IMG_c1) associated with the vita color distinction (White) and the preset reference color data (e.g., [100, 0.0, 0.0]) for the vita color distinction (White) can be calculated.
[0119] In other words, when first cut-out image information (IMG_c1) associated with two or more (B1, White) among a plurality of reference colors (600) is acquired, the color conversion data can be determined as the average value of the first difference (d1) of the vita color distinction (B1) and the second difference (d2) of the vita color distinction (White).
[0120] For example, the second cropped image information (IMG_c2) of FIG. 6 can be obtained from at least one second location (P2) with a predetermined size and shape.
[0121] In this case, at least one second location (P2) may correspond to a true dentin reflect zone corresponding to the area that is the basic color of the tooth (i.e., the area where the opaque layer of the tooth is most distinct).
[0122] That is, at least one second location (P2) can be implemented based on an automatic recognition function for a region that is the basic color of the tooth (e.g., True dentin reflect zone) as described above.
[0123] Specifically, a predetermined object recognition algorithm may be applied to the artificial intelligence-based tooth shade diagnosis system according to the present embodiment so as to recognize at least one second location (P2) directly associated with the teeth of an actual user.
[0124] Here, the object recognition algorithm can correspond to the Haar Cascades algorithm of the OpenCV library or the deep learning-based YOLO (You Only Look Once) algorithm.
[0125] According to this embodiment, second color data can be generated by applying second tooth photo information (6000) and color conversion data.
[0126] In other words, second color data can be generated by applying color conversion data to color data obtained from second cropped image information (IMG_c2) associated with second tooth photograph information (6000).
[0127] Specifically, second color data can be generated by applying color conversion data corresponding to the difference between the first color data and the reference color data to CIE L*a*b data obtained from the second cropped image information (IMG_c2) associated with the second tooth photograph information (6000).
[0128] In addition, when the second color data is input into the tooth shade learning model according to the present specification (e.g., 200 of FIG. 2), it can be implemented so that prediction information associated with a tooth whitening grade specified by the pre-learned tooth shade learning model is generated.
[0129] FIG. 7 is a conceptual diagram illustrating a method for generating prediction information indicating a tooth whitening grade based on a support vector machine according to an embodiment of the present invention.
[0130] The support vector machine (SVM) of Fig. 7 is one of the machine learning algorithms based on supervised learning, and can be implemented in a supervised learning manner that learns data with labels or dependent variables and then predicts labels (or dependent variables).
[0131] In particular, the support vector machine (SVM) of the present specification can be implemented for classification of 16 tooth whitening grades.
[0132] Additionally, the support vector machine (SVM) of the present specification may determine one or more hyperplanes corresponding to a decision boundary to maximize the margin (i.e., to be as far away as possible) from a group of data associated with a plurality of groups in order to determine a plurality of groups.
[0133] Referring to FIGS. 1 to 7, a model learned for determining tooth whitening grade for the implementation of a support vector machine (SVM) (hereinafter, “tooth shade learning model”) can be expressed in two dimensions as shown in FIG. 7.
[0134] For example, the X-axis of the tooth shade learning model of FIG. 7 may correspond to the CIE a value of the CIE L*a*b data, and the Y-axis of the tooth shade learning model may correspond to the CIE b value of the CIE L*a*b data.
[0135] Meanwhile, it will be understood that the CIE L value in the CIE L*a*b data may not be displayed as separate axis information as in Fig. 7, considering the technical characteristic that the first tooth photograph information of this specification is acquired in a predetermined illuminance range.
[0136] Additionally, the tooth shade learning model according to the present embodiment can be learned based on tooth class information corresponding to 16 vitamin color classifications.
[0137] Referring to FIG. 7, among the 16 Vita color classifications, Shade tab A associated with at least one of the A classifications (A1, A2, A3, A3.5, A4) and Shade tab B associated with at least one of the B classifications (B1, B2, B3, B4) may be displayed.
[0138] The points corresponding to Shade tab A and Shade tab B shown in Fig. 7 can be understood as learning data corresponding to the CIE a value and CIE b value generated from the preprocessed second tooth photograph information.
[0139] Additionally, the plurality of support vectors shown in Fig. 7 refer to data points located close to the decision boundary, and can be understood as being defined to define the decision boundary.
[0140] In addition, the point corresponding to the Natural tooth shown in FIG. 7 may correspond to second color data (i.e., CIE a value and CIE b value) obtained by applying color conversion data to CIE L*a*b data for second cropped image information (IMG_c2) obtained from second tooth photograph information transmitted from at least one terminal (e.g., 210 of FIG. 2).
[0141] In this case, in response to the second tooth photograph information corresponding to the natural tooth, the action server device (e.g., 220 of FIG. 2) can transmit prediction information indicating a tooth whitening grade associated with at least one of the A classifications (A1, A2, A3, A3.5, A4) for at least one terminal (e.g., 210 of FIG. 2) that transmitted the second tooth photograph information.
[0142] For example, the prediction information indicating the tooth whitening grade can be implemented to specify one of a plurality of class information based on the correlation between the second color data (i.e., CIE a value and CIE b value) corresponding to the natural tooth and a plurality of preset hyperplanes.
[0143] For example, the process of specifying specific class information based on the correlation between the second color data and multiple hyperplanes can be understood based on mathematical expressions 1 to 8 mentioned in FIGS. 10 and 11 described below.
[0144] In FIG. 7, only a hyperplane is shown to distinguish between Shade tab A associated with at least one of the A classes (A1, A2, A3, A3.5, A4) and Shade tab B associated with at least one of the B classes (B1, B2, B3, B4), but this is only an example, and it will be understood that one or more hyperplanes may be set to individually distinguish between the A classes (A1, A2, A3, A3.5, A4) associated with Shade tab A, or one or more hyperplanes may be set to individually distinguish between the B classes (B1, B2, B3, B4) associated with Shade tab B.
[0145] For example, it will be appreciated that at least 15 non-linear hyperplanes can be established for the 16 color distinctions in this specification.
[0146] FIG. 8 is a flowchart illustrating a process for acquiring first tooth photograph information in a predetermined illuminance range according to an embodiment of the present invention.
[0147] Referring to FIGS. 1 to 8, at least one user terminal (810) of FIG. 8 may correspond to at least one user terminal (e.g., 210 of FIG. 2) mentioned above.
[0148] For example, the user terminal (810) can be understood as a wireless communication device in which an application associated with the tooth shade diagnosis system (200) is pre-installed.
[0149] Additionally, the action server device (820) of FIG. 8 may correspond to the action server device mentioned above (e.g., 220 of FIG. 2).
[0150] For example, the operation server device (820) can be understood as a cloud server linked with at least one user terminal (810).
[0151] In step S810, at least one user terminal (810) can determine whether the illuminance of the location where the user is located falls within a predetermined illuminance range. Here, the predetermined illuminance range may be 250 to 300 lux.
[0152] Specifically, at least one user terminal (810) may be implemented to determine whether an illuminance value measured by a pre-equipped illuminance sensor at a location where the user is located in order to obtain first tooth photograph information corresponds to a pre-determined illuminance range.
[0153] If the measured illuminance value is determined to not fall within the predetermined illuminance range, the procedure proceeds to step S820.
[0154] Meanwhile, if the measured illuminance value falls within a predetermined illuminance range, the procedure proceeds to step S830.
[0155] For example, when the measured illuminance value is determined to fall within a predetermined illuminance range, a shooting button may be pre-activated through a pre-installed application on at least one user terminal (810).
[0156] At step S820, at least one user terminal (810) may be implemented to generate a message instructing the user to adjust the shooting lighting or shoot from a different location.
[0157] At step S830, at least one user terminal (810) can generate first tooth photo information.
[0158] For example, the first tooth photo information can be acquired when the activated shooting button is pressed after the user has adjusted the camera module (not shown) of at least one user terminal (810) so that the user's teeth are clearly visible.
[0159] The first tooth photo information according to the present embodiment can be transmitted to the linked operation server device (820).
[0160] Accordingly, in the operating server device (820) according to the present embodiment, image information regarding the teeth of a user of at least one user terminal (810) corresponding to a predetermined illuminance range can be accumulated and managed in the form of a database (DB).
[0161] The method of collecting users' dental images through existing applications was implemented without considering the types of user terminals, the performance of the camera module (not shown) equipped in the user terminal, and the variety of lighting environments depending on the shooting location, so there were limitations in utilizing the collected dental image information for the administrator.
[0162] In contrast, according to the present embodiment, since image information on the teeth of users corresponding to a predetermined illuminance range is collected, it will be understood that this is advantageous not only for securing data for artificial intelligence learning on the teeth but also for building and improving an artificial intelligence learning model.
[0163] FIG. 9 is a flowchart for explaining the operation method of the artificial intelligence-based tooth shade diagnosis system according to the present embodiment.
[0164] Referring to FIGS. 1 to 9, in step S910, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can individually obtain first tooth photograph information captured in a predetermined illuminance range from at least one user terminal (e.g., 210 of FIG. 2).
[0165] For example, the first tooth photograph information (e.g., 4000 of FIG. 4) may include a tooth image photographed to show multiple teeth of at least one user's face and an image of a gray card (e.g., 40 of FIG. 4) on which multiple reference colors are displayed.
[0166] At step S920, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can perform preprocessing on the first tooth photo information (e.g., 4000 of FIG. 4) to generate the second tooth photo information (e.g., 6000 of FIG. 6).
[0167] In step S930, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can generate color conversion data based on first color data obtained from a plurality of reference colors (e.g., 600 of FIG. 6) included in second tooth photo information (e.g., 6000 of FIG. 6) and preset reference color data for the plurality of reference colors (e.g., 600 of FIG. 6).
[0168] For example, the first color data can be generated based on first cropped image information (e.g., IMG_c1 in FIG. 6) obtained from at least one first location (e.g., P1 in FIG. 6) associated with a plurality of reference colors (e.g., 600 in FIG. 6) included in a gray card (e.g., 60 in FIG. 6).
[0169] For example, color conversion data can be determined based on the difference between first color data generated from first cropped image information (e.g., IMG_c1 in FIG. 6) and preset reference color data for a plurality of reference colors (e.g., 600 in FIG. 6).
[0170] At step S940, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can apply color conversion data to second tooth photo information (e.g., 6000 of FIG. 6) to generate second color data.
[0171] For example, the second tooth photo information (e.g., 6000 of FIG. 6) may correspond to the second cropped image information (e.g., IMG_c2 of FIG. 6) obtained from at least one predetermined second location (e.g., P2 of FIG. 6) to determine a tooth color for at least one user terminal (e.g., 210 of FIG. 2).
[0172] In this case, it will be understood that the second color data can be obtained from image information obtained by applying color conversion data to second cropped image information (e.g., IMG_c2 in FIG. 6) obtained from at least one second location (e.g., P2 in FIG. 6).
[0173] However, the present specification is not limited thereto, and the second color data may be implemented so that color conversion data is first applied to the entire second tooth photograph information, and then second cropped image information acquired from at least one predetermined second location is generated. In other words, the process of generating the second color data may be implemented in various ways depending on the implementation method.
[0174] At step S950, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can input second color data into a pre-learned tooth shade learning model to generate prediction information indicating a tooth whitening grade for at least one terminal.
[0175] For example, the tooth shade learning model according to the present embodiment can be implemented based on a support vector machine (hereinafter referred to as 'SVM').
[0176] In this case, the support vector machine (SVM) for the tooth shade learning model according to the present embodiment can be implemented based on a plurality of pre-determined hyperplanes according to artificial intelligence supervised learning.
[0177] For example, a tooth whitening grade may be associated with at least one of a plurality of class information representing the tooth shade described above through FIG. 1. Additionally, the plurality of class information may be implemented to be individually associated with a plurality of hyperplanes.
[0178] For example, the process of generating predictive information indicating a tooth whitening grade may be as follows.
[0179] First, the tooth shade learning model according to the present embodiment can be implemented to specify one class information among a plurality of class information based on the correlation between the second color data and a plurality of hyperplanes set through pre-learning.
[0180] Next, the tooth shade learning model according to the present embodiment can be implemented to include specific class information in the prediction information.
[0181] In step S960, the artificial intelligence-based tooth shade diagnosis system according to the present embodiment (e.g., 200 of FIG. 2) can be implemented so that prediction information is transmitted from an operation server device (e.g., 220 of FIG. 2) to at least one user terminal (e.g., 210 of FIG. 2).
[0182] FIG. 10 and FIG. 11 are conceptual diagrams for explaining the learning and prediction process of a tooth shade learning model for an artificial intelligence-based tooth shade diagnosis system according to an embodiment of the present invention.
[0183] Referring to (a) of Fig. 10, for the implementation of a support vector machine (SVM) corresponding to an artificial intelligence supervised learning algorithm, the learning data of the tooth shade learning model according to the present embodiment can be expressed in two dimensions as in (a) of Fig. 10.
[0184] For example, the learning data of the tooth shade learning model of Fig. 10 (a) can be expressed so as to correspond to the CIE a value of the CIE L*a*b data on the X-axis and to correspond to the CIE b value of the CIE L*a*b data on the Y-axis.
[0185] In (a) of Fig. 10, Shade tab A associated with at least one of the A classifications (A1, A2, A3, A3.5, A4) among the 16 vitamin color classifications and Shade tab B associated with at least one of the B classifications (B1, B2, B3, B4) may be partially displayed.
[0186] Referring to (a) of FIG. 10, it may be difficult to set one or more hyperplanes corresponding to decision boundaries for non-linear classification required for the tooth shade learning model according to the present embodiment.
[0187] Accordingly, for the tooth shade learning model according to the present embodiment, a mapping function ( ) can be defined as in the following mathematical expression 1.
[0188]
[0189] In other words, for the two-dimensional tooth shade learning model of Fig. 10 (a), the mapping function of mathematical expression 1 is applied to the given learning data ( ) may be applied.
[0190] Referring to (b) of Fig. 10, when the given learning data for the tooth shade learning model is expressed in three dimensions, one or more nonlinear hyperplanes (hp) can be set.
[0191] That is, it will be understood that a Soft-Margin Non-linear SVM as in FIG. 7 can be applied to the tooth shade learning model according to the present embodiment.
[0192] Meanwhile, given the above mathematical expression 1, the mapping function for the tooth shade learning model implemented with Soft-Margin Non-linear SVM ( ) can be optimized based on the following mathematical expression 2.
[0193]
[0194] Here, in mathematical expression 2 And, And, And, It can correspond to an nxn matrix.
[0195] According to the application of the above mathematical expressions 1 and 2, the mapping function ( ) as well as the computational amount itself. ) increases the amount of calculation.
[0196] To reduce this amount of computation, it will be understood that a kernel (k) such as the following mathematical expression 4 can be used. For a clear and concise understanding of this specification, it can be implicitly assumed that the kernel (k) has a relationship such as the following mathematical expression 3.
[0197]
[0198] For example, a Radial Basis Function (hereinafter, 'RBF') kernel as in mathematical equation 4 below can be applied to the kernel (k).
[0199]
[0200] Meanwhile, in the optimization process of mathematical expression 2, formulas for the weight (w) of mathematical expression 5 below and the bias (b) of mathematical expression 6 below can be obtained.
[0201]
[0202]
[0203] Here, in mathematical expressions 5 and 6, and can correspond to the two-dimensional coordinates of the given training data or the two-dimensional coordinates of the input variables.
[0204] Meanwhile, the tooth shade learning model according to the present embodiment implemented with Soft-Margin Non-linear SVM has input variables ( , ) can be implemented to perform prediction according to the following mathematical expression 7.
[0205] For example, input variables ( , ) may correspond to the CIE a value and CIE b value included in the second color data.
[0206]
[0207] In addition, when the kernel (k) is applied in mathematical expression 7, the above mathematical expression 7 can be defined as mathematical expression 8 below.
[0208]
[0209] According to the present embodiment, it will be understood that in the process of training the tooth shade learning model based on mathematical expressions 1 to 8, a plurality of hyperplanes for the tooth shade learning model according to the present embodiment can be set.
[0210] Furthermore, the input variables corresponding to the CIE a and CIE b values included in the second color data ( , ) is input into the tooth shade learning model, it will also be understood that prediction information indicating a tooth whitening grade for at least one terminal (e.g., 210 of FIG. 2) can be generated based on the correlation according to Equations 1 to 8.
[0211] Referring to FIG. 11, a tooth shade learning model according to an embodiment of the present invention is illustrated in a three-dimensional space for distinguishing 16 vitamin colors. The X, Y, and Z axes of FIG. 11 may correspond to CIE L values, CIE a values, and CIE b values.
[0212] Meanwhile, although the nonlinear hyperplane is not illustrated in FIG. 11, it can be assumed that the nonlinear hyperplane is preset for the tooth shade learning model in three-dimensional space according to the present embodiment.
[0213] According to the present embodiment, the CIE L value can be obtained in advance during the process of receiving the first tooth photograph information as a value falling within a predetermined illuminance range. That is, the input variables corresponding to the CIE a value and the CIE b value included in the second color data on the tooth shade learning model in the three-dimensional space of FIG. 11 , ) is input to the tooth shade learning model implemented in the three-dimensional space of Fig. 11, it will be understood that prediction information indicating a tooth whitening grade associated with any one of the 16 vitamin color classifications (i.e., multiple class information) can be generated based on the correlation according to the above-described mathematical expressions 1 to 8.
[0214] FIG. 12 is a block diagram illustrating a computing environment including a computing device suitable for use in exemplary embodiments.
[0215] In the illustrated embodiment, each component may have different functions and capabilities other than those described below, and may include additional components other than those described below.
[0216] The illustrated computing environment (10) includes a computing device (12). In one embodiment, the computing device (12) may correspond to an operation server device (e.g., 220 of FIG. 2) referred to herein.
[0217] A computing device (12) includes at least one processor (14), a computer-readable storage medium (16), and a communication bus (18). The processor (14) may cause the computing device (12) to operate according to the exemplary embodiments mentioned above. For example, the processor (14) may execute one or more programs stored in the computer-readable storage medium (16). The one or more programs may include one or more computer-executable instructions, which, when executed by the processor (14), may be configured to cause the computing device (12) to perform operations according to the exemplary embodiments.
[0218] A computer-readable storage medium (16) is configured to store computer-executable instructions or program code, program data, and / or other suitable forms of information. A program (20) stored in the computer-readable storage medium (16) includes a set of instructions executable by the processor (14). In one embodiment, the computer-readable storage medium (16) may be a memory (volatile memory such as random access memory, non-volatile memory, or a suitable combination thereof), one or more magnetic disk storage devices, optical disk storage devices, flash memory devices, any other form of storage medium that can be accessed by the computing device (12) and store desired information, or a suitable combination thereof.
[0219] A communication bus (18) interconnects various other components of the computing device (12), including the processor (14) and computer-readable storage media (16).
[0220] The computing device (12) may also include one or more input / output interfaces (22) and one or more network communication interfaces (26) that provide interfaces for one or more input / output devices (24). The input / output interfaces (22) and the network communication interfaces (26) are connected to a communication bus (18).
[0221] The input / output device (24) may correspond to at least one terminal (e.g., 210 of FIG. 2) mentioned herein, and may be connected to other components of the computing device (12) via the input / output interface (22). Exemplary input / output devices (24) may include input devices such as a pointing device (such as a mouse or a trackpad), a keyboard, a touch input device (such as a touchpad or a touchscreen), a voice or sound input device, various types of sensor devices and / or photographing devices, and / or output devices such as a display device, a printer, a speaker, and / or a network card. The exemplary input / output device (24) may be included within the computing device (12) as a component constituting the computing device (12), or may be connected to the data management server (110) as a separate device distinct from the computing device (12).
[0222] While the embodiments of the present invention have been described above with reference to the attached drawings, those skilled in the art will appreciate that the present invention can be implemented in other specific forms without altering the technical spirit or essential characteristics thereof. Therefore, the embodiments described above should be understood to be illustrative in all respects and not restrictive.
[0223] While the detailed description of this specification has described specific embodiments, various modifications are possible without departing from the scope of this specification. Therefore, the scope of this specification should not be limited to the above-described embodiments, but should be determined not only by the claims set forth below but also by equivalents of the claims of this invention.
Claims
1. In an operating method for a tooth shade diagnosis device based on artificial intelligence performed by an operating server device, Individually acquire first tooth photograph information taken in a predetermined illuminance range from at least one user terminal, A step in which the first tooth photograph information includes an image of a gray card on which a plurality of reference colors are displayed and an image of a tooth part; A step of generating second tooth photo information by performing preprocessing on the first tooth photo information; A step of generating color conversion data based on first color data obtained from the plurality of reference colors included in the second tooth photograph information and preset reference color data for the plurality of reference colors; A step of generating second color data by applying the color conversion data based on the second tooth photo information; A step of inputting the second color data into a pre-learned tooth shade learning model to generate prediction information indicating a tooth whitening grade for at least one terminal; and A method comprising the step of transmitting the above prediction information to at least one user terminal.
2. In paragraph 1, A method wherein the first color data is generated based on first cropped image information obtained from at least one first location associated with the plurality of reference colors.
3. In paragraph 1, A method wherein the second tooth photograph information corresponds to second cropped image information obtained from at least one second location determined in advance to determine a tooth color for at least one user terminal.
4. In paragraph 1, A method in which the above tooth shade learning model is implemented based on a support vector machine based on a plurality of predetermined hyperplanes.
5. In paragraph 4, The above tooth whitening grade is associated with any one of a plurality of class information for representing tooth shade, and A method in which the above plurality of class information is implemented to be individually associated with the above plurality of hyperplanes.
6. In paragraph 5, The step of generating prediction information indicating the above tooth whitening grade is: A step of specifying one of the plurality of class information based on a correlation between the second color data and the plurality of hyperplanes; and A method comprising the step of including the specified class information in the prediction information.
7. In paragraph 1, A method further comprising a step of determining whether the predetermined illuminance range is within the at least one user terminal when the capture button for the first tooth photo information is activated on the at least one user terminal.
8. In paragraph 7, A method implemented such that when it is determined by at least one user terminal that the predetermined illuminance range is exceeded, the generation of the first tooth photo information is skipped and a message is generated instructing the at least one user terminal to adjust the shooting illuminance or take the photo at a different location.
9. In paragraph 7, A method implemented so that the first tooth photo information is generated when it is determined by at least one user terminal that the predetermined illuminance range is within the predetermined illuminance range.
10. At least one user terminal generating first tooth photograph information captured in a predetermined illuminance range; and Including an operation server device linked to the above user terminal, The above-mentioned operating server device, It is implemented to individually obtain the first tooth photo information from at least one user terminal, The above first tooth photograph information includes an image of a gray card on which a plurality of reference colors are displayed and a tooth image captured by at least one terminal, It is implemented to perform preprocessing on the above first tooth photo information to generate second tooth photo information, It is implemented to generate color conversion data based on first color data obtained from the plurality of reference colors included in the second tooth photo information and preset reference color data for the plurality of reference colors, It is implemented to generate second color data by applying the color conversion data based on the second tooth photo information, It is implemented to input the second color data into a pre-learned tooth shade learning model to generate prediction information indicating a tooth whitening grade for at least one terminal, and An artificial intelligence-based tooth shade diagnosis system implemented to transmit the above prediction information to at least one user terminal.
Citation Information
Patent Citations
Method and System for selecting a teeth shade
KR102056008B1
Dental diagnosis system using big data-based artificial intelligence
KR102284579B1
Method of manufacturing transparent and electrically conductive organic film and conductive organic film and devices manufactured by the method
KR102353552B1
Artificial intelligence-based tooth shade diagnostic system and operation method therefor
KR102563621B1
System of a dental blank and an associated shade guide
US20200315761A1
