Object evaluation method using biofluorescence image-based color analysis and system therefor

Abstract

The present invention relates to a method for evaluating the state of a specific object, and more particularly, to a method for evaluating a specific object or target by utilizing biofluorescence image-based multi-chromatic analysis, and a system therefor.

Description

Object evaluation method using color analysis based on biofluorescence images and system therefor

[0001] The present invention relates to a method for evaluating the state of a specific object, and more particularly, to a method for evaluating a specific object or target using multi-chromatic analysis based on biofluorescence images, and a system therefor.

[0002] Various methods are used to assess the condition of an arbitrary object, one of which is analyzing the color observed from the object's exterior. If the color of a specific object can be defined when it is in good condition and the color of a specific object when it is in poor condition, it would be possible to assess the condition of any arbitrary object of the same type. The present invention was proposed in response to this need.

[0003] For example, oral hygiene has a significant impact on overall health, and regular checkups and management are crucial for preventing oral diseases. However, existing oral hygiene assessment methods often rely on the subjective judgment of experts, primarily on the visible appearance and color of teeth, and thus lack accuracy and objectivity. Recently, biofluorescence imaging technology has been applied. This technology observes the phenomenon (fluorescence phenomenon) in which substances produced during the metabolic processes of microorganisms present in the object (e.g., microorganisms that form dental plaque) absorb light when illuminated with a light source of a certain wavelength and then emit a unique light beam different from the light from the initial light source. This has enabled oral hygiene assessment and oral disease diagnosis with considerable accuracy. However, a system capable of systematically analyzing dental images is not yet available, and this needs to be supplemented.

[0004] The present invention has been proposed in consideration of such problems, and relates to a method and system for quickly and accurately determining the state of an object by performing multi-color analysis based on a biofluorescence image.

[0005] The purpose of the present invention is to provide a method and system for objectively evaluating the condition of an object by performing color analysis based on a biofluorescence image. In particular, conventional analysis methods based on biofluorescence images have had problems in that the analysis results differ depending on the researcher evaluating them due to difficulties in automatic analysis, and slightly different values ​​are presented each time the analysis is performed, which has caused inconvenience in actual clinical practice. The purpose of the present invention is to automate the analysis process, and to complement the difficulty in evaluating dark and bright areas of color due to the use of only RGB values ​​in the past by comprehensively evaluating Lab values ​​as well.

[0006] Specifically, the present invention aims to quantitatively analyze the color information of a biofluorescence image to objectively evaluate the condition of an object, for example, the oral hygiene condition such as dental calculus, dental plaque, cavities, and gum inflammation in the case of the oral cavity.

[0007] In addition, the present invention aims to provide users with accurate and detailed information on objects, such as the condition of teeth and gums, based on analysis results, thereby enabling personalized oral care.

[0008] In addition, the present invention aims to provide analysis results intuitively so that users can visually check and understand objects, for example, the condition of the oral cavity, thereby motivating them to take care of their oral cavity.

[0009] In addition, the present invention aims to build a system that can evaluate the hygiene condition of an object, for example, an oral cavity, through a biofluorescence image without specialized knowledge, thereby making it easy to manage oral health in various environments such as home, school, and work.

[0010] Meanwhile, the technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned can be clearly understood by those skilled in the art from the description below.

[0011] The present invention is intended to solve the above problems, and a method for evaluating an object using color analysis based on a biofluorescence image by a computing device including a central processing unit and a memory according to the present invention may include the steps of: receiving a biofluorescence image in which an arbitrary object is photographed; obtaining RGB values ​​and Lab values ​​for an evaluation area within the biofluorescence image; and performing color analysis based on the obtained RGB values ​​and Lab values, and displaying the color analysis result for the object on a specific coordinate system.

[0012] In addition, the object evaluation method using the color analysis may further include, after the step of receiving the biofluorescence image, a step of identifying only an evaluation area from within the biofluorescence image.

[0013] Additionally, in the object evaluation method using the color analysis, the color analysis may include a process of comparing the acquired color information with a previously generated color cluster to determine a color cluster to which a plurality of points within the evaluation area belong.

[0014] In addition, in the object evaluation method using the color analysis, the coordinate system may be characterized in that the ratio of the G value to the R value is set as the x-axis, and the L value is set as the y-axis.

[0015] Additionally, in the object evaluation method using the color analysis, the coordinate system may be characterized by including a plurality of color clusters defined in advance.

[0016] In addition, in the object evaluation method using the color analysis, the step of displaying the color analysis result may be characterized by displaying points mapped to a plurality of points within the evaluation area on the coordinate system.

[0017] In addition, in the object evaluation method using the color analysis, the step of displaying the color analysis result may further include a step of determining which color clusters the plurality of mapped points belong to, and calculating the ratio of the color clusters to which the plurality of points belong.

[0018] In addition, in the object evaluation method using the color analysis, the step of displaying the color analysis result may further include a step of determining which risk group the object belongs to among the high-risk group, the medium-risk group, or the low-risk group, based on the ratio of the color clusters to which the plurality of points belong.

[0019] Meanwhile, according to another embodiment of the present invention, a method for clustering for color analysis based on a biofluorescence image, which comprises a central processing unit and a memory, may include the steps of: collecting a plurality of biofluorescence images; identifying only an evaluation area to be analyzed for each of the biofluorescence images; obtaining RGB values ​​and Lab values ​​for the evaluation area within the biofluorescence images; and displaying all color distributions corresponding to the evaluation area on a coordinate system based on the obtained RGB values ​​and Lab values; determining the number of clusters and determining a representative value of each cluster; and displaying RGB values ​​corresponding to the representative value of each cluster on a coordinate system.

[0020] Meanwhile, in a system including a central processing unit and a memory according to another embodiment of the present invention, the central processing unit is characterized in that it executes commands for executing an object evaluation method stored in the memory, wherein the object evaluation method may include the steps of: receiving a biofluorescence image in which an arbitrary object is photographed; obtaining RGB values ​​and Lab values ​​for an evaluation area within the biofluorescence image; and performing color analysis based on the obtained RGB values ​​and Lab values, and displaying the color analysis result for the object on a specific coordinate system.

[0021] Meanwhile, according to another embodiment of the present invention, a method for supporting medical treatment by a processing device including a central processing unit and a memory includes the steps of: (a) specifying a patient to be treated; (b) providing medical treatment support information for the patient; wherein the medical treatment support information includes an oral image and color analysis information of the patient, wherein the oral image includes a biofluorescence image, and the color analysis information is color analysis information based on a biofluorescence image.

[0022] In addition, in a treatment support system including a central processing unit and a memory according to another embodiment of the present invention, the central processing unit is characterized in that it executes commands for executing a treatment support method stored in the memory, wherein the treatment support method includes: (a) a step of specifying a patient to be treated; (b) a step of providing treatment support information for the patient; and the treatment support information includes an oral image and color analysis information of the patient, wherein the oral image includes a biofluorescence image, and the color analysis information is color analysis information based on a biofluorescence image.

[0023] According to the present invention, there is an effect that enables objective and quantitative evaluation of an object through color analysis based on a biofluorescence image.

[0024] In particular, it has the effect of moving away from the existing subjective evaluation method in evaluating oral hygiene status and enabling accurate identification of the presence and severity of various oral diseases such as tartar, dental plaque, cavities, and gum inflammation.

[0025] In addition, the present invention provides analysis results in a user-friendly manner, allowing users to intuitively understand their oral condition, ultimately increasing users' awareness of oral health and encouraging active participation in oral care, thereby contributing to the prevention of oral diseases.

[0026] Meanwhile, the effects of the present invention are not limited to those mentioned above, and other technical effects not mentioned can be clearly understood by those skilled in the art from the description below.

[0027] Figure 1 is a conceptual diagram illustrating an overview of an object evaluation method and system according to the present invention.

[0028] Figure 2 sequentially lists one embodiment of an object evaluation method according to the present invention.

[0029] Figure 3 illustrates a coordinate system utilized in an object evaluation method according to the present invention.

[0030] Figure 4 illustrates the oral hygiene status of a random patient as expressed through a coordinate system, and depicts the status of a patient whose oral hygiene status belongs to a high-risk group.

[0031] Figure 5 illustrates the appearance of a patient whose oral hygiene status falls into the medium-risk group.

[0032] Figure 6 illustrates the appearance of a patient whose oral hygiene status belongs to the low-risk group.

[0033] Figure 7 sequentially lists a pre-clustering method for color analysis according to another embodiment of the present invention.

[0034] Figure 8 illustrates a medical treatment support system according to another embodiment of the present invention.

[0035] FIG. 9 illustrates an embodiment of a treatment support method according to another embodiment of the present invention.

[0036] Figures 10 to 13 illustrate examples of interfaces provided to a user (diagnostician) by the medical treatment support system according to the present invention.

[0037]

[0038] The purpose, technical configuration, and resulting operational effects of the present invention will be more clearly understood through the following detailed description based on the drawings attached to the specification of the present invention. Reference will now be made to the accompanying drawings, which will further describe embodiments of the present invention.

[0039] The embodiments disclosed herein should not be construed or used to limit the scope of the present invention. Those skilled in the art will readily appreciate that the descriptions herein, including the embodiments, have a wide range of applications. Therefore, any embodiments described in the detailed description of the present invention are intended to serve as illustrative examples to better illustrate the present invention and are not intended to limit the scope of the present invention to the embodiments.

[0040] The functional blocks depicted in the drawings and described below are merely examples of possible implementations. Other implementations may utilize other functional blocks without departing from the spirit and scope of the detailed description. Furthermore, while one or more functional blocks of the present invention are depicted as individual blocks, one or more of the functional blocks of the present invention may be a combination of various hardware and software configurations that perform the same function.

[0041] Additionally, the expression “including certain components” is an “open” expression, simply indicating the presence of those components, and should not be construed as excluding additional components.

[0042] Furthermore, when it is said that a component is “connected” or “connected” to another component, it should be understood that it may be directly connected or connected to that other component, but there may also be other components in between.

[0043]

[0044] Figure 1 is a diagram for understanding the overall outline of an object evaluation method and system according to one embodiment of the present invention. The system may basically include a computing device (100). When a biofluorescence image of a specific object is input, the computing device (100) performs multi-chromatic analysis and outputs an evaluation result regarding the condition of the object, as shown in the drawing.

[0045] For example, when an image of a patient's teeth taken by a biofluorescence imaging device at a dental clinic is input into a computing device (100), the method according to the present invention can be utilized to segment the teeth and gum areas, analyze the color information of the tooth area to be evaluated, and evaluate the presence and severity of tartar, dental plaque, and caries. The computing device (100) provides the evaluation results to a dentist, and the dentist can establish a customized diagnosis and treatment plan for the patient based on the results. For example, if tartar is severe, scaling can be recommended, or if caries is suspected, additional tests can be performed.

[0046] As another example, the present invention can be implemented so that a general consumer or food inspector can input an image of food taken with a biofluorescence imaging device to a processing unit (100) to check the freshness of the food, and view the evaluation results. The method according to the present invention analyzes the color information of the food to evaluate the freshness, degree of spoilage, etc., and provides the results to the consumer or inspector. The consumer or inspector can make more informed choices when purchasing food by referring to these evaluation results. For example, if the freshness of fruit is evaluated as low, the purchase and sale can be withheld, or if the degree of spoilage of meat is evaluated as high, the decision can be made whether to discard it.

[0047] In this way, the present invention is basically configured to provide a user with the results of color analysis when a biofluorescence image of an object to be evaluated exists. Below, the process through which a system including a computing device (100) performs evaluation will be discussed in more detail.

[0048]

[0049] FIG. 2 is a flowchart sequentially listing an object evaluation method using color analysis according to one embodiment of the present invention. Referring to FIG. 2, an embodiment of the present invention will be described in detail.

[0050] Before going into a detailed explanation, it should be understood that the object evaluation method according to the present invention can be executed by a computing device (100) equipped with a central processing unit and memory. The types of such computing devices can include both portable terminals such as smart phones, PDAs, and tablet PCs, and terminals that are fixedly placed in a certain location such as desktop PCs. The central processing unit may also be called a controller, a microcontroller, a microprocessor, a microcomputer, etc. In addition, the central processing unit may be implemented by hardware, firmware, software, or a combination thereof. When implemented using hardware, the central processing unit may be implemented by an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), etc., and when implemented using firmware or software, the firmware or software may be configured to include modules, procedures, or functions that perform the functions or operations described above. Additionally, the memory can be implemented as ROM (Read Only Memory), RAM (Random Access Memory), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory, SRAM (Static RAM), HDD (Hard Disk Drive), SSD (Solid State Drive), etc.

[0051] In some cases, the above-described computing device (100) may be a server, and in this case, the server may be a device that stores and executes a set of commands, i.e., a program for actually implementing the object evaluation method according to the present invention. The server may be in the form of at least one server PC managed by a specific user, or in the form of a cloud server provided by another company, i.e., a cloud server that a user can use by registering as a member.

[0052] Additionally, in some cases, the method according to the present invention may be executed on a cluster system comprised of multiple computational units rather than a single computational unit (100). Within the cluster system, multiple computational units required to execute the object evaluation method may be configured to perform different operations, respectively.

[0053] Below, we will look at various related embodiments with reference to the drawings.

[0054]

[0055] Referring to the drawing, the object evaluation method according to the present invention starts from the step (S101) in which the computing device (100) receives a biofluorescence image from a user or an external device. A biofluorescence image refers to an image captured in which microorganisms (bacteria), etc. existing on an object exhibit a fluorescence phenomenon when light of a specific wavelength selected from the visible light range is irradiated toward the object. For example, a porphyrin substance generated during the metabolic process of a microorganism can exhibit a fluorescence phenomenon in blue light of 380 to 420 nm, and an image captured in which this phenomenon is captured can be defined as a biofluorescence image. The process of capturing the biofluorescence image may further include a process such as passing it through a color filter, but since this is not a matter for detailed discussion in this detailed description, it will be briefly described here.

[0056] Meanwhile, the images received at this stage are not limited to biofluorescent images of teeth. For example, biofluorescent images captured can be utilized to assess the freshness of food. Biofluorescent images acquired from food can detect moisture content and texture changes within the food, providing freshness information that is difficult to detect with the naked eye. Biofluorescent images of food can also be received at this stage. Furthermore, biofluorescent images captured to analyze skin condition can also be received.

[0057] Meanwhile, the computing device (100) can receive biofluorescence images in various ways. For example, a user can capture a biofluorescence image using a mobile device such as a smartphone or tablet PC and transmit it to the computing device via a wireless communication method such as Bluetooth or Wi-Fi. Alternatively, a dedicated biofluorescence imaging device can be connected to the computing device (100) via a wired connection to directly transmit the image.

[0058]

[0059] After step S101, the computing device (100) can perform an operation (S102) to identify only the area to be evaluated from the image, i.e., the evaluation area. If the object to be evaluated is a tooth, the biofluorescence image is likely to be an image that captures the inside of the oral cavity. In this case, the computing device (100) can perform an operation to identify only the tooth surface as the evaluation area, thereby enabling an accurate evaluation of the object.

[0060] Various algorithms can be utilized in this step. For example, since teeth and gums have relatively distinct color differences, a color-based segmentation algorithm can be utilized to separate the two regions using color information. Specifically, this algorithm analyzes the color distribution of teeth and gums in various color spaces, such as RGB, HSV, and Lab, and uses thresholding or clustering techniques to distinguish the two regions. Furthermore, morphological algorithms can be utilized to emphasize the shape of the teeth and clarify the boundary with the gums. For example, erosion operations can be used to remove subtle surface irregularities on the tooth surface, and dilation operations can be used to expand the tooth region and clarify its boundary with the gums, thereby identifying the evaluation region. Another example is an edge detection algorithm that can be implemented to identify the boundary between the teeth and gums. Edge detection algorithms seek areas with significant brightness variations within an image. Because the boundary between teeth and gums has a significant brightness difference, edge detection can effectively separate the two regions. Active contour algorithms can also be utilized. This algorithm is characterized by setting an initial outline and extracting the boundary of an object by transforming the outline based on the features of the image. By setting the rough shape of the tooth as the initial outline and transforming the outline to fit the boundary of the tooth using information such as color and brightness of the biofluorescence image, the evaluation area can be identified.

[0061] Meanwhile, it is most preferable to segment the tooth region in the biofluorescence image using a Deep Learning-based Segmentation algorithm, more specifically, a deep learning algorithm such as U-Net or Mask R-CNN. The deep learning algorithm learns the characteristics of teeth and gums through a large amount of learning data, and based on this, can accurately segment the tooth region in a new image. As will be described later, the present invention can be implemented to utilize the results of analyzing a large amount of biofluorescence images that have been databased in advance for coordinate system creation and deep learning learning, so it is most preferable to use such a learning-type algorithm when identifying the evaluation region.

[0062] Meanwhile, the computational unit can accurately identify evaluation areas in biofluorescence images using the various algorithms mentioned above, either alone or in combination. For example, color-based segmentation can be used to roughly separate teeth and gums, and then edge detection or an active contour model can be applied to more accurately extract the boundaries. Furthermore, deep learning-based segmentation models can be used to improve the initial segmentation results, or morphological operations can be applied to post-process the segmentation results.

[0063]

[0064] After step S102, the computing device (100) can execute step S103 of acquiring color information of an evaluation area within a biofluorescence image. The color information may include an RGB value and a Lab value. The RGB value is a value that expresses color with a combination of three color components: red (R), green (G), and blue (B), and the Lab value is a value that expresses color with three values: brightness (L), green-red axis (a), and blue-yellow axis (b). At this time, the RGB value includes all values ​​that can be detected in the image in which the tooth was previously identified. In addition, the Lab value has the advantage of being designed to be similar to human color perception, so that it can express color differences more accurately.

[0065] Meanwhile, step S103 may be followed by a step (S104) of performing color analysis, i.e., multi-color analysis, based on the color information. More precisely, this step can also be understood as a process of mapping the color information obtained from the evaluation area to a unique coordinate system.

[0066] Specifically, in step S104, the calculation device (100) can perform an operation to map all color information that can be extracted from the evaluation area on a predefined coordinate system in which the x-axis is the a value (the degree of green-red) among Lab values ​​and the y-axis is the L value (brightness) among Lab values.

[0067]

[0068] Figure 3 illustrates an example of a coordinate system utilized in the present invention. Figure 3 illustrates the results of analyzing an evaluation area (teeth surface) of a specific object (inside a patient's oral cavity) on a unique coordinate system proposed by the present invention.

[0069] Referring to FIG. 3, the coordinate system is a two-dimensional coordinate system in which the x-axis is the a value (the degree of green-red) among Lab values, and the y-axis is the L value (brightness) among Lab values, as mentioned above. It can be confirmed that the entire color distribution (T) of the evaluation area is displayed in pixel units on the coordinate system based on the color information obtained from the evaluation area. That is, the T area indicated on the drawing can also be defined as a set of blue dots, and each blue dot is mapped by a pair of L values ​​and a values ​​obtained based on the color information obtained from the evaluation area (tooth surface). The items displayed in this coordinate system, especially the distribution map of a specific shape, include all values ​​that can be displayed by biofluorescence from the subject (e.g., the patient's oral cavity). From this, the range of L and the distribution of green and red can be confirmed, and in particular, the range of L values ​​can be used as an important indicator for evaluating wear or cracks in teeth in the oral cavity, and can further be used as an important indicator for evaluating enamel and dentin. It is noteworthy that in the multi-color analysis according to the present invention, the B value among RGB values ​​and the b value among Lab values ​​are not used, and the B value or b value can be implemented so that an offset value is set as needed. The exclusion of the B value or b value has the effect of reducing information that is not necessarily necessary for color analysis, and thus has the effect of reducing the computational load of the computational device (100). In addition, the exclusion of the b value among Lab values ​​has a technical significance in that the evaluation is performed only on the fluorescent region generated by blue light.

[0070] Meanwhile, within the entire color distribution (T) area for the evaluation area, a point (Tm) representing the average value of the entire color distribution in the area is displayed, allowing the user to intuitively judge the status of the evaluation area. In addition, a sub-area corresponding to a normal area can be separately searched from the evaluation area, and this can be searched according to conditions set by the user in advance or searched by a learned artificial intelligence algorithm. For the normal area, the average values ​​of the area can be defined as Ga, Ra, and La.

[0071] In addition, if a point in the evaluation area is specified, the color distribution (P) of the point can be displayed as a partial area as in the drawing, and similarly, the average value (Pm) of the partial area for the point can also be displayed.

[0072] Additionally, a separate value display area may exist in the upper right corner of the coordinate system, which displays specific analysis values ​​for the evaluation area. In the drawing, Tm, Pm, and the proportion of the color distribution of a point within the color distribution area of ​​the entire evaluation area can be displayed. The value display area may also display other calculated values. As these values ​​are obtained, various quantitative evaluations can be made. For example, if the difference between the maximum value and the average value within a certain area is placed in the denominator, and the difference between the maximum value and the test value at a point is placed in the numerator, the condition of the teeth can be quantitatively evaluated.

[0073] Meanwhile, it is noteworthy that the coordinate system of the present invention defines arbitrary risk areas consisting of a low-risk area (G), a medium-risk area (Y), and a high-risk area (R), as shown in the drawing. These risk areas are defined as a result of clustering based on a large number of biofluorescence images in advance. As will be mentioned again in the description below, the clustering process can be performed by graphing the L and a values ​​among the Lab values ​​obtained from a large number of biofluorescence images on a coordinate system to indicate all areas that a tooth may have, determining a representative value for each cluster based on a set number of clusters, and indicating the R and G values ​​corresponding to each representative value on the coordinate system. It is noteworthy that RGB values ​​are used in the pre-clustering process, and among these, only the R and G values ​​are used, and the B value is excluded from use here as well.

[0074] Also, for reference, the values ​​displayed in the coordinate system may be displayed as different values ​​even if the same object is photographed depending on the photographing device (camera) used. However, please understand that this detailed description does not take such circumstances into consideration and explains on the premise that the object is photographed with the same photographing device (camera). However, if necessary, if the object is photographed with different photographing devices, a process of normalizing the result values ​​by scaling the obtained values ​​(RGB, Lab) within a reference range may be included.

[0075] With reference to Figure 3, we have looked into the coordinate system utilized in the present invention.

[0076]

[0077] Returning to the description of FIG. 2, after the multi-color analysis is performed on the coordinate system in step S104, a step (S105) of displaying the color analysis results to the user may be executed. This step may be understood as a step in which the calculation device (100) outputs the color analysis results displayed on the coordinate system through a display device (not shown), or may also be understood as a step in which the calculation device (100) displays the color analysis results through a display means of another terminal connected to the network.

[0078] With reference to FIGS. 2 and 3, an object evaluation method according to one embodiment of the present invention was examined.

[0079]

[0080] Figures 4 to 6 illustrate the results of multi-color analysis by case, and are intended to explain how the color analysis results displayed on the unique coordinate system mentioned above can be actually utilized.

[0081] First, Fig. 4 is an example of evaluating the oral hygiene status of a patient belonging to the high-risk group (R). Fig. 4 (a) shows the color analysis result when a biofluorescence image obtained by photographing the patient's oral cavity (front of the teeth) is received. Referring to the color analysis result displayed in the coordinate system, it can be seen that the evaluation area of ​​the patient, i.e., the tooth surface, shows a color distribution skewed toward an area that is generally red. This can be easily inferred from the fact that the red fluorescent area, i.e., the area where microorganisms exist, appears quite wide in the patient's biofluorescence image. When viewed in the coordinate system, the color distribution (T) of the evaluation area (tooth surface) is distributed across the low-risk group area (G), the medium-risk group area (Y), and the high-risk group area (R). However, when examining which area it is distributed the most in, it can be seen that it is overwhelmingly distributed in the high-risk group area (R). Through this, the user (e.g., dentist) can intuitively know that the patient in question is a high-risk patient with considerably poor oral hygiene.

[0082] Fig. 4 (b) illustrates an additional display of the color distribution for a point (point S) specified in a biofluorescence image on a coordinate system. Referring to the drawing, it can be confirmed that the color distribution area corresponding to point S is displayed (light purple area) on the coordinate system, and that the average value for the point is also displayed. Fig. 4 (b) may be an output on the coordinate system when, for example, a user (e.g., a dentist) clicks a point (S) on a biofluorescence image using an input device such as a mouse. In this way, the user can mark specific points on the biofluorescence image and show the color distribution for each point marked to the patient, thereby helping the patient understand the oral hygiene condition.

[0083] Meanwhile, as a noteworthy point in Fig. 4, the patient above has had a prosthesis (implant) placed, and the color distribution corresponding to the prosthesis can be implemented to be displayed in the black area at the bottom of the coordinate system on the coordinate system. In the biofluorescence image, the prosthesis, not the natural teeth, is displayed darkly, and by utilizing this characteristic, it is possible to determine how much area the prosthesis occupies within the evaluation area on the coordinate system, i.e., how much the prosthesis is placed, using only the coordinate system.

[0084] Figure 5 shows an evaluation of the oral hygiene status of a patient whose color distribution area falls significantly in the medium-risk area (Y). When examining the biofluorescence image of the patient, it can be seen that most of the patient's teeth area (evaluation area) is cleanly and well-managed, but there is a small area of ​​plaque (red fluorescence) in the lower right corner. Looking at the coordinate system illustrated in Figure 5 (a), it can be seen that the overall color distribution of the patient mostly falls into the low-risk area (G) and the medium-risk area (Y), but some groups of blue dots are distributed across the high-risk area (R). In other words, due to the presence of the small plaque area mentioned above, the U area can be clearly displayed on the coordinate system in the color analysis results, which allows the user to intuitively understand the patient's condition.

[0085] Figure 5 (b) illustrates the above flag region by specifying a point (point J). In the coordinate system of (b), it can be confirmed that the color distribution of the region corresponding to point J is displayed in a different color (light purple).

[0086]

[0087] Figure 6 shows an evaluation of the oral hygiene status of a patient belonging to the low-risk group (G). As can be seen in the biofluorescence image, the above patient shows no red fluorescence at all, and when looking at the color analysis results, it can be seen that all areas are color-distributed only in the low-risk group area (G) and the medium-risk group area (Y), and there is no color distribution at all in the high-risk group area (R).

[0088] With reference to Figures 4 to 6, we examined examples utilizing the results of multi-color analysis displayed on a coordinate system.

[0089] Meanwhile, color analysis results from multi-color analysis can be implemented to define disease-specific color distribution areas by learning which diseases they correspond to through an AI algorithm. In other words, quantitative evaluations of examination sites are possible, and this can be utilized for supervised or unsupervised learning of AI algorithms. Thus, each time an image of a subject is captured or a diagnosis of a subject's condition is made based on a captured image, the color analysis results can be learned to match which disease.

[0090] Specifically, if the results analyzed in step S104 are provided to a user (e.g., a dentist), and the user has visually observed the inside of an actual patient's oral cavity and determined the disease or diagnosis, the color distribution in the analysis can be mapped to which disease or diagnosis it corresponds. This mapping-based learning can be implemented by an artificial intelligence algorithm. By doing so, when a specific color distribution area is obtained as a test result, it is possible to determine which disease the patient is likely to have or which diagnosis it is likely to receive. This information can then be directly delivered to the user (e.g., a dentist) or the patient, thereby improving the accuracy and speed of overall treatment. In addition, if the color distribution is mapped to a specific disease or diagnosis through learning by the artificial intelligence algorithm, the user or patient can select a specific point or area on the coordinate system and receive the evaluation results for the area. Depending on the implementation, chatbot chatting can be enabled, allowing the user or patient to obtain accurate and easy information about their condition.

[0091]

[0092] Figure 7 illustrates another embodiment of the present invention, sequentially listing pre-clustering methods for color analysis. The embodiment of the present invention will be described in detail with reference to Figure 7.

[0093] The clustering method first includes a step (S201) of collecting a plurality of biofluorescence images. In this detailed description, to facilitate understanding of the invention, an example using images captured using a dental biofluorescence imaging device to evaluate the condition of teeth in the oral cavity is described. For example, the computing device (100) can collect images captured using various types of biofluorescence imaging devices, such as a device for examining dental caries activity, a device for examining dental plaque activity, and a device for detecting tartar.

[0094] After step S201, the computing device (100) can identify only the evaluation area to be analyzed for each of the collected biofluorescence images (S202). The evaluation area refers to a specific area in the biofluorescence image for which the oral condition is to be evaluated through color analysis. For example, in the case of a tooth biofluorescence image, the tooth surface may be set as the evaluation area. The evaluation area may be directly designated by the user or automatically extracted through an image processing technique. It should be noted that this step does not necessarily have to be performed by the computing device (100) and may also be performed by utilizing an external service server.

[0095] After step S202, the computing device (100) can obtain RGB values ​​and Lab values ​​for the evaluation area in the biofluorescence images (S203). The RGB values ​​represent the values ​​of the red (R), green (G), and blue (B) components of each pixel, and the Lab values ​​represent the brightness (L), green-red axis (a), and blue-yellow axis (b) values ​​of each pixel. The RGB value is the most basic way to express the color of an image, and the Lab value has the advantage of being able to express color differences more accurately because it is designed to be similar to human color perception. As briefly mentioned above, one of the features of the present invention is that the use of the B value among the RGB values ​​is excluded, and the use of the b value among the Lab values ​​is excluded. In addition, an offset value may be set for the B value or b value in order to help the user's visual understanding on the coordinate system, but except for such cases, the B value or b value will not contribute to the actual analysis process.

[0096] After step S203, the calculation device (100) can display all color distributions corresponding to the evaluation area on a coordinate system based on the acquired RGB values ​​and Lab values ​​(S204). To be precise, in this step, among the Lab values ​​of each pixel, the L and a values ​​are expressed on a two-dimensional coordinate system, and all of these points are displayed to visualize the color distribution.

[0097] After step S204, the computing device (100) can determine the number of clusters and the representative value of each cluster (S205). Clustering is a process of grouping pixels with similar colors. Although the clustering according to the present invention utilizes the K-means clustering algorithm, it is understood that this algorithm is not necessarily required. The number of clusters can be determined by any algorithm, and preferably, it can be determined so that the clusters can be uniformly arranged on the coordinate system based on the overall color distribution state. In addition, the representative value of each cluster can be determined by the coordinate value of the cluster center point.

[0098] After step S205, the computational unit (100) displays RGB values ​​corresponding to the representative values ​​of each cluster on a coordinate system (S206). In this step, the representative values ​​of each cluster are converted into an RGB color space and displayed on the coordinate system. This allows for visual confirmation of the color range represented by each cluster.

[0099]

[0100] Hereinafter, a medical treatment support system and method according to another embodiment of the present invention will be described.

[0101] Figures 8 and 9 are diagrams for understanding the overall outline of a medical support service and a medical support method according to another embodiment of the present invention. The service may be executed by the processing device (100) described above in Figure 1, but is not necessarily limited thereto. However, for ease of understanding, this detailed description assumes that the processing device (100) also provides the medical support service described below.

[0102] Referring to FIG. 8, when a biofluorescence image of a specific patient's oral cavity is input, the computing device (100) can perform multi-chromatic analysis and provide the evaluation results of the patient's condition in the form of treatment support information as shown in the drawing.

[0103] In this way, the present embodiment is implemented to provide the user (diagnostician) with the results of color analysis when a biofluorescence image of a patient to be evaluated exists, and the following will discuss in more detail what interfaces the system including the computing device (100) provides to support the user's (diagnostician's) diagnosis process.

[0104] Referring to FIG. 9, the treatment support method includes a step (S900) in which the computing device (100) receives a patient selection input. This step can be understood as a step in which the user receives an input through the interface illustrated in FIG. 10. Referring to FIG. 10, a patient list may be output on the monitor screen viewed by the user, and this screen may display an area (1001) indicating the reservation status of each patient, an area (1002) indicating information such as name, date of birth, gender, and contact information, and additionally, an area (1003) in which an indication is displayed of whether the patient has downloaded and linked a specific APP (e.g., a patient-only APP that can provide treatment-related information in conjunction with a diagnosis support system), a button for opening a function to contact each patient via text message, etc.

[0105] Referring back to FIG. 9, after step S900, a step (S1000) of providing treatment support information for a selected patient may be executed. This step includes detailed steps in which the computing device (100) analyzes the patient's oral image and outputs color analysis information, treatment reports, a list of examination items, etc., and the information provided to the user (diagnostician) in this step will be collectively referred to as treatment support information. In other words, treatment support information includes all types of information that the user (diagnostician) can refer to when treating a patient, and among these, it may be characterized by including color analysis information, treatment reports, a list of examination items, etc. in particular.

[0106] Step S1000 includes a step (S1010) of first outputting an oral image, which can be performed on the premise that a biofluorescence image has been obtained by a user (diagnostician) taking a picture of the patient's oral cavity.

[0107] Meanwhile, the images output at this stage are not limited to biofluorescence images of teeth, but may also include images taken of the inside of the patient's oral cavity in a conventional manner.

[0108] FIG. 11 illustrates a screen where a patient's oral cavity image is output to a user (diagnostician). Referring to the drawing, it can be seen that the screen displays images of the patient's oral cavity (1101), such as images taken of the patient's oral cavity in a general manner and biofluorescence images. In addition, a color analysis result image (1102) that allows for easy confirmation of the color analysis results, which will be described later, particularly the results analyzed by artificial intelligence, can also be displayed.

[0109] In addition, when referring to FIG. 11, for the convenience of the user (diagnostician), a selection area (1103; a selection button for selecting an image taken with which photographing device (cam pro, Pen C, etc.)) may be provided to select the type of oral image, a button (1102a) for selecting whether to display the results of the fluorescence image color analysis together, a button (1104) for additionally uploading an oral image, etc. may be provided.

[0110] After step S1010, a step (S1020) for outputting color analysis information is executed. Fig. 12 illustrates an example of color analysis information provided to a user (diagnostician). Referring to the drawing, it can be confirmed that the color analysis information includes information such as a color distribution map (1201), a comprehensive score (1202), and a natural tooth ratio (1203). Among these, the color distribution map (1201) can be said to be core information among the color analysis information, and the comprehensive score, natural tooth ratio, etc. can be said to be information provided so that the user (diagnostician) can understand the patient's condition more intuitively.

[0111] Among these, the color distribution map (1201) can be obtained as a result of the analysis operation of the calculation device (100), and the analysis operation of the calculation device (100) may include a step of identifying only the evaluation area to be analyzed from the biofluorescence image, a step of obtaining color information (RGB values ​​and Lab values) for the evaluation area, and a step of performing color analysis based on the color information. These steps will be described with reference to the contents explained above through FIGS. 3 to 6, etc.

[0112] Meanwhile, referring to Fig. 12, a comprehensive score (1202) for the patient's oral condition can be created based on the previously analyzed results. In particular, the ratio of high-risk group, low-risk group, and medium-risk group can be identified through multi-color analysis to create a color distribution map (1201), and thus a comprehensive score can be calculated based on this. In addition, the natural tooth ratio (1203) can also be obtained by calculating the result obtained through multi-color analysis, i.e., the ratio value of natural teeth and non-natural teeth (prosthetics).

[0113]

[0114] Referring again to FIG. 9, the step of providing treatment support information (S1000) includes a step of printing a treatment report (S1030), which will be explained with reference to FIG. 13.

[0115] When referring to the drawing, the medical report may include the patient's daily examination history (1301). This may include information on what type of image was taken of the patient's oral cavity (or what device was used to take the image) and which teeth were examined.

[0116] In addition, the treatment report may include an opinion (1302) generated based on the independent judgment of the computing device (100) (judgment through artificial intelligence algorithm), which may include at least information about teeth requiring management, and may additionally include measures to be taken for dental health.

[0117] In addition, the treatment report may further include the patient's next treatment schedule (1303), a tooth selection area (1304) where the patient can enter the tooth currently being treated, etc., thereby increasing the convenience of the user (diagnostician) during the treatment process.

[0118]

[0119] Referring back to FIG. 9, the step of providing treatment support information (S1000) may further include a step of outputting a list of examination items (S1040). This step may be understood as a step of selecting only the examination items necessary for the patient based on the information obtained during the previous multi-color analysis and treatment report generation processes and showing them to the user (diagnostician). The user (diagnostician) may input a selection of several examination items that are determined to be most necessary for the patient among the selected examination items, thereby enabling the examination scheduling for the patient to be performed quickly.

[0120]

[0121] Here, we have looked at an object evaluation method using color analysis according to the present invention and a system therefor, and also looked at a method for providing a treatment support service to a user (diagnostician) by utilizing such an evaluation method. Meanwhile, the present invention is not limited to the specific embodiments and application examples described above, and various modifications can be made by a person skilled in the art to which the present invention pertains without departing from the gist of the present invention claimed in the claims, and such modifications should not be understood as being distinct from the technical idea or prospect of the present invention.

Claims

1. A method for evaluating an object using color analysis based on a biofluorescence image, wherein the computational device including a central processing unit and a memory, A step of receiving a biofluorescence image of an arbitrary object; A step of obtaining RGB values ​​and Lab values ​​for the evaluation area in the above biofluorescence image; and A step of performing color analysis based on the acquired RGB values ​​and Lab values ​​and displaying the color analysis results for the object on a specific coordinate system; including, A method for evaluating objects using color analysis.

2. In paragraph 1, After receiving the above biofluorescence image, A step of identifying only the evaluation area from within the above biofluorescence image; including more, A method for evaluating objects using color analysis.

3. In paragraph 2, The above color analysis is, A process of comparing the acquired color information with a previously generated color cluster to determine a color cluster to which a plurality of points within the evaluation area belong, A method for evaluating objects using color analysis.

4. In paragraph 3, The above coordinate system is characterized in that the ratio of the G value to the R value is on the x-axis and the L value is on the y-axis. A method for evaluating objects using color analysis.

5. In paragraph 4, The above coordinate system is, characterized by including a plurality of color clusters defined in advance, A method for evaluating objects using color analysis.

6. In paragraph 1, The step of displaying the above color analysis results is: Characterized in that points mapped to multiple points within the evaluation area are displayed on the coordinate system. A method for evaluating objects using color analysis.

7. In paragraph 6, The step of displaying the above color analysis results is: A method characterized by further comprising a process of determining which color clusters a plurality of mapped points belong to and calculating the proportion of the color clusters to which the plurality of points belong. A method for evaluating objects using color analysis.

8. In paragraph 7, The step of displaying the above color analysis results is: A method characterized in that it further includes a process of determining whether the object belongs to a high-risk group, a medium-risk group, or a low-risk group based on the ratio of the color clusters to which the plurality of points belong. A method for evaluating objects using color analysis.

9. A method for clustering for color analysis based on biofluorescence images, wherein the computing device including the central processing unit and the memory A step of collecting multiple biofluorescence images; A step of identifying only the evaluation area to be analyzed for each of the above biofluorescence images; A step of obtaining RGB values ​​and Lab values ​​for the evaluation area in the above biofluorescence images; and A step of displaying all color distributions corresponding to the evaluation area on a coordinate system based on the acquired RGB values ​​and Lab values; A step of determining the number of clusters and determining a representative value for each cluster; A step of displaying RGB values ​​corresponding to the representative values ​​of each of the above clusters on a coordinate system; including, A clustering method for color analysis based on biofluorescence images.

10. In a system including a central processing unit and memory, The above central processing unit is characterized in that it executes commands for executing an object evaluation method using color analysis stored in the memory, The object evaluation method using the above color analysis is as follows: A step of receiving a biofluorescence image of an arbitrary object; A step of obtaining RGB values ​​and Lab values ​​for the evaluation area in the above biofluorescence image; and A step of performing color analysis based on the acquired RGB values ​​and Lab values ​​and displaying the color analysis results for the object on a specific coordinate system; including, System.

11. A method for supporting treatment by a computing device including a central processing unit and memory, (a) The stage where the patient to be treated is identified; (b) a step of providing treatment support information for the above patient; Includes, The above treatment support information includes the patient's oral image and color analysis information, wherein the oral image includes a biofluorescence image, and the color analysis information is color analysis information based on the biofluorescence image. How to get medical support.

12. In a treatment support system including a central processing unit and memory, The central processing unit is characterized in that it executes commands for executing a treatment support method stored in the memory. The above treatment support method is: (a) The stage where the patient to be treated is identified; (b) a step of providing treatment support information for the above patient; Includes, The above treatment support information includes the patient's oral image and color analysis information, wherein the oral image includes a biofluorescence image, and the color analysis information is color analysis information based on the biofluorescence image. Medical support system.

Images