Operation method of electronic device for assisting diagnosis of age-related brain disease through analysis of tongue image

Abstract

An operation method of an electronic device for assisting diagnosis of age-related brain disease is disclosed. The operation method comprises steps in which an electronic device: acquires at least one target image obtained by photographing the tongue of a subject; extracts feature information about at least one of teeth marks, cracks, tongue coating, and degree of curvature from the acquired target image; and predicts, on the basis of the extracted feature information, whether the subject has onset of age-related brain disease.

Description

Method of operation of an electronic device for assisting in the diagnosis of geriatric brain diseases through analysis of tongue images

[0001] The present disclosure relates to an operating method of an electronic device, and more particularly, to an operating method of an electronic device that assists in the diagnosis of geriatric brain disease through analysis of a tongue image.

[0002] In Oriental medicine, tongue diagnosis is used to diagnose a patient's condition based on the relationship between the tongue and various body parts. Tongue diagnosis is a diagnostic method that assesses a patient's health by observing the tongue's color, shape, coating, and other aspects. Through tongue diagnosis, the cause and progression of a disease can be identified.

[0003] Conventional techniques only provide images that can be used for diagnosis by correcting them so that the actual color observed with the naked eye matches the color of the tongue included in the image, as the color of the tongue included in the image differs depending on the shooting environment, and the actual diagnosis is performed by the examiner.

[0004] However, in order for the examinee to conveniently receive a diagnosis at a desired time and monitor his or her health, a technology that performs a diagnosis based on an image of the tongue is needed.

[0005] The present disclosure seeks to provide an operating method of an electronic device that assists in the diagnosis of geriatric brain disease through analysis of a tongue image.

[0006] The purposes of the present disclosure are not limited to those mentioned above, and other purposes and advantages of the present disclosure not mentioned above can be understood through the following description and will be more clearly understood through the embodiments of the present disclosure. Furthermore, it will be readily apparent that the purposes and advantages of the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0007] A method of operating an electronic device for assisting in the diagnosis of a geriatric brain disease according to one embodiment of the present disclosure includes a step of the electronic device obtaining at least one target image in which a tongue of a subject is photographed, a step of the electronic device extracting feature information on at least one of a tooth mark, a crack, a tongue coating, and a degree of bending from the obtained target image, and a step of the electronic device predicting whether the subject has developed a geriatric brain disease based on the extracted feature information.

[0008] The step of extracting the above feature information may include the electronic device inputting the target image into a first artificial intelligence model for extracting feature information from an image, thereby obtaining feature information about the target image from the first artificial intelligence model.

[0009] The step of predicting whether the subject has developed a geriatric brain disease may include: the electronic device inputting the feature information extracted from an image of the tongue into a second artificial intelligence model for predicting the disease, and obtaining prediction information including whether the subject has developed a geriatric brain disease from the second artificial intelligence model.

[0010] At this time, the operating method of the electronic device may include a step of the electronic device obtaining training data including a plurality of images in which the tongues of each of the plurality of patients are photographed and disease information of the patient corresponding to the tongue included in each of the plurality of images, and a step of the electronic device training the second artificial intelligence model to predict the onset of a geriatric brain disease using the obtained training data.

[0011] Meanwhile, the operating method of the electronic device may include a step of the electronic device receiving a plurality of verification images from a verification database in which disease information of each of a plurality of patients and a plurality of verification images in which the tongues of each of the plurality of patients are photographed are matched and stored, and identifying a similarity of the target image to each of the plurality of received verification images; when a verification image having the similarity exceeding a threshold is identified, the electronic device obtaining disease information matched with the verification image having the similarity exceeding the threshold, and identifying a match between the obtained disease information and prediction information including the predicted onset of disease; and when the disease information and the prediction information do not match, the electronic device inputting feature information extracted from the target image again into the second artificial intelligence model to re-predict whether or not a geriatric brain disease is onset for the subject.

[0012] Meanwhile, the operating method of the electronic device may include, when a plurality of image frames in which the tongue of the subject is continuously photographed are acquired, a step in which the electronic device selects consecutive image frames corresponding to some of the plurality of frames based on the similarity of each of the plurality of image frames, a step in which the electronic device extracts feature information from the selected image frames, and a step in which the electronic device predicts whether the subject has a geriatric brain disease based on the feature information extracted from the selected image frames.

[0013] The step of selecting the above continuous image frames may include identifying the outline of the tongue included in each of the plurality of image frames, selecting an image frame in which the size of the identified outline is the largest and setting it as a target image frame, identifying the similarity of the target image frame to each of the plurality of image frames, and selecting image frames continuous with the target image frame among the image frames in which the identified similarity is greater than or equal to a threshold value.

[0014] An electronic device according to one embodiment of the present disclosure can predict the onset of a geriatric brain disease based on an image or video of a tongue.

[0015] In addition, an electronic device according to an embodiment of the present disclosure can provide predictive diagnostic information with improved accuracy through comparison with information stored in a database.

[0016] FIG. 1 is a flowchart illustrating a process by which an electronic device predicts the onset of a geriatric brain disease according to one embodiment of the present disclosure;

[0017] FIG. 2 is a drawing for explaining a tongue element according to one embodiment of the present disclosure;

[0018] FIG. 3 is a diagram illustrating an operation of an electronic device to predict the location of onset of a geriatric brain disease according to an embodiment of the present disclosure;

[0019] FIG. 4 is a diagram illustrating an operation of an electronic device and a database transmitting and receiving information according to one embodiment of the present disclosure;

[0020] FIG. 5 is a flowchart illustrating an operation of an electronic device to select some of a plurality of image frames and predict whether a subject has a senile brain disease according to an embodiment of the present disclosure; and

[0021] FIG. 6 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

[0022] Before describing the present disclosure in detail, the description method of the specification and drawings will be described.

[0023] First, the terms used in this specification and claims are general terms selected based on their functions in the various embodiments of the present disclosure. However, these terms may vary depending on the intentions of those skilled in the art, legal or technical interpretations, and the emergence of new technologies. Furthermore, some terms may have been arbitrarily selected by the applicant. These terms may be interpreted according to the meanings defined in this specification. In the absence of a specific definition, they may be interpreted based on the overall content of this specification and common technical knowledge in the relevant field.

[0024] Additionally, the same reference numbers or symbols in each drawing attached to this specification represent parts or components that perform substantially the same functions. For convenience of explanation and understanding, the same reference numbers or symbols are used in different embodiments. In other words, even if components with the same reference numbers are all depicted in multiple drawings, the multiple drawings do not necessarily represent a single embodiment.

[0025] Additionally, terms including ordinal numbers, such as "first," "second," etc., may be used in this specification and claims to distinguish between components. These ordinal numbers are used to distinguish identical or similar components from each other, and the use of these ordinal numbers should not be interpreted in a limited manner. For example, components associated with these ordinals should not be restricted in their order of use or arrangement by their numbers. If necessary, each ordinal number may be used interchangeably.

[0026] In this specification, singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, terms such as "comprise" or "consist of" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood not to preclude the possibility of the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0027] In the embodiments of the present disclosure, terms such as "module," "unit," "part," etc. are terms used to refer to components that perform at least one function or operation, and such components may be implemented as hardware or software, or a combination of hardware and software. In addition, a plurality of "modules," "units," "parts," etc. may be integrated into at least one module or chip and implemented as at least one processor, except in cases where each needs to be implemented as a separate, specific hardware.

[0028] Additionally, in the embodiments of the present disclosure, when a part is said to be connected to another part, this includes not only a direct connection but also an indirect connection through another medium. Furthermore, unless specifically stated otherwise, the statement that a part includes a certain component does not exclude other components, but rather implies that other components may be included.

[0029] FIG. 1 is a flowchart illustrating a process by which an electronic device predicts the onset of a geriatric brain disease according to one embodiment of the present disclosure.

[0030] The electronic device (100) is a device for predicting the onset of geriatric brain disease based on an image including a tongue.

[0031] Age-related brain diseases can include Alzheimer's, Parkinson's, and stroke.

[0032] For example, the electronic device (100) may be implemented as a device or system comprising at least one computer. In addition, the electronic device (100) may be implemented as a server, smartphone, tablet PC, laptop PC, desktop PC, etc.

[0033] For example, the electronic device (100) may provide a platform by linking with a terminal device through at least one web page or application.

[0034] At this time, the electronic device (100) can communicate with a plurality of terminal devices through at least one web page and / or application constituting the platform, and can also provide information on the occurrence of a predicted geriatric brain disease based on image data input from the terminal device through at least one web page and / or application to the terminal device.

[0035] Additionally, the electronic device (100) may correspond to a terminal of the subject.

[0036] Meanwhile, the electronic device (100) may include an image processing module, an object recognition module, an RGB extraction module, a data analysis module, etc. Each of these modules may correspond to a functional unit module implemented in software and / or hardware.

[0037] The image processing module is a module for extracting feature information related to the tongue contained in the image.

[0038] For example, the image processing module can extract feature information about at least one of a scratch, a crack, a warp, and a degree of bending included in the image.

[0039] The object recognition module is a module for recognizing the tongue included in the image.

[0040] For example, an object recognition module can recognize an object by dividing an image into multiple unit areas and then identifying whether an object is included in each unit area.

[0041] Additionally, the object recognition module can recognize objects based on data acquired by moving a bounding box of a preset size pixel by pixel on the image.

[0042] Additionally, the object recognition module may extract objects using a third artificial intelligence model for recognizing objects included in an image.

[0043] The electronic device (100) may obtain and use a third artificial intelligence model trained to extract objects from an external source, or may train the third artificial intelligence model based on a plurality of images including at least one of a philtrum and a tongue to extract objects corresponding to the philtrum and the tongue from the images.

[0044] Meanwhile, the third artificial intelligence model can be a variety of learning algorithm models such as R-CNN (Region-based Convolutional Neural Networks), CNN (Convolutional Neural Networks), U-Net, and Transformer, but is not limited thereto.

[0045] The RGB extraction module is a module for extracting the RGB values ​​of each pixel that makes up an image.

[0046] RGB values ​​are values ​​that represent the color information that each pixel represents, with R representing the value of the red channel, G representing the value of the green component, and B representing the value of the blue component, and the value of each component being expressed as a number between 0 and 255.

[0047] For example, the RGB extraction module can extract the RGB values ​​of each of the multiple pixels included in the image.

[0048] The data analysis module is a module for predicting the onset of geriatric brain disease based on feature information extracted from images.

[0049] For example, the data analysis module can predict the onset of geriatric brain disease by analyzing the correlation between feature information extracted from the image (e.g., correlation between elements (e.g., tooth marks, cracks, tongue coating, degree of bending), correlation between elements and geriatric brain disease, etc.).

[0050] For example, the data analysis module may be a module trained to identify correlations between elements and correlations between elements and geriatric brain diseases based on learning data including multiple images of the tongues of each of multiple patients and disease information of the patients corresponding to the tongues included in each of the multiple images, and to predict the onset of geriatric brain diseases based on feature information extracted from the images.

[0051] Specifically, the data analysis module can identify the correlation between elements based on the frequency with which each element is simultaneously observed through learning data, and compare the elements included in an image of the tongue of a patient diagnosed with a geriatric brain disease with the elements included in an image of the tongue of a patient not diagnosed with a geriatric brain disease to identify the correlation between the elements and geriatric brain disease.

[0052] At this time, the disease information may include the diagnosis of the patient (e.g., Alzheimer's, stroke, etc.), the location of onset of the patient's geriatric brain disease, etc., and each of the plurality of images may include label data for at least one of the marks, cracks, coatings, and degree of bending included in each image.

[0053] Referring to FIG. 1, the electronic device (100) can obtain a target image including the tongue of the subject (S110).

[0054] As an example, the electronic device (100) can obtain a target image through user input.

[0055] The electronic device (100) can extract feature information from a target image (S120).

[0056] In one embodiment, the electronic device (100) may extract feature information about at least one element of a tooth mark, a crack, a coating, and a degree of curvature included in the tongue of the subject from a target image including the tongue of the subject.

[0057] A dental impression is an imprint of a tooth identified along the outer contour of the tongue included in the image.

[0058] Crack refers to a crack or split that occurs in the tongue.

[0059] Tongue coating is a deposit that forms on the surface of the tongue and can come in a variety of colors, including pink, white, yellow, gray, and black.

[0060] The degree of curvature may include the direction in which the tongue is curved.

[0061] Meanwhile, the electronic device (100) can use the first artificial intelligence model to extract feature information from the target image.

[0062] As an example, the electronic device (100) may extract feature information on at least one element of a tongue mark, a crack, a tongue coating, and a degree of curvature from a target image through a first artificial intelligence model.

[0063] At this time, the feature information may include, but is not limited to, the location of each element, color, and whether each element is identifiable.

[0064] Meanwhile, the first artificial intelligence model may include a CNN (Convolution Neural Network)-based model for extracting feature information by performing a convolution operation on an image.

[0065] For example, the first artificial intelligence model can perform a convolution operation on an input image using a plurality of convolutional layers and a plurality of pooling layers to obtain a feature map, and input the obtained feature map into a fully connected layer to output feature information on at least one of a tooth mark, a crack, a tongue coating, and a degree of warping.

[0066] Meanwhile, the electronic device (100) may obtain from the outside and use a first artificial intelligence model trained to extract feature information from an image, or may train the first artificial intelligence model to extract feature information on at least one of a tooth mark, crack, tongue coating, and degree of curvature included in the tongue from the image.

[0067] For example, the electronic device (100) may train a first artificial intelligence model based on a plurality of images including at least one of a tooth mark, a crack, a tongue coating, and a degree of curvature included in the tongue, and including label data for at least one of the tooth mark, the crack, the tongue coating, and the degree of curvature.

[0068] As an additional example, the electronic device (100) can extract feature information for at least one of a scratch, a crack, and a stain through the RGB value of each pixel included in the target image.

[0069] Specifically, the electronic device (100) can extract RGB values ​​of each of a plurality of pixels included within the outline of the tongue included in the target image, and extract feature information including the presence or absence or location of at least one of a tooth mark, a crack, and a tongue coating based on the extracted RGB values.

[0070] For example, the electronic device (100) can identify at least one of a dent, a crack, and a dent included in a target image through a fourth artificial intelligence model trained to identify at least one of a dent, a crack, and a dent based on the RGB values ​​of each of a plurality of pixels.

[0071] The electronic device (100) can train a fourth artificial intelligence model based on a plurality of images including at least one of a tooth mark, a crack, and a tongue coating included in the tongue, and including label data for at least one of the tooth mark, the crack, and the tongue coating.

[0072] The fourth artificial intelligence model may be an artificial intelligence model based on various learning algorithms such as MASK R-CNN (MASK Region-based Convolutional Neural Networks) and U-Net, but is not limited thereto.

[0073] The electronic device (100) can predict whether or not a subject has a geriatric brain disease based on feature information extracted from the target image (S130).

[0074] As an example, the electronic device (100) can predict whether a subject has a geriatric brain disease through a second artificial intelligence model for predicting a disease based on feature information extracted from an image including a tongue.

[0075] Specifically, the electronic device (100) can input feature information extracted from the target image into a second artificial intelligence model and obtain prediction information including whether the subject has developed a geriatric brain disease from the second artificial intelligence model.

[0076] Predictive information may include whether or not a geriatric brain disease will occur, the location of the geriatric brain disease, etc.

[0077] To this end, the electronic device (100) can train a second artificial intelligence model based on training data including a plurality of images in which the tongues of each of a plurality of patients are photographed and disease information of the patient corresponding to the tongue included in each of the plurality of images.

[0078] Disease information may include the patient's diagnosis (e.g., Alzheimer's, stroke, etc.), the location of onset of the patient's geriatric brain disease, etc.

[0079] At this time, the electronic device (100) can obtain a plurality of images including label data for at least one of a scratch, a crack, a stain, and a degree of bending included in each image to form training data.

[0080] Meanwhile, the second artificial intelligence model may be a model trained based on various learning algorithms such as CNN (Convolutional Neural Network), R-CNN (Region-based Convolutional Neural Networks), U-Net, and Transformer for prediction, but is not limited thereto.

[0081] FIG. 2 is a drawing for explaining a tongue element according to one embodiment of the present disclosure.

[0082] Referring to FIG. 2, the elements of the tongue may include a tooth mark (1), a crack (2), a degree of curvature (3), and a tongue coating (4).

[0083] Tooth marks (1) refer to the traces of teeth identified along the external contour of the tongue.

[0084] As an example, the electronic device (100) can identify the presence or absence of a mark based on feature information extracted from a target image.

[0085] Specifically, the electronic device (100) can input a target image into the first artificial intelligence model to obtain feature information including the presence or absence of a mark.

[0086] As an additional example, the electronic device (100) may input a target image into a fourth artificial intelligence model that extracts feature information based on RGB values ​​to obtain feature information including the presence or absence of a mark.

[0087] Crack (2) refers to a crack or split that occurs in the tongue.

[0088] As an example, the electronic device (100) can identify the location of a crack based on feature information extracted from a target image.

[0089] Specifically, the electronic device (100) can obtain feature information including the location of a crack by inputting a target image into the first artificial intelligence model.

[0090] As an additional example, the electronic device (100) may obtain feature information including the location of a crack by inputting a target image into a fourth artificial intelligence model that extracts feature information based on RGB values.

[0091] The degree of bending (3) may include the direction in which the tongue is bent.

[0092] For example, the electronic device (100) can identify the degree of tongue curvature based on the philtrum included in the target image.

[0093] Specifically, the electronic device (100) can extract the outline of the tongue and the outline of the philtrum included in the target image through the object recognition module.

[0094] At this time, the electronic device (100) can set the center point of the tongue based on a plurality of points forming the outline of the tongue, and can set the center point of the philtrum based on a plurality of points forming the outline of the philtrum.

[0095] For example, the electronic device (100) may set a center point by calculating a coordinate value corresponding to the center point of the contour based on the coordinate values ​​of each of the plurality of points (pixels) constituting each contour, but is not limited thereto.

[0096] In addition, the electronic device (100) can set the point that is the farthest from the center point of the philtrum among the multiple points forming the tongue outline as the tip point of the tongue.

[0097] Through this, the electronic device (100) can obtain a first connecting line connecting the center point of the philtrum and the center point of the tongue and a second connecting line connecting the center point of the philtrum and the tip of the tongue, and identify the direction in which the second connecting line is located based on the first connecting line as the direction in which the tongue is bent.

[0098] Tongue coating (4) is an attachment that forms on the surface of the tongue and can appear in various colors such as pink, white, yellow, gray, and black.

[0099] In one embodiment, the electronic device (100) can identify the location of the snow or the color of the snow based on feature information extracted from the target image.

[0100] Specifically, the electronic device (100) can input a target image into the first artificial intelligence model to obtain feature information including the location of the snow coat or the color of the snow coat.

[0101] As an additional example, the electronic device (100) may input a target image into a fourth artificial intelligence model that extracts feature information based on RGB values ​​to obtain feature information including the location of the snow coat or the color of the snow coat.

[0102] FIG. 3 is a diagram illustrating an operation of an electronic device to predict the location of onset of a geriatric brain disease according to one embodiment of the present disclosure.

[0103] Referring to Figure 3, cracks can appear in various areas of the tongue.

[0104] In one embodiment, the electronic device (100) can identify the location of a crack within a target image based on feature information extracted from the target image.

[0105] Specifically, the electronic device (100) can identify the location of a crack within the target image through feature information extracted from the target image, and select a portion of the tongue based on the identified location of the crack.

[0106] For example, the electronic device (100) may select a portion of the tongue that includes cracks where the distance between cracks is less than a threshold value, based on the location of each crack identified based on the characteristic information.

[0107] Additionally, the electronic device (100) can select a portion of the tongue where the tongue coating is identified within the target image.

[0108] Additionally, the electronic device (100) may select a portion of the tongue where the tooth mark is located within the target image.

[0109] Specifically, the electronic device (100) can select a portion of an area that includes pixels in which the deviation of pixel values ​​of each adjacent pixel exceeds a threshold deviation value, within an area identified as having a mark according to feature information.

[0110] At this time, the electronic device (100) can predict the location of onset of geriatric brain disease based on the selected area.

[0111] To this end, the electronic device (100) can set the center line of the tongue based on a plurality of points that constitute the outline of the tongue.

[0112] Specifically, the electronic device (100) can select two points from among a plurality of points forming the outline of the tongue in the order of distance from the center point of the tongue, and set the midpoint of the two selected points as a reference point.

[0113] Additionally, the electronic device (100) can set the point with the greatest distance from the reference point among the plurality of points forming the tongue outline as the tip point of the tongue.

[0114] At this time, the electronic device (100) can set the line connecting the reference point, the center point of the tongue, and the tip point of the tongue as the center line of the tongue.

[0115] Meanwhile, the electronic device (100) can identify the direction in which the selected area is located based on the center line, and identify the location of onset of geriatric brain disease in the brain of the subject based on the identified direction.

[0116] For example, if the selected area is located on the right side of the image with respect to the center line, the electronic device (100) can predict that the geriatric brain disease has developed in the left brain of the subject.

[0117] Additionally, if the selected area is located on the left side of the image with respect to the center line, the electronic device (100) can predict that the geriatric brain disease has developed in the right brain of the subject.

[0118] FIG. 4 is a diagram for explaining an operation of an electronic device and a database transmitting and receiving information according to one embodiment of the present disclosure.

[0119] Referring to FIG. 4, the electronic device (100) can receive information from a database (DB). At this time, the database (DB) may be stored in the electronic device (100) or at least one device connected to the electronic device (100).

[0120] For example, the database (DB) may correspond to a training database in which multiple training images of the tongues of each of multiple patients are captured and disease information of the patient corresponding to the tongue included in each of the multiple training images (e.g., diagnosis name (e.g., Alzheimer's, stroke, etc.), location of onset of geriatric brain disease, etc.) are matched and stored.

[0121] In one embodiment, the electronic device (100) may receive a plurality of training images in which the tongues of each of a plurality of patients are photographed from a training database and disease information of the patient corresponding to the tongue included in each of the plurality of training images, and configure training data for training a second artificial intelligence model based on the received information.

[0122] Additionally, the database (DB) may correspond to a verification database in which disease information of each of a plurality of patients and a plurality of verification images of the tongues of each of the plurality of patients are matched and stored.

[0123] In one embodiment, the electronic device (100) can receive multiple verification images from a verification database.

[0124] At this time, the electronic device (100) can identify the similarity of the target image to each verification image received from the verification database.

[0125] Specifically, the electronic device (100) can identify, for each verification image, a similarity, which is the degree to which each verification image matches the target image.

[0126] For example, the electronic device (100) can extract feature information from each verification image and identify similarity based on the degree to which the feature information extracted from each verification image matches the feature information extracted from the target image.

[0127] Specifically, the electronic device (100) can identify a high degree of similarity of the target image to the verification image when the same element is identified at a similar location (e.g., tip of the tongue, left side of the tongue, center of the tongue, right side of the tongue, etc.) based on feature information extracted from the verification image and feature information extracted from the target image.

[0128] As an additional example, the electronic device (100) can identify the outline of the lips included in the verification image and the outline of the lips included in the target image, and move the verification image and the target image so that the outlines of the identified lips overlap, thereby identifying the similarity of the target image to the verification image.

[0129] In addition, the electronic device (100) can identify the center point and the tip point of the tongue included in the verification image and the center point and the tip point of the tongue included in the target image, and move the verification image and the target image so that the identified center point and the tip point of the tongue overlap, thereby identifying the similarity of the target image to the verification image.

[0130] At this time, the electronic device (100) can identify a higher degree of similarity as the number of overlapping pixels between the verification image and the target image increases.

[0131] Meanwhile, when a verification image with a similarity exceeding a threshold is identified, the electronic device (100) can obtain disease information matched with the verification image with a similarity exceeding the threshold.

[0132] In addition, when two or more verification images with similarity exceeding a threshold are identified, the electronic device (100) can obtain disease information matched with the verification image with the highest similarity.

[0133] At this time, the electronic device (100) can identify whether there is a match between the acquired disease information (target disease information) and the predicted information (information including whether or not a geriatric brain disease has occurred as predicted based on the target image).

[0134] For example, if the patient is identified as not having been diagnosed with a geriatric brain disease based on the target disease information (disease information matched with a verification image whose similarity exceeds a threshold), but is identified as having developed a geriatric brain disease based on the prediction information, the electronic device (100) can identify that the target disease information and the prediction information do not match.

[0135] In addition, if the patient is identified as having been diagnosed with a geriatric brain disease from the target disease information but is identified as not having developed a geriatric brain disease through the prediction information, the electronic device (100) can identify that the target disease information and the prediction information do not match.

[0136] If the target disease information and the predicted information do not match, the electronic device (100) can re-input the feature information extracted from the target image into the second artificial intelligence model to re-predict whether or not the subject will develop a geriatric brain disease, thereby obtaining predicted information.

[0137] Additionally, if the target disease information and the prediction information do not match, the electronic device (100) can input the feature information extracted from the target image into the second artificial intelligence model to identify whether there is a match between the obtained prediction information and the target disease information.

[0138] As an example, the electronic device (100) may repeat an operation (prediction operation) of predicting whether or not a subject will develop a geriatric brain disease through a second artificial intelligence model until prediction information matching the target disease information is obtained.

[0139] When prediction information matching the target disease information is obtained, the electronic device (100) can calculate the number of repetitions of the prediction operation until prediction information matching the target disease information is obtained.

[0140] At this time, the electronic device (100) may additionally perform a prediction operation the same number of times as the number of repetitions if the number of repetitions is greater than or equal to a preset first reference number (e.g., 5 times, 8 times, 10 times, etc.), and may not additionally perform a prediction operation if the number of repetitions is less than the first reference number.

[0141] Specifically, when the number of repetitions is greater than or equal to a preset first reference number, the electronic device (100) can identify whether the prediction information obtained by performing the prediction operation a number of times equal to the number of repetitions matches the target disease information.

[0142] Accordingly, the electronic device (100) can calculate the number of matches, which is the number of times the prediction operation is performed, in which the prediction information obtained matches the target disease information by performing the prediction operation the same number of times as the number of repetitions.

[0143] At this time, the electronic device (100) may perform additional training of the second artificial intelligence model if the calculated number of matches is less than a preset second standard number (e.g., 2 times, 4 times, 5 times, etc.), and may not perform additional training of the second artificial intelligence model if the calculated number of matches is greater than or equal to the second standard number.

[0144] Through this, the electronic device (100) can prevent overfitting in exceptional cases.

[0145] Meanwhile, if additional training of the second artificial intelligence model is required, the electronic device (100) can train the second artificial intelligence model based on the target disease information and the verification image corresponding to the target disease information.

[0146] FIG. 5 is a flowchart illustrating an operation of an electronic device selecting some of a plurality of image frames to predict whether a subject has developed a geriatric brain disease according to an embodiment of the present disclosure.

[0147] Referring to FIG. 5, the electronic device (100) can acquire multiple image frames in which the tongue of the subject is continuously photographed (S510).

[0148] As an example, the electronic device (100) can obtain multiple image frames through user input.

[0149] The electronic device (100) can select consecutive image frames corresponding to some of the plurality of frames based on the similarity of each of the plurality of image frames (S520).

[0150] In one embodiment, the electronic device (100) can identify the outline of the tongue included in each of a plurality of image frames through an object recognition module.

[0151] At this time, the electronic device (100) can select an image frame containing the largest outline among the identified tongue outlines and set it as a target image frame.

[0152] For example, the electronic device (100) can compare the number of pixels contained within the tongue outline and identify the tongue outline containing the largest number of pixels as having the largest size.

[0153] The electronic device (100) can identify the similarity of a target image frame to each of a plurality of image frames.

[0154] Specifically, the electronic device (100) can identify, for each image frame, a similarity, which is the degree to which each image frame matches a target image frame.

[0155] For example, the electronic device (100) can compare the pixel values ​​of each of the plurality of pixels included in each image frame with the pixel values ​​of each of the plurality of pixels included in the target image frame, and identify the similarity based on the degree to which each matches.

[0156] Specifically, the electronic device (100) can identify similarity based on the ratio of matching pixel values ​​by comparing pixel values ​​between pixels having the same coordinate values, but is not limited thereto.

[0157] At this time, the electronic device (100) can select image frames that are continuous with the target image frame among the image frames whose identified similarity is greater than a threshold value.

[0158] The electronic device (100) can predict whether or not a subject has developed a geriatric brain disease based on feature information extracted from selected image frames (S530).

[0159] Specifically, the electronic device (100) can input selected image frames into a first artificial intelligence model to obtain feature information, and input the obtained feature information into a second artificial intelligence model to obtain prediction information on whether or not the subject has developed a geriatric brain disease.

[0160] FIG. 6 is a diagram illustrating a configuration of an electronic device according to an embodiment of the present disclosure.

[0161] Referring to FIG. 6, the electronic device (100) may include a memory (110), a processor (120), and a communication unit (130).

[0162] The memory (110) is a configuration for storing an operating system (OS) for controlling the overall operation of components of the electronic device (100) and at least one instruction or data related to the components of the electronic device (100).

[0163] The memory (110) may include non-volatile memory such as ROM, flash memory, etc., and may include volatile memory composed of DRAM, etc. In addition, the memory (110) may include a hard disk, SSD (Solid state drive), etc.

[0164] The processor (120) is configured to control the overall electronic device (100).

[0165] In one embodiment, the processor (120) may control an image processing module, an object recognition module, an object recognition module, etc. Each of these modules may correspond to a functional unit module implemented in software and / or hardware.

[0166] The image processing module is a module for extracting feature information related to the tongue contained in the image.

[0167] For example, the image processing module can extract feature information related to at least one of a tooth mark, a crack, and a stain included in the image using the first artificial intelligence model.

[0168] The object recognition module is a module for recognizing the tongue included in the image.

[0169] For example, an object recognition module can recognize an object by dividing an image into multiple unit areas and then identifying whether an object is included in each unit area.

[0170] Additionally, the object recognition module can recognize objects based on data acquired by moving a bounding box of a preset size pixel by pixel on the image.

[0171] The data analysis module is a module for predicting the onset of geriatric brain disease based on feature information extracted from the target image.

[0172] For example, the data analysis module can generate prediction information, including whether or not a geriatric brain disease has occurred, based on feature information extracted from the target image using a second artificial intelligence model.

[0173] Meanwhile, the processor (120) may include a general-purpose processor such as a CPU (Central Processing Unit), an AP, a DSP (Digital Signal Processor), a graphics-only processor such as a GPU (Graphics Processor Unit), a VPU (Vision Processing Unit), or an artificial intelligence-only processor such as an NPU (Neural Processing Unit). The artificial intelligence-only processor may be designed with a hardware structure specialized for training or using a specific artificial intelligence model.

[0174] The communication unit (130) is configured to perform communication with the outside.

[0175] The communication unit (130) may include circuits, modules, chips, etc. for performing communication using various wired and wireless communication methods. The communication unit (130) may also be connected to external devices and servers through various networks.

[0176] Depending on the area or scale, a network may be a personal area network (PAN), a local area network (LAN), or a wide area network (WAN), and depending on the openness of the network, it may be an intranet, an extranet, or the Internet.

[0177] The communication unit (130) can be connected to external devices and servers through various wireless communication methods such as LTE (long-term evolution), LTE-A (LTE Advance), 5G (5th Generation) mobile communication, CDMA (code division multiple access), WCDMA (wideband CDMA), UMTS (universal mobile telecommunications system), WiBro (Wireless Broadband), GSM (Global System for Mobile Communications), DMA (Time Division Multiple Access), WiFi (Wi-Fi), WiFi Direct, Bluetooth, BLE (Bluetooth Low Energy), NFC (near field communication), Zigbee, and LoRa.

[0178] Additionally, the communication unit (130) may be connected to external devices and servers through wired communication methods such as Ethernet, optical network, Universal Serial Bus (USB), and ThunderBolt.

[0179] In addition, the communication unit (130) may be configured to utilize various communication methods / technologies that will be newly designed in the future.

Claims

1. In the operating method of an electronic device that assists in the diagnosis of geriatric brain disease, The electronic device acquires at least one target image of the subject's tongue; A step in which the electronic device extracts feature information on at least one of a scratch, a crack, a coating, and a degree of bending from the acquired target image; and An operating method of an electronic device, comprising: a step of predicting whether the subject has developed a geriatric brain disease based on the extracted feature information; 2. In paragraph 1, The step of extracting the above feature information is: An operating method of an electronic device, wherein the electronic device inputs the target image into a first artificial intelligence model for extracting feature information from the image and obtains feature information about the target image from the first artificial intelligence model.

3. In paragraph 2, The step of predicting whether the above subject will develop geriatric brain disease is as follows: An operating method of an electronic device, wherein the electronic device inputs the feature information extracted from an image of a tongue into a second artificial intelligence model for predicting a disease, and obtains prediction information including whether the subject has developed a geriatric brain disease from the second artificial intelligence model.

4. In paragraph 3, The method of operating the above electronic device is as follows: The electronic device acquires training data including a plurality of images in which the tongues of each of a plurality of patients are photographed and disease information of the patient corresponding to the tongue included in each of the plurality of images; and An operating method of an electronic device, comprising: a step of training the second artificial intelligence model to predict the onset of a geriatric brain disease using the acquired training data; 5. In paragraph 3, The method of operating the above electronic device is as follows: A step of the electronic device receiving a plurality of verification images from a verification database in which disease information of each of a plurality of patients and a plurality of verification images of the tongues of each of the plurality of patients are matched and stored, and identifying the similarity of the target image to each of the received plurality of verification images; When a verification image having a similarity exceeding a threshold is identified, the electronic device acquires disease information matched with the verification image having a similarity exceeding a threshold, and identifies whether there is a match between the acquired disease information and the prediction information including the predicted onset; and An operating method of an electronic device, comprising: a step of re-inputting feature information extracted from the target image into the second artificial intelligence model to re-predict whether or not the subject has a geriatric brain disease, if the disease information and the prediction information do not match.

6. In paragraph 1, The method of operating the above electronic device is as follows: When a plurality of image frames in which the tongue of the subject is continuously photographed are acquired, the electronic device selects consecutive image frames corresponding to some of the plurality of frames based on the similarity of each of the plurality of image frames; The electronic device extracts feature information from the selected image frames; and An operating method of an electronic device, comprising: a step of predicting whether the subject has a geriatric brain disease based on feature information extracted from the selected image frames; 7. In paragraph 6, The step of selecting the above consecutive image frames is: Identifying the outline of the tongue contained in each of the plurality of image frames, Select the image frame with the largest size of the above-identified outline and set it as the target image frame, Identifying the similarity of the target image frame to each of the plurality of image frames, An operating method of an electronic device, wherein image frames consecutive to the target image frame are selected from among the image frames having a similarity level greater than a threshold value.

Images