Information processing device, information processing method, and information processing program

Abstract

To easily manage the physical condition of a user. [Solution] This information processing device includes an acquisition unit that acquires the results of user authentication based on a facial image of the user captured by an imaging unit included in the information processing device, a storage unit that acquires and stores features of the conjunctiva image included in the facial image if user authentication is successful, and a judgment unit that judges the health condition of the user based on the features of the conjunctiva image stored in the storage unit.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] In the above technical field, Patent Document 1 discloses a technique for imaging a user to assist treatment. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2021 / 070472 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in the above document requires a monitor device that monitors one or more physical condition indicators of the user, and a device that acquires the user's medication information, etc., and does not allow for easy management of the user's physical condition.

[0005] An object of the present invention is to provide a technique for solving the above-mentioned problems. [Means for solving the problem]

[0006] In order to achieve the above object, the device according to the present invention comprises: an acquisition unit that acquires a result of user authentication based on a face image of the user captured by an imaging unit included in the information processing device; a storage unit that acquires and stores features of a conjunctiva image included in the face image when the user authentication is successful; a determination unit that determines a health state of the user based on the feature amount of the conjunctiva image stored in the storage unit; Equipped with 、 The determination unit determines the health state of the user based on continuous changes in the feature amount.It is an information processing device.

[0007] In order to achieve the above object, the method according to the present invention comprises: an acquisition step of acquiring a result of user authentication based on a face image of the user captured by the imaging unit; a determining step of determining a health state of the user based on a feature amount of a conjunctiva image included in the face image when the user authentication is successful; An information processing method for making a computer execute And, The determining step determines the health condition of the user based on continuous changes in the feature amount. is.

[0008] In order to achieve the above object, the program according to the present invention comprises: an acquisition step of acquiring a result of user authentication based on a face image of the user captured by an imaging unit included in the information processing device; a determining step of determining a health state of the user based on a feature amount of a conjunctiva image included in the face image when the user authentication is successful; An information processing program that causes a computer to execute And, The determining step includes: determining the health condition of the user based on continuous changes in the feature amount; is. [Effects of the Invention]

[0009] According to the present invention, the user's physical condition can be easily managed. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to a first embodiment. [Figure 2] FIG. 10 is a block diagram showing the configuration of an information processing system according to a second embodiment. [Figure 3] FIG. 10 is a diagram illustrating a usage state of an information processing system according to a second embodiment. [Figure 4] FIG. 10 is a diagram illustrating a usage state of an information processing system according to a second embodiment. [Figure 5]10 is a flowchart showing the flow of processing in an information processing system according to a second embodiment. [Figure 6] FIG. 10 is a diagram showing a table configuration of an information processing system according to a second embodiment. [Figure 7] FIG. 10 is a block diagram showing the configuration of an information processing system according to a third embodiment. [Figure 8] FIG. 10 is a block diagram showing the configuration of an information processing system according to a fourth embodiment. [Figure 9] FIG. 11 is a block diagram showing the configuration of an information processing system according to a fifth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the components described in the following embodiments are merely examples and are not intended to limit the technical scope of the present invention.

[0012] [First embodiment] An information processing device 100 according to a first embodiment of the present invention will be described with reference to Fig. 1. The information processing device 100 is a device that performs face authentication of a user on a daily basis.

[0013] As shown in FIG. 1, the information processing device 100 includes an authentication result acquisition unit 101, a storage unit 102, and a determination unit 103.

[0014] The authentication result acquisition unit 101 acquires the result of user authentication based on a face image of the user captured by an imaging unit included in the information processing device.

[0015] If the authentication result acquired by the authentication result acquisition unit indicates that the user authentication has been successful, the accumulation unit 102 acquires and accumulates the feature amount of the conjunctiva image included in the face image.

[0016] The determining unit 103 determines the health condition of the user based on the feature amount of the conjunctival image stored in the storage unit 102.

[0017] According to this embodiment, the user's health condition can be checked during the process of face authentication, making it possible to keep track of changes in the user's health condition on a daily basis.

[0018] [Second embodiment] Next, an information processing system according to a second embodiment of the present invention will be described with reference to Fig. 2 and subsequent figures. Fig. 2 is a system configuration diagram, and Figs. 3 and 4 are diagrams for explaining how to use the information processing system according to this embodiment.

[0019] 2, information processing system 200 includes smartphone 201 and server 202. Smartphone 201 includes imaging unit 211, face authentication unit 212, authentication result acquisition unit 213, feature amount accumulation unit 214, health condition determination unit 215, and notification unit 216. It is desirable to package authentication result acquisition unit 213, feature amount accumulation unit 214, health condition determination unit 215, and notification unit 216 as a single application and provide it to the user to install on the smartphone.

[0020] The imaging unit 211 includes a camera capable of capturing still images and moving images and an image processing device.

[0021] The face authentication unit 212 is an application that is initially installed in the smartphone 201, and is a function that ensures the security of the smartphone 201 by performing face authentication instead of passkey entry or fingerprint authentication when the user unlocks the smartphone 201. Specifically, the face authentication unit 212 compares the facial features of the user that have been registered in advance with the facial features included in the image captured by the imaging unit 211.

[0022] The authentication result acquisition unit 213 acquires the result of user authentication by the face authentication unit 212. Specifically, it acquires the conclusion of whether the user authentication was successful or unsuccessful. If the user authentication was successful, it proceeds to health condition diagnosis processing.

[0023] When the authentication result acquisition unit 213 recognizes that face authentication has been successful, it may automatically switch the image capture settings (exposure, white balance, ISO, focal length, shooting distance, etc.) of the image capture unit 211 to predetermined fixed values ​​and acquire image data that makes it easy to extract image features of the conjunctiva of the eyeball. Alternatively, the settings may be switched as described above before face authentication. When switching the settings after face authentication and continuing image capture, the notification unit 216 may display, for example, a message such as "Face authentication has been successful. Next, a health diagnosis will be performed." By changing the image capture settings in this way, it is possible to reduce variations in the image capture environment and obtain conjunctival images with high reproducibility.

[0024] It is also possible to incorporate algorithms that combine sensor information (ambient light sensor, distance sensor, etc.) with the photographic data taken during face recognition to appropriately correct variations in photographic conditions. For example, it is preferable to perform automatic white balance correction, exposure correction, and optimization of the photographic position to consistently obtain high-quality images.

[0025] Furthermore, for example, color component values ​​of the iris of the eye may be extracted from an image of a healthy user captured under optimal conditions and registered as a reference value, and the feature values ​​of the conjunctiva image may be extracted from a facial image that has been adjusted so that the color component values ​​of the iris of the eye in the facial image always match the reference value, because the color components of the iris are considered to change less than the conjunctiva.

[0026] If the authentication result acquired by the authentication result acquisition unit 213 indicates a successful user authentication, the feature accumulation unit 214 extracts feature amounts from the image of the conjunctiva of the eyeballs included in the face image and stores the feature amounts in association with the user ID. Specifically, the feature accumulation unit 214 extracts the eyeballs from the face image captured by the imaging unit 211 during face authentication and acquires color data of the whites of the eyes. If the authentication result acquired by the authentication result acquisition unit 213 indicates a failed user authentication, the feature accumulation unit 214 does not extract feature amounts.

[0027] The feature amount storage unit 214 may store an estimated value of serum bilirubin calculated based on the yellow component value of the conjunctival image as a feature amount of the conjunctival image. A table or a calculation formula showing the correlation between the yellow component of the conjunctival image and the estimated value of serum bilirubin may be stored in advance.

[0028] The feature amount storage unit 214 may store, in addition to the above-mentioned feature amounts of the conjunctiva image, age, sex, race, location information and date and time information of the conjunctiva image capture, linked to the user ID. Age and sex can be obtained from information registered in the smartphone's OS (operating system), and location information and date and time information of the capture can be obtained from the camera app.

[0029] The health condition determination unit 215 determines the health condition of the user based on the feature amount of the conjunctival image stored in the storage unit 213. Specifically, if the density of the yellow component in the conjunctival image (white part of the eye) exceeds a threshold, it is determined that there is a possibility of jaundice.

[0030] The health condition determination unit 215 may estimate the concentration of a component in the user's blood based on the feature amount of the conjunctival image stored in the storage unit 213. The notification unit 216 urges the user to undergo a medical examination at a medical institution based on the estimated concentration of a component in the blood. The concentration of a component in the blood includes at least one value of bilirubin, hemoglobin, and oxygen saturation.

[0031] More specifically, the density of the yellow component of the conjunctival image (white of the eye) accumulated as a feature may be converted into an estimated value of serum bilirubin by the health condition determination unit 215. Specifically, based on the red component (R), green component (G), and blue component (B) extracted from the conjunctival image, the total bilirubin level (T-Bil) can be estimated using, for example, the following conditions: If R / G ≥ 2.0 and B < 100: T-Bil ≥ 3.0 mg / dL If 1.2 ≦ R / G < 2.0: T-Bil ≒ 2.0~3.0mg / dL If R / G < 1.2: T-Bil < 2.0 mg The notification unit 216 then compares the estimated serum bilirubin value with a predetermined threshold, and if it determines that the estimated serum bilirubin value exceeds the predetermined threshold, it notifies the user with an alert.

[0032] Jaundice, a condition in which the conjunctiva (whites of the eye) turn yellow, is known as a condition caused by an abnormal rise in bilirubin levels in the blood. The bilirubin that causes jaundice is a modified form of hemoglobin (Hb), which carries oxygen released from waste red blood cells broken down in the spleen, and indirect (unconjugated) bilirubin is conjugated in the liver to become direct (conjugated) bilirubin. Total bilirubin = direct bilirubin + indirect bilirubin. Bilirubin is a component of bile produced in the liver.

[0033] When the common hepatic duct or common bile duct becomes narrowed or blocked due to gallstones, inflammation, bile duct cancer, or compression, bilirubin backs up into the bloodstream, resulting in jaundice. Therefore, jaundice is an important sign of bile duct cancer, pancreatic head cancer, or ampullary cancer. Early detection of jaundice is highly desirable, especially since these cancers have no other obvious early symptoms, but the problem is that subtle increases in blood pressure cannot be detected visually.

[0034] However, even for experts, it is difficult to detect jaundice early. Even with a visual examination, doctors often only notice yellowing of the bulbar conjunctiva (white of the eye) when the total bilirubin in the blood reaches 7.0 mg / dl or higher.

[0035] In this embodiment, the "face recognition function" that is used on a daily basis is utilized to quickly detect abnormalities in the bulbar conjunctiva even when the total bilirubin in the blood is less than 7.0 mg / dl, making it possible to prompt the user to seek medical attention.

[0036] The health condition determination unit 215 may determine the user's health condition based on "changes" in the feature values ​​of the conjunctival images stored in the storage unit 213. That is, if the amount of change in the concentration of the yellow component in the conjunctival image or the rate of change over a predetermined period exceeds a threshold, the user may be prompted to seek medical treatment at a medical institution. Alternatively, if the continuous fluctuation in blood component concentration, specifically, the amount of change in the bilirubin value estimated from the concentration of the yellow component in the conjunctival image or the rate of change over a predetermined period exceeds a threshold, the user may be prompted to seek medical treatment at a medical institution. This enables early detection of jaundice with extremely high accuracy, regardless of race or environment. Even if the change within a day is very subtle, it is possible to determine whether the change is problematic by going back a predetermined period.

[0037] In addition, the server 202 may be provided with a feature accumulation unit 222 and a health condition determination unit 223. In this case, the feature accumulation unit 222 accumulates features (which may be concentrations of components in the blood) of conjunctival images collected from multiple users as big data, and the health condition determination unit 223 may use the big data to determine the user's health condition and transmit it to the smartphone 201.

[0038] Figure 3 shows how the system is used by a user. When a user 310 using a smartphone 201 takes a picture of their own face for facial recognition using the imaging unit 211, the facial recognition application in the smartphone 201 uses facial recognition technology to determine whether the user is an authorized user.

[0039] If the authentication result indicates that the user is indeed a legitimate user of the smartphone 201, it is determined whether or not jaundice is present, and if it is determined that jaundice is present, a message 320 is displayed to prompt the user 310 to seek a diagnosis at a medical institution.

[0040] 4 is a diagram showing how the system is used when it is incorporated into a personal computer 401 instead of a smartphone. When a user 310 using the personal computer 401 takes a picture of his or her own face for face authentication using an image capturing unit 411, a face authentication application in the personal computer 401 uses face authentication technology to determine whether or not the user is an authorized user.

[0041] If the authentication result indicates that the user is a legitimate user, it is determined whether or not jaundice is present, and if it is determined that jaundice is present, a message 420 is displayed to prompt the user 310 to seek a diagnosis at a medical institution.

[0042] Even people who do not normally visit a hospital can make a preliminary jaundice diagnosis simply by using a smartphone or personal computer, making it possible to effectively protect the user's health.

[0043] In addition to the messages 320 and 420 described above, a graph showing the change in the estimated bilirubin value may be displayed, or a comparison image with a facial image on a day when the estimated bilirubin value was low may be displayed to the user, and these images may be sent to a medical institution or an industrial physician at the workplace.

[0044] At the same time, the user may be asked to enter other subjective symptoms such as fatigue and weight loss, family medical history (family history of pancreatic cancer), oral medications, etc., and send these to a medical institution.

[0045] Blood sampling data obtained during health checkups or hospital visits may be input and linked to the features of conjunctival images and estimated bilirubin values. Specifically, accurate bilirubin values ​​may be obtained by blood sampling, and the bilirubin estimation method (calculation formula) or bilirubin estimation model may be updated.

[0046] 5 is a flowchart showing the processing flow in the present system 200. If face authentication is successful in step S501, feature extraction of the conjunctival region is performed in step S503, and the bilirubin level is estimated in step S505. Next, in step S507, it is determined whether the estimated bilirubin level is equal to or greater than a predetermined value. If it is equal to or greater than the predetermined value, the process proceeds to step S509, where a warning message is displayed, urging the user to seek medical attention.

[0047] 6 is a diagram showing an example of a table 600 serving as the feature amount storage unit 214. The table 600 stores the photographing date and time, age, sex, conjunctival feature amount, and estimated bilirubin value, linked to a user ID.

[0048] According to this embodiment, the user's health condition can be checked during the facial recognition process, making it possible to keep track of changes in the user's health condition on a daily basis. In particular, because the facial recognition system is used daily, data can be automatically accumulated over time without imposing a burden on the user. Feature data is accumulated on the cloud, and numerical fluctuations in jaundice (increasing or decreasing trends) are analyzed. If abnormal values ​​or persistent changes are detected, notifications can be sent within the app or software, or the system can be linked to a health management service, and the results can be displayed to the user using numerical values ​​and graphs, encouraging them to visit a hospital as soon as possible.

[0049] Bilirubin levels can temporarily rise due to liver damage caused by drugs or viral infections. For example, if a patient undergoes a health check after a viral infection and their bilirubin levels are abnormal, they may visit a hospital and undergo detailed testing, but the level may have naturally returned to normal, resulting in unnecessary further testing.

[0050] This system can automatically and continuously estimate bilirubin levels without placing a burden on the user. This makes it possible to determine whether the increase in bilirubin levels is a transient one or a sustained increase over several weeks. Furthermore, by referencing the results of multiple measurements, it is possible to reduce the possibility of false positives and false negatives that can occur with a single estimation, resulting in a more accurate model.

[0051] Jaundice is generally judged by the whites of the eyes, but the present invention is not limited to this and a highly accurate judgment can be made by combining the color of the entire face. Some people have a constitutionally high level of bilirubin and yellowish conjunctiva, but by examining the amount or rate of change, jaundice can be detected with high accuracy even in such people.

[0052] [Third embodiment] Next, an information processing system 700 according to a third embodiment of the present invention will be described with reference to FIG. 7. FIG. 7 is a diagram illustrating the configuration of the information processing system according to this embodiment. The information processing system according to this embodiment differs from the second embodiment in that it utilizes position information to improve the accuracy of jaundice detection. Other configurations and operations are similar to those of the second embodiment, and therefore the same configurations and operations are denoted by the same reference numerals and detailed descriptions thereof will be omitted.

[0053] 7, the information processing system 700 further includes a location information acquisition unit 717 that acquires location information at the time of user authentication. The health condition determination unit 215 determines the user's health condition using the location information in addition to the feature amount of the conjunctiva image.

[0054] Specifically, position information at the time of face authentication is stored in the feature amount storage unit 214, and conjunctival images captured at the same position are preferentially used to derive the estimated bilirubin value. Furthermore, conjunctival images captured within a predetermined time range may be preferentially used to derive the estimated bilirubin value.

[0055] For example, if this system is installed on a company's personal computer, it is desirable to make the judgment when the person arrives at work, and if it is installed on a personal smartphone, it is desirable to make the judgment when the person wakes up. By using location information, it is possible to standardize the shooting conditions, enabling more accurate judgment.

[0056] [Fourth embodiment] Next, an information processing system 800 according to a fourth embodiment of the present invention will be described with reference to Fig. 8. Fig. 8 is a diagram illustrating the configuration of the information processing system 800 according to this embodiment. The information processing system 800 according to this embodiment differs from the second embodiment in that it has a function for directly notifying medical institutions. Other configurations and operations are the same as those of the second embodiment, so the same configurations and operations are denoted by the same reference numerals and detailed descriptions thereof will be omitted.

[0057] The smartphone 801 is equipped with a transmission unit 817 that automatically transmits, with the user's consent, the captured conjunctival image itself and estimated data on blood component concentrations such as estimated bilirubin levels to the medical institution's server 802.

[0058] This will enable support for online medical consultations and remote monitoring, as well as reduce the burden on patients when explaining the possibility of jaundice during consultations.

[0059] Furthermore, in this system, if a user periodically measures bilirubin, hemoglobin, etc. through blood tests, the actual measured values ​​can be linked to the system to construct a data set specific to each individual user. In this embodiment, by incorporating each user's actual test results as feedback, it is possible to optimize the model specific to that user, thereby improving estimation accuracy.

[0060] [Fifth embodiment] Next, an information processing system 900 according to a fifth embodiment of the present invention will be described with reference to Fig. 9. Fig. 9 is a diagram illustrating the configuration of the information processing system 900 according to this embodiment. The information processing system 900 according to this embodiment differs from the second embodiment in that it estimates bilirubin levels using machine learning. Other configurations and operations are similar to those of the second embodiment, and therefore the same configurations and operations are denoted by the same reference numerals and detailed descriptions thereof will be omitted.

[0061] The smartphone 901 is equipped with a machine learning unit 917 that extracts features from captured facial images using a learning model generated by a convolutional neural network (CNN). The machine learning unit 917 then connects and integrates the features with a numerical vector obtained by preprocessing patient-specific data such as age, gender, underlying diseases, and other data. The integrated features are trained through a fully connected layer, and ultimately output a numerical prediction of jaundice, such as bilirubin levels.

[0062] Alternatively, the machine learning unit 917 estimates the bilirubin level using a learning model generated by a convolutional neural network (CNN), and then connects and integrates the estimated bilirubin level with a numerical vector obtained by preprocessing age, sex, underlying diseases, and other patient-specific data to calculate the possibility of disease.

[0063] The server 902 may be provided with a feature accumulation unit 921 that accumulates big data showing the relationship between bilirubin measurement values ​​and image feature amounts (yellow components in the whites of the eyes) of a large number of users (e.g., 300 or more). In this case, the health condition determination unit 922 can estimate the bilirubin level of each individual user with high accuracy using a learning model that has learned the big data from the feature accumulation unit 921.

[0064] [Other embodiments] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. For example, eyelid edema, dark circles, and other pigmentation abnormalities (e.g., redness, age spots, acne) may be analyzed and diagnosed from a facial image captured during face recognition, and if necessary, the user may be encouraged to see a doctor at a medical institution or receive advice on sleep and skin care. Because a strong correlation between eyelid edema and physical condition has been discovered, it is also possible to predict fever by detecting the degree of eyelid edema from a facial image (for example, a person with single eyelids may develop double eyelids due to edema, or a person with double eyelids may develop triple eyelids).

[0065] (Hemoglobin level determination) The red color of blood is primarily due to the high concentration of hemoglobin present within red blood cells. Hemoglobin is a pigmented protein containing iron that can reversibly bind oxygen. Its light absorption properties are known to be responsible for its red color. Because plasma and other blood components (e.g., white blood cells and platelets) do not typically contain hemoglobin, the red color of blood is highly correlated with the abundance of red blood cells, i.e., the hemoglobin concentration. (RGB camera-based simultaneous measurements of percutaneous arterial oxygen saturation, tissue oxygen saturation, pulse rate, and respiratory rate, 19 September 2022 Sec. Physio-logging Volume 13 - 2022 https: / / doi.org / 10.3389 / fphys.2022.933397 by Izumi Nishidate, Riku Yasui, Nodoka Nagao, Haruta Suzuki, Yohei Takara, Kaoru Ohashi, Fuminori Ando, ​​Naoki Noro, Yasuaki Kokubo)

[0066] Therefore, this system may estimate the blood hemoglobin concentration by acquiring color information from the capillaries of the bulbar conjunctiva during face recognition. Specifically, the relationship between the red component and the luminance component (Y = 0.299R + 0.587G + 0.114B) or the red saturation of the image may be considered. For example, the hemoglobin concentration may be estimated under the following conditions: if R ≥ 200: Hb ≥ 13.0 g / dL, if R < 200 and Y ≥ 130: Hb ≒ 10.0-13.0 g / dL, and if R < 200 and Y < 130: Hb < 10.0 g / dL.

[0067] By estimating hemoglobin concentration in this way, this system can be used to determine anemia. There are many causes of anemia, but anemia in the elderly, for example, can be caused by digestive cancer (stomach cancer, colon cancer). However, because such anemia is relatively mild and progresses slowly, there are no noticeable symptoms, and blood tests are often not required, leading to delayed detection. This system can detect changes in hemoglobin levels over time without placing a burden on the user, leading to early hospital visits and further investigation of the cause.

[0068] (Oxygen saturation assessment) Oxygen saturation (SpO2) is the percentage of oxygenated hemoglobin that is bound to oxygen among the hemoglobin molecules present in the blood. In other words, SpO2 is defined as follows: SpO2 = [oxygenated hemoglobin] / [oxygenated hemoglobin + reduced hemoglobin] × 100 (%). In this way, SpO2 is the ratio of oxygenated hemoglobin to reduced hemoglobin, and is an index that reflects the state of hemoglobin in arterial blood. For example, if all hemoglobin in the blood is estimated to be oxygenated hemoglobin, the oxygen saturation is approximately 100%. If 90% of the hemoglobin in the blood is estimated to be oxygenated hemoglobin and 10% is reduced hemoglobin, the oxygen saturation can be estimated to be approximately 90%.

[0069] This system may estimate the oxygen saturation of arterial blood by estimating the proportion of oxygenated hemoglobin. Specifically, arteries in the conjunctiva of the eye may be identified in a facial image, and the oxygen saturation may be estimated from the color of the arteries. The system utilizes the color change between oxygenated hemoglobin (bright red) bound to oxygen and reduced hemoglobin (dark red to purple) that is not bound to oxygen. For example, focusing on the ratio of red and blue components, oxygen saturation can be estimated based on an estimation formula such that if R / B≧2.0, the oxygen saturation is approximately 95% or higher; if 1.5≦R / B<2.0, the oxygen saturation is approximately 90% or higher but less than 95%; and if R / B<1.5, the oxygen saturation is less than approximately 90%.

[0070] Furthermore, color shading can be used to help distinguish between arteries and veins. For example, by selecting areas where R / B > 1.0, oxygen saturation estimates exceed 85%. Because normal oxygen saturation in venous blood is around 70-80%, setting such a standard can accurately estimate arterial blood oxygen saturation. Current methods for measuring oxygen saturation use pulse oximeters, which require a sensor attached to the user's fingertip. These devices are rarely owned or used by ordinary users. For patients with chronic obstructive pulmonary disease (COPD), which is caused by smoking, early smoking cessation and medication improve prognosis. However, because smokers adapt to chronic hypoxia and experience few subjective symptoms, many cases are overlooked and do not seek medical attention. For such patients, our system can estimate oxygen saturation during facial recognition without burdening the user, potentially leading to behavioral changes such as early medical consultation and smoking cessation.

[0071] In either case, more accurate estimation is possible by learning from the measurement values ​​of the corresponding blood components (hemoglobin concentration, oxygen saturation) as training data.

[0072] The configuration and details of the present invention may be modified in various ways that are understandable to those skilled in the art within the technical scope of the present invention. Furthermore, any system or device that combines the individual features included in each embodiment is also included in the technical scope of the present invention.

[0073] The present invention may also be applied to a system consisting of multiple devices or to a single device. Furthermore, the present invention may also be applied when an information processing program that realizes the functions of the embodiments is supplied to a system or device and executed by a built-in processor. The technical scope of the present invention also includes a program installed on a computer to realize the functions of the present invention, a medium storing the program, a server from which the program is downloaded, and a processor that executes the program. In particular, the technical scope of the present invention includes at least a non-transitory computer-readable medium storing a program that causes a computer to execute the processing steps included in the above-described embodiments.

Claims

1. an acquisition unit that acquires a result of user authentication based on a face image of the user captured by an imaging unit included in the information processing device; a storage unit that acquires and stores features of a conjunctiva image included in the face image when the user authentication is successful; a determination unit that determines a health state of the user based on the feature amount of the conjunctiva image stored in the storage unit; Equipped with The determination unit is an information processing device that determines the health state of the user based on continuous changes in the feature amount.

2. the determination unit determines a health state of the user based on a component concentration in the user's blood estimated from the conjunctiva image as a feature amount of the conjunctiva image; The information processing device according to claim 1 , further comprising a notification unit that prompts the user to have a medical examination at a medical institution based on the health condition determined by the determination unit.

3. The information processing device according to claim 2 , wherein the blood component concentration includes at least one value of bilirubin, hemoglobin, and oxygen saturation.

4. The information processing device according to claim 1 or 2, wherein the determination unit estimates the component concentration in the user's blood using a learning model generated by learning the relationship between the feature amount of the conjunctiva image and the component concentration in the user's blood.

5. The information processing device according to claim 1 , wherein the determining unit estimates the concentration of a component in the blood of the user using a relationship between a feature amount of conjunctival images collected from a plurality of users and the concentration of the component in the blood.

6. a location information acquisition unit that acquires location information at the time of performing the user authentication, The information processing device according to claim 1 , wherein the determining unit determines the health condition of the user by using the location information in addition to the feature amount of the conjunctiva image.

7. the determination unit estimates a concentration of a component in the blood of the user based on a feature amount of the conjunctiva image stored in the storage unit; The information processing device according to claim 1 , further comprising a transmitting unit that transmits the estimated blood constituent concentrations to a medical institution.

8. The determination unit determines whether or not the user is exhibiting jaundice from the conjunctival image, The information processing apparatus according to claim 1 , further comprising a notification unit that, when it is determined that the user is suffering from jaundice, notifies the user of this and urges the user to seek medical attention at a medical institution.

9. an acquisition step of acquiring a result of user authentication based on a face image of the user captured by the imaging unit; a determining step of determining a health state of the user based on a feature amount of a conjunctiva image included in the face image when the user authentication is successful; An information processing method for causing a computer to execute the following: The determining step is an information processing method for determining the health state of the user based on continuous changes in the feature amount.

10. an acquisition step of acquiring a result of user authentication based on a face image of the user captured by an imaging unit included in the information processing device; a determining step of determining a health state of the user based on a feature amount of a conjunctiva image included in the face image when the user authentication is successful; An information processing program that causes a computer to execute the following: The determining step is an information processing program for determining the health state of the user based on continuous changes in the feature amount.

Images