Information processing system, information processing method, and program

Abstract

This information processing system (1) is provided with: a registration unit (301) that acquires, for each patient, a reference image in which the patient is imaged and state information associated with at least one of the state of physical condition of the patient and the situation of progression of the patient at the time of the capturing of the image; a model production unit (304) that produces, for each patient, a physical condition estimation model for estimating the physical condition of the patient on the basis of the reference image and the state information; a physical condition information production unit (307) that produces information associated with the physical condition of a target patient by inputting a captured image or predetermined state information of the target patient to a physical condition estimation model for the target patient; and an output control unit (308) that outputs information associated with the physical condition of the target patient.

Description

[Technical field]

[0001] The present disclosure relates to an information processing system, an information processing method, and a non-transitory computer-readable medium, and more particularly to an information processing system, an information processing method, and a non-transitory computer-readable medium for estimating a patient's physical condition. [Background technology]

[0002] It is necessary to monitor the physical condition of patients after discharge from hospital and take early action if their physical condition deteriorates or if their medicine is not effective. For example, Patent Document 1 discloses an estimation device that estimates the physical condition of a subject using the skin cell image data of the subject as input data, using an estimation model that is learned by collecting teacher data that is a pair of skin cell image data and physical condition data of multiple people.

[0003] For example, Patent Document 2 discloses a system that determines whether a caregiver needs to be notified about a worsening of a patient's condition based on individual values ​​corresponding to time in a sequence of quantitative values ​​of the patient's condition.

[0004] Incidentally, Patent Document 3 discloses a method for identifying a patient from dental image data, which is image data of the patient's body. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-085856 A [Patent Document 2] JP 2020-000871 A [Patent Document 3] JP 2020-108598 A Summary of the Invention [Problem to be solved by the invention]

[0006] Here, there is a need for the patient or a person related to the patient to easily understand the level of deterioration of physical condition and the effectiveness of medicine without requiring special equipment. However, the above-mentioned Patent Document 1 requires a dedicated device for acquiring cell image data, and the above-mentioned Patent Document 2 requires a dedicated device for quantifying symptoms. Therefore, it is difficult to apply this system at home or in a simple clinic where the equipment is insufficient.

[0007] Furthermore, there are individual differences in outward reactions and facial colors to deterioration of physical condition or pain, and the method described in the above-mentioned Patent Document 2 does not take individual differences into consideration.

[0008] In view of the above-mentioned problems, an object of the present disclosure is to provide an information processing system, an information processing method, and a non-transitory computer-readable medium that can perform personalized physical condition estimation for individual patients using simple equipment. [Means for solving the problem]

[0009] An information processing system according to one aspect of the present disclosure includes: a registration means for acquiring, for each patient, a reference image of the patient and status information relating to at least one of the physical condition and progress of the patient at the time of the image acquisition; a model generating means for generating, for each patient, a physical condition estimation model for estimating a physical condition of the patient based on the reference image and the condition information; a physical condition information generating means for generating information related to the physical condition of the target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; an output control means for outputting information related to the physical condition of the subject patient; Equipped with.

[0010] An information processing method according to one aspect of the present disclosure includes: For each patient, a reference image of the patient and status information regarding the physical condition or progress of the patient at the time of the image acquisition are obtained; generating a physical condition estimation model for estimating a physical condition of each patient based on the reference image and the condition information; generating information related to the physical condition of the target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; Information relating to the subject patient's physical condition is output.

[0011] According to one aspect of the present disclosure, there is provided a non-transitory computer-readable medium, comprising: A procedure for acquiring, for each patient, a reference image of the patient and status information relating to the physical condition or progress of the patient at the time of the image acquisition; generating, for each patient, a physical condition estimation model for estimating a physical condition of the patient based on the reference image and the condition information; A step of generating information related to the physical condition of the target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; outputting information related to the physical condition of the target patient; A program for causing a computer to execute the above is stored. Effect of the Invention

[0012] The present disclosure makes it possible to provide an information processing system, an information processing method, and a non-transitory computer-readable medium that can perform personalized physical condition estimation for individual patients using simple equipment. [Brief description of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a configuration of an information processing system according to a first embodiment. [Diagram 2] 1 is a flowchart showing the flow of an information processing method according to the first embodiment. [Diagram 3] FIG. 11 is a block diagram showing the overall configuration of an information processing system according to a second embodiment. [Figure 4] FIG. 11 is a block diagram showing a configuration of a server according to the second embodiment. [Diagram 5] FIG. 11 is a sequence diagram showing an example of the flow of a registration process according to the second embodiment. [Figure 6] FIG. 11 is a diagram showing an example of a display of a hospital terminal according to the second embodiment. [Figure 7] FIG. 11 is a sequence diagram showing an example of the flow of a process for outputting health-related information according to the second embodiment. [Figure 8] FIG. 11 is a diagram showing an example of a display of a patient terminal according to the second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a display of a patient terminal according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a display of a patient terminal according to the second embodiment. [Figure 11] FIG. 11 is a block diagram showing a configuration of a server according to a third embodiment. [Figure 12] FIG. 13 is a sequence diagram showing an example of the flow of a process for outputting health-related information according to the third embodiment. [Figure 13] FIG. 13 is a diagram showing an example of a display of a patient terminal according to the third embodiment. [Figure 14] FIG. 1 illustrates an example of the configuration of a computer. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and duplicated descriptions will be omitted as necessary for clarity of explanation.

[0015] <Embodiment 1> First, a first embodiment of the present disclosure will be described. Fig. 1 is a block diagram showing a configuration of an information processing system 1 according to the first embodiment. The information processing system 1 is a computer system including one or more computer devices for estimating a state related to the physical condition of a patient. In the following, the "state" refers to a physical condition or a progress status.

[0016] The information processing system 1 includes a registration unit 301 , a model generation unit 304 , a physical condition information generation unit 307 , and an output control unit 308 .

[0017] The registration unit 301 is also called a registration means. The registration unit 301 acquires, for each patient, a reference image of the patient and condition information of the patient at the time of capturing the reference image. The reference image is an image captured by a camera of all or part of the patient's body. The reference image may be a still image or a video. Hereinafter, the part to be captured, that is, the part corresponding to the image area included in the reference image, may be referred to as the target part. The target part may include a part where the patient's complexion or facial expression can be detected, or a part used by medical personnel to judge the case of the patient. As an example, the target part may be the face, eyelids, arms, legs, or neck. Note that when the camera captures the entire patient's body, the target part is the whole body. The time of shooting may be the time of shooting, or may be any time within a predetermined period from the time of shooting. The condition information is, for example, information on at least one of a physical condition and a progress status. The physical condition may indicate a condition level related to a physical condition, such as a symptom level, a disease progression level, a disease recovery level, an injury damage level, or a consciousness level. The condition level may include, for example, a good physical condition, a normal physical condition, or a bad physical condition. The physical condition may also be the effectiveness of a medicine or treatment for an injury or illness. Here, the progress status may be the patient's status at the time of imaging, with a certain status as a starting event. The information on the progress status may include, for example, information indicating that imaging was performed at the time of admission, on the nth day of admission (n is a natural number), or at the time of discharge. In this case, the information on the progress status indicates a status suggesting the progression of the disease, and the starting event is admission. As another example, the information on the progress status may include information regarding the time elapsed since taking medication or treatment. In this case, the information on the progress status indicates a status suggesting the progression of physical condition after taking medication or treatment, and the starting event is taking medication or treatment. The condition information may also be information that combines the physical condition and progress.

[0018] Then, the registration unit 301 registers a set of the reference image and condition information of the patient in a database (DB) (not shown).

[0019] The model generating unit 304 is also called a model generating means. The model generating unit 304 generates a physical condition estimation model for estimating the physical condition of a patient based on a set of reference images and condition information acquired by the registration unit 301 for each patient and registered in the DB. Details of the input and output of the physical condition estimation model will be described later.

[0020] The physical condition information generating unit 307 is also called a physical condition information generating means. The physical condition information generating unit 307 generates information related to the physical condition of the target patient using a physical condition estimation model of the target patient. The information related to the physical condition is also called physical condition related information. The physical condition related information may be various information related to the physical condition, and may be, for example, the following (Case 1) or (Case 2).

[0021] (Case 1) The health-related information may be estimated condition information, which is condition information estimated from a health estimation model, or may be information generated based on the estimated condition information. The estimated condition information may indicate at least one of the patient's current health condition and progress status. When the estimated condition information indicates the patient's current health condition, the information generated based on the estimated condition information may be, for example, information indicating which past condition (for example, at the time of admission, the nth day of admission, or at the time of discharge) the patient's current health condition is similar to. In the above case, the information generated based on the estimated condition information may be information indicating whether or not patient P needs to be examined by a doctor. In the case of (Case 1), the above-mentioned physical condition estimation model is a first physical condition estimation model that receives a photographed image of the target part of the patient as an input and outputs estimated condition information of the patient. The input photographed image may be a still image or a video. The first physical condition estimation model may include a convolutional neural network (CNN). The first physical condition estimation model may also be a regression model using a regression equation that indicates the relationship between the feature amount or the amount of change of the target part of the patient included in the photographed image and the condition at the time of photographing. The reference image and condition information registered in the DB are used to generate the first physical condition estimation model. The physical condition information generation unit 307 is configured to use the estimated condition information, which is the output result of the first physical condition estimation model, as physical condition-related information, or to generate physical condition-related information based on the estimated condition information.

[0022] (Case 2) The health-related information may be a simulation image that serves as a guide for estimating the health condition. The simulation image may be used by the patient, the patient's family, or other related parties to compare with the patient's current appearance in order to determine whether the patient needs to be examined by a doctor. The simulation image may also be used by doctors or medical staff at a hospital to compare with the patient's current appearance in order to easily grasp the effectiveness of a medicine or treatment, for example, the effectiveness of anesthesia. In the case of (Case 2), the above-mentioned health condition estimation model is a second health condition estimation model that receives predetermined condition information as input and outputs a simulation image of the target part of the patient. The second health condition estimation model may include a Generative Adversarial Network (GAN) or a CNN decoder-type network. The health condition information generation unit 307 can obtain a simulation image as health condition related information by inputting predetermined condition information into the second health condition estimation model. The reference image and condition information registered in the DB are used to generate the second health condition estimation model.

[0023] The output control unit 308 is also called an output control means. The output control unit 308 outputs the health-related information of the target patient generated by the health information generating unit 307. The output may be a transmission, a transmission to a predetermined display device for display, or, if the health-related information is text data, a transmission to a predetermined audio output device for output. For example, the output control unit 308 may transmit the health-related information of the target patient to a terminal used by the target patient or the target patient's family. In addition, for example, the output control unit 308 may transmit the health-related information of the target patient to a terminal managed by the hospital where the target patient was admitted or the family hospital. The terminal that receives the health-related information may display the health-related information of the target patient, or, if the health-related information is text data, may output the health-related information of the target patient by voice.

[0024] 2 is a flowchart showing the flow of the information processing method according to the first embodiment. First, the information processing system 1 repeats the processes shown in S10 to S11 for each patient. In S10, the registration unit 301 of the information processing system 1 acquires a reference image and condition information of the patient. Then, the registration unit 301 accumulates a set of the reference image and condition information of the patient in a DB. Next, in S11, the model generation unit 304 generates a physical condition estimation model based on the reference image and condition information registered in the DB.

[0025] Next, in S12, the physical condition information generating unit 307 generates physical condition related information of the target patient by inputting the captured image of the target part of the target patient or predetermined condition information to the physical condition estimation model of the target patient. Then, in S13, the output control unit 308 outputs the physical condition related information to the destination terminal.

[0026] Thus, according to the first embodiment, the information processing system 1 generates and outputs information related to the patient's physical condition using a physical condition estimation model generated for each patient based on the reference image and the condition information. Therefore, the information processing system 1 can generate a physical condition estimation model personalized for each patient using a simple device such as a camera. The information processing system 1 also generates physical condition related information by inputting a captured image or condition information. Therefore, the information processing system 1 can provide physical condition related information personalized for each patient using a simple device such as a camera or an input device. In particular, at home or in a simple clinic where the equipment is insufficient, the patient's physical condition can be easily understood, and it is possible to respond early when the patient's physical condition deteriorates or when the medicine is not effective. Furthermore, the information processing system 1 is useful not only at home or in a simple clinic, but also when doctors and medical staff at a hospital simply grasp the patient's physical condition and the effectiveness of the medicine.

[0027] <Embodiment 2> Next, a second embodiment of the present disclosure will be described. The second embodiment is a specific example in which the health-related information is the above-mentioned (Case 1). That is, in the second embodiment, the health condition estimation model is a first health condition estimation model that receives an image of a target part of a patient as an input and outputs estimated condition information of the patient.

[0028] FIG. 3 is a block diagram showing the overall configuration of an information processing system 1a according to the second embodiment. The information processing system 1a is an example of the above-mentioned information processing system 1. The information processing system 1a includes a plurality of patient systems 10-1, 10-2, and 10-3, a hospital system 20, and an information processing device (hereinafter referred to as a server) 300. Each device and system is connected to a wired or wireless network N. Note that the number of patient systems 10 is an example and is not limited to this. In the following description, the family of the patient P, the medical staff of the simple clinic, or other doctors and medical staff who wish to know the health-related information are collectively referred to as the related parties.

[0029] (Hospital System 20) The hospital system 20 is a computer system of a hospital where the patient P is admitted or where the patient P is receiving treatment. The hospital system 20 acquires reference images of the patient P who is admitted or undergoing a medical examination, associates the images with physical condition-related information, and transmits the images to the server 300.

[0030] Specifically, the hospital system 20 includes a camera 210 and a hospital terminal 200 .

[0031] Camera 210 is installed in a hospital. For example, camera 210 is installed in a room, examination room, or examination room of hospitalized patient P. As an example, camera 210 captures a target part of patient P lying on a bed in a room. As another example, camera 210 captures a target part of patient P during examination or treatment. As another example, camera 210 captures a target part of patient P during examination in an examination device. Camera 210 is connected to hospital terminal 200, and transmits a reference image generated by capturing the image to hospital terminal 200.

[0032] The hospital terminal 200 is an information terminal provided in a hospital, or an information terminal managed by a doctor or other staff of the hospital. For example, the hospital terminal 200 is a personal computer, a smartphone, or a tablet terminal. The hospital terminal 200 is connected to a network N. The hospital terminal 200 acquires a reference image of the patient P from the camera 210. The hospital terminal 200 also acquires status information corresponding to the reference image. For example, the hospital terminal 200 acquires the above-mentioned status information by accepting an input of status information close to the time when the reference image was taken from a doctor or staff of the hospital. For example, the hospital terminal 200 may read out chart information, which is information written in the chart of the patient P, and may acquire the above-mentioned status information by extracting status information close to the time when the reference image was taken from the chart information of the patient P. The time close to the time of taking the image may refer to any time within a predetermined period from the time of taking the image. The hospital terminal 200 then transmits an image registration request including information associating the reference image with the status information for each patient to the server 300 via the network N.

[0033] (Patient System 10) The patient system 10 is a computer system in the home of the patient P. Alternatively, however, the patient system 10 may be a computer system in a simple clinic or other remote facility. For example, the patient system 10 transmits to the server 300 photographed images of a target area of ​​the patient P after discharge or examination at a hospital, and receives from the server 300 physical condition-related information generated based on the photographed images.

[0034] Specifically, the patient system 10 includes a camera 110 and a patient terminal 100 .

[0035] The camera 110 is connected to the patient terminal 100. The camera 110 photographs a target body part of the patient P who is remotely discharged from the hospital in response to an operation on an application by the patient P or a person related to the patient P, or under automatic control by the application. The camera 110 transmits a photographed image generated by photographing to the patient terminal 100. The camera 110 may be integrally mounted on the patient terminal 100.

[0036] The patient terminal 100 is an information terminal used by the patient P or a person related to the patient P. For example, the patient terminal 100 is a personal computer, a smartphone, or a tablet terminal. The patient terminal 100 is connected to a network N. The patient terminal 100 starts an application and acquires, from the camera 110, a photographed image of a target part of the patient P after discharge from the hospital or after a medical examination at the hospital. The patient terminal 100 then transmits an output request for health-related information, including the photographed image and the patient ID, to the server 300. The patient ID is information for identifying the patient, and may be the patient's name, the patient card number, or another identification number. The patient terminal 100 receives the physical condition related information of the patient P from the server 300 that has received the output request. Then, the patient terminal 100 displays the information on a display unit (not shown) or outputs the information as sound to an audio output unit (not shown).

[0037] (Server 300) The server 300 is a computer device for estimating the physical condition of each patient. When the server 300 receives an image registration request from the hospital terminal 200, the server 300 generates learning data for the patient P in which the reference image and the condition information are associated with each other. The server 300 then generates a first physical condition estimation model for estimating the condition of the patient P based on the learning data for the patient P. The server 300 then generates the first physical condition estimation model for each patient.

[0038] Furthermore, when the server 300 receives an output request for health-related information from the patient terminal 100, the server 300 generates health-related information for the target patient using the first health condition estimation model for the patient P. Then, the server 300 transmits the health-related information to the patient terminal 100.

[0039] 4 is a block diagram showing the configuration of the server 300 according to the embodiment 2. The server 300 includes a registration unit 301a, a learning DB 302, a model generation unit 304a, an estimation model DB 305, an information acquisition unit 306, a physical condition information generation unit 307a, and an output control unit 308a.

[0040] The registration unit 301a is an example of the registration unit 301 described above. When the registration unit 301a receives a registration request (patient registration request) for each patient P from the hospital terminal 200, the registration unit 301a determines the target part of the patient P. For example, when the registration unit 301a acquires a target part designated by a doctor or staff of the hospital from the hospital terminal 200, the registration unit 301a may determine the designated target part as the target part of the patient P. In addition, for example, the registration unit 301a may acquire medical record information of the patient P from the hospital terminal 200 and determine the target part based on the medical record information of the patient P. As an example, the registration unit 301a may determine the target part according to the disease name of the patient P included in the medical record information as the target part of the patient P. When linked with the medical record information, it is possible to eliminate the effort of inputting by a doctor or medical staff. Furthermore, when the registration unit 301a receives an image registration request from the hospital terminal 200, the registration unit 301a generates learning data by labeling the reference image included in the image registration request with the state information included in the image registration request. In addition, in the second embodiment, the state information may be a value, vector, or matrix that quantifies the state, or a class name to which the state belongs. The registration unit 301a associates the learning data with a patient ID and registers it in the learning DB 302.

[0041] The learning DB 302 is a storage device that stores learning data of a plurality of patients. The learning DB 302 stores a patient ID 3020, a reference image 3021, and status information 3022 in association with each other. The reference image 3021 and the status information 3022 are learning data.

[0042] The model generating unit 304a is an example of the model generating unit 304 described above. The model generating unit 304a learns a first physical condition estimation model for each patient using the learning data included in the learning DB 302. Learning the first physical condition estimation model may mean optimizing the parameters of the first physical condition estimation model. For example, the model generating unit 304a may learn all the parameters of the predetermined first physical condition estimation model for the patient P using the learning data of the patient P. Alternatively, the model generating unit 304a may learn some of the parameters of the predetermined first physical condition estimation model for the patient P using the learning data of the patient P. In this way, the model generating unit 304a can generate a first physical condition estimation model for each patient. Then, the model generating unit 304a stores the first physical condition estimation model for each patient in the estimation model DB 305.

[0043] The estimation model DB 305 is a storage device that stores a first physical condition estimation model for each patient. Specifically, the estimation model DB 305 stores a patient ID 3051 and a first physical condition estimation model 3052a in association with each other.

[0044] The information acquisition unit 306 is also called information acquisition means. The information acquisition unit 306 receives an output request for health-related information including a captured image of a target part of the patient P from the patient terminal 100. In response to this, the information acquisition unit 306 acquires the captured image of the target part of the patient P. The information acquisition unit 306 then supplies the acquired captured image and a patient ID associated with the patient terminal 100 that issued the output request to the health information generation unit 307a.

[0045] The physical condition information generating section 307a is an example of the physical condition information generating section 307 described above. The physical condition information generating unit 307a refers to the estimation model DB 305 and reads out the first physical condition estimation model associated with the patient ID acquired from the information acquiring unit 306. Then, the physical condition information generating unit 307a uses the first physical condition estimation model to generate physical condition related information from the captured image acquired from the information acquiring unit 306. Specifically, the physical condition information generating unit 307a inputs the captured image to the first physical condition estimation model and obtains estimated condition information as an output result. Then, the physical condition information generating unit 307a sets the output result as physical condition related information or generates physical condition related information based on the output result.

[0046] As an example, when the condition information indicates a condition level related to a physical condition, the estimated condition information which is the output result of the first physical condition estimation model may indicate a condition level related to the current physical condition of the patient P. When the condition information further includes the condition of the patient P at the time of imaging, the estimated condition information which is the output result of the first physical condition estimation model may indicate which past condition the current condition of the patient P is comparable to. When the health-related information indicates whether or not it is necessary to be examined by a doctor, the health information generating unit 307a may determine whether or not it is necessary to be examined by a doctor based on the current condition of the patient P. For example, when the current condition level of the patient P is equal to or lower than the condition level in a predetermined situation (e.g., at the time of hospitalization), the health information generating unit 307a may generate health-related information indicating that it is necessary to be examined by a doctor. On the other hand, when the current condition level of the patient P is better than the condition level in the predetermined situation, the health information generating unit 307a may generate health-related information indicating that it is not necessary to be examined by a doctor. Furthermore, in addition to or instead of the current condition level of the patient P, the physical condition information generating unit 307a may determine whether or not it is necessary for the patient P to be examined by a doctor based on the progress of the patient P. For example, when the condition level of the patient P is deteriorating at a rate equal to or faster than a predetermined standard, the physical condition information generating unit 307a may generate physical condition related information indicating that it is necessary for the patient P to be examined by a doctor. Also, for example, when the condition level of the patient P has not changed for a predetermined period of time and the current condition level of the patient P is equal to or lower than the condition level in a predetermined situation, the physical condition information generating unit 307a may generate physical condition related information indicating that it is necessary for the patient P to be examined by a doctor.

[0047] The output control unit 308a is an example of the above-mentioned output control unit 308. The output control unit 308a transmits the health-related information of the patient P to the patient terminal 100 of the patient P and causes it to be displayed.

[0048] 5 is a sequence diagram showing an example of the flow of a registration process according to embodiment 2. First, the hospital terminal 200 transmits a patient registration request including a patient ID and medical record information to the server 300 (S100).

[0049] The registration unit 301a of the server 300 determines the target body part of the patient P based on the medical record information included in the patient registration request (S101). Then, the registration unit 301a generates a record of the patient P in the learning DB 302 (S102). Specifically, the registration unit 301a generates a record corresponding to the patient ID of the patient P in the learning DB 302. The registration unit 301a notifies the hospital terminal 200 of the target body part of the patient P (S103).

[0050] Next, the hospital terminal 200 acquires a reference image of the target area of ​​the patient P from the camera 210, and acquires condition information based on the medical record information (S104). Then, the hospital terminal 200 transmits an image registration request to the server 300 (S105). The image registration request may include the patient ID, the reference image of the target area of ​​the patient P, and condition information at the time of capturing the reference image.

[0051] Next, the registration unit 301a of the server 300 associates the reference image and condition information with each other by labeling or the like in a record corresponding to the patient ID of patient P in the learning DB 302, and registers them as learning data (S106).

[0052] The information processing system 1a then repeats the processes shown in S104 to S106, and ends the repetition when a predetermined condition is satisfied (S107). For example, the information processing system 1a may end the repetition when the processes shown in S104 to S106 have been repeated a predetermined number of times. The information processing system 1a may also end the repetition when a required amount of learning data or more is stored in the learning DB 302. The information processing system 1a may also end the repetition when a required amount of learning data or more corresponding to a predetermined condition level or situation is stored in the learning DB 302. The number of repetitions or the required amount of learning data is the number of repetitions or the amount required to ensure the accuracy of the first physical condition estimation model. The number of repetitions or the required amount of learning data may be changed depending on the type of target part, condition information, or physical condition related information.

[0053] Next, when the repetition is completed, the model generation unit 304a of the server 300 generates a first physical condition estimation model of the patient P using the learning data in the learning DB 302 (S108). Then, the model generation unit 304a of the server 300 stores the first physical condition estimation model in the estimation model DB 305 in association with the patient ID of the patient P (S109).

[0054] The server 300 can store in the learning DB 302 a first physical condition estimation model personalized for each patient by repeating the above-mentioned flow for each patient.

[0055] Here, when the hospital terminal 200 transmits an image registration request in S105, a display screen shown in FIG. 6 may be displayed on the display unit of the hospital terminal 200. FIG. 6 is a diagram showing an example of the display of the hospital terminal 200 according to the second embodiment. In this example, the target part is the face. For example, from this screen, a doctor or staff of the hospital can upload a reference image of the face of the patient P for each of the status information of "at the time of admission", "mid-hospitalization", and "at the time of discharge". According to the reference image to be uploaded, as time passes from "at the time of admission", "mid-hospitalization", and "at the time of discharge", the facial expression of the patient P changes from a facial expression when the physical condition is extremely bad to a facial expression when the physical condition is recovered. On the display unit of the hospital terminal 200, an operation area for determining a combination of the reference image and the status information to be uploaded by the doctor or staff of the hospital is displayed. When the doctor or staff of the hospital taps this area, the hospital terminal 200 can transmit an image registration request to the server 300.

[0056] 7 is a sequence diagram showing an example of the flow of an output process of health-related information according to the second embodiment. First, the patient terminal 100 starts an application for viewing health-related information (S110). In response to starting the application, the patient terminal 100 transmits a start notification to the server 300 (S111). The start notification may include the patient ID of the patient P.

[0057] Upon receiving the start-up notification, the server 300 notifies the patient terminal 100 of the target body part (S112). FIG. 8 shows the display screen of the patient terminal 100 when notifying the target body part. FIG. 8 is a diagram showing an example of the display of the patient terminal 100 according to the second embodiment. For example, the display unit of the patient terminal 100 may display a message saying, "We will manage the physical condition of Patient A after he is discharged from the hospital. The target body part of Patient A to be photographed is the 'face'."

[0058] Returning to Fig. 7, the explanation will be continued. Upon receiving the notification of the target body part, the patient terminal 100 acquires a captured image of the current target body part of the patient P from the camera 110 (S113). Then, the patient terminal 100 transmits an output request for health-related information to the server 300 (S114). The output request may include the captured image of the target body part of the patient P and the patient ID.

[0059] As a result, the information acquisition unit 306 of the server 300 acquires the captured image of the target part of the patient P and the patient ID. Then, the health condition information generation unit 307a of the server 300 refers to the estimation model DB 305 and reads out the first health condition estimation model associated with the patient ID in the estimation model DB 305 (S115). Next, the health condition information generation unit 307a inputs the captured image to the first health condition estimation model (S116). Next, the health condition information generation unit 307a generates health condition related information based on the output result of the first health condition estimation model (S117). The output control unit 308a of the server 300 transmits the health condition related information to the patient terminal 100 that originated the output request (S118).

[0060] Then, the patient terminal 100 receives the physical condition related information and displays it on the display unit (S119).

[0061] 9 and 10 are diagrams showing an example of the display of the patient terminal 100 according to the second embodiment. In Fig. 9, for example, a message urging the patient to be examined again at a hospital is displayed as health-related information on the display unit of the patient terminal 100, together with a message indicating that the patient's health is deteriorating. By viewing this screen, the patient P or related parties can easily monitor the patient P's health and take early action when the patient's health deteriorates or when the medicine is not effective.

[0062] In Fig. 10, for example, a message is displayed on the display unit of the patient terminal 100 indicating that the progress since discharge is going well. By viewing this screen, the patient P or related parties can understand that there is currently no need for a post-discharge medical examination. This can therefore avoid worrying about whether or not to see a doctor, or going to the hospital unnecessarily. In addition, a message encouraging the patient P may be displayed on the display unit of the patient terminal 100 to reduce the psychological burden on the patient P.

[0063] Thus, according to the second embodiment, a camera, which is a simple device, can be used to perform a personalized physical condition estimation for each patient. This allows the patient P and related parties to easily confirm the appropriate timing for the patient P to visit the hospital after discharge, thereby eliminating the excessive anxiety of the patient P after discharge. For the hospital, this allows the hospital to avoid unnecessary visits to the hospital and allows the patient P to visit the hospital at the appropriate timing when the patient's physical condition deteriorates, thereby safely shortening the length of the patient P's hospital stay. This allows for efficient hospital management.

[0064] <Embodiment 3> Next, a third embodiment of the present disclosure will be described. The third embodiment is a specific example in which the health-related information is the above-mentioned (Case 2). That is, in the third embodiment, the health estimation model is a second health estimation model that receives predetermined condition information as an input and outputs a simulation image of a target part of the patient.

[0065] 11 is a block diagram showing the configuration of a server 300b according to embodiment 3. The server 300b has a model generation unit 304b, an estimation model DB 305b, an information acquisition unit 306b, and a physical condition information generation unit 307b, instead of the model generation unit 304a, the estimation model DB 305, the information acquisition unit 306, and the physical condition information generation unit 307a.

[0066] The model generating unit 304b is an example of the model generating unit 304 described above. The model generating unit 304b learns a second physical condition estimation model for each patient using the learning data included in the learning DB 302. Learning the second physical condition estimation model may mean optimizing the parameters of the second physical condition estimation model. For example, the model generating unit 304b may learn all the parameters of the predetermined second physical condition estimation model for the patient P using the learning data of the patient P. The model generating unit 304b may also learn some of the parameters of the predetermined second physical condition estimation model for the patient P using the learning data of the patient P. This allows the model generating unit 304b to generate a second physical condition estimation model for each patient. The model generating unit 304b then stores the second physical condition estimation model for each patient in the estimation model DB 305b.

[0067] The estimation model DB 305b is a storage device that stores a second physical condition estimation model for each patient. Specifically, the estimation model DB 305b stores a patient ID 3051 and a second physical condition estimation model 3052b in association with each other.

[0068] The information acquisition unit 306b is also called information acquisition means. The information acquisition unit 306b receives an output request for health-related information including a patient ID from the patient terminal 100. In response to this, the information acquisition unit 306b acquires the patient ID of the patient P. The information acquisition unit 306b then supplies the patient ID to the health information generation unit 307b.

[0069] The physical condition information generating unit 307b is an example of the above-mentioned physical condition information generating unit 307. The physical condition information generating unit 307b refers to the estimation model DB 305b and reads out the second physical condition estimation model associated with the patient ID acquired from the information acquiring unit 306b. The physical condition information generating unit 307b then inputs predetermined condition information into the second physical condition estimation model, and obtains as an output result a simulation image of the target part of the patient P corresponding to the condition information. The physical condition information generating unit 307b then sets the output result as physical condition related information. The physical condition information generating unit 307b then supplies the physical condition related information to the output control unit 308.

[0070] 12 is a sequence diagram showing an example of the flow of the output process of health-related information according to the third embodiment. First, steps S120 to S122 similar to steps S110 to S112 are executed. Next, in step S123, the patient terminal 100 that has received the notification of the target part transmits an output request for health-related information to the server 300b. The output request may include the patient ID.

[0071] As a result, the information acquisition unit 306b of the server 300b acquires the patient ID. The health condition information generation unit 307b of the server 300b then refers to the estimation model DB 305b and reads out the second health condition estimation model associated with the patient ID in the estimation model DB 305b (S124). Next, the health condition information generation unit 307b inputs predetermined condition information into the second health condition estimation model to generate a simulation image of the target part of the patient P for each condition (S125). Next, the output control unit 308b transmits the simulation image of the target part of the patient P for each condition as health condition related information to the patient terminal 100 that issued the output request (S126).

[0072] Then, the patient terminal 100 receives the physical condition related information and displays it on the display unit (S127).

[0073] FIG. 13 is a diagram showing an example of the display of the patient terminal 100 according to the third embodiment. In FIG. 13, the predetermined condition information, which is the input of the second physical condition estimation model, is a bad condition (condition S-1), a normal condition (condition S-2), and a good condition (condition S-3). For example, on the display unit of the patient terminal 100, a simulation image I-1 corresponding to the condition S-1, a simulation image I-2 corresponding to the condition S-2, and a simulation image I-3 corresponding to the condition S3 are displayed in association with the respective condition information. The patient P or a related person can easily grasp the physical condition of the patient P by comparing the simulation image with the current target part of the patient P. Therefore, the patient P or a related person can easily monitor the physical condition of the patient P and can take early action when the physical condition deteriorates or the medicine is not effective.

[0074] Therefore, according to the third embodiment, the same effects as those of the second embodiment can be achieved.

[0075] Although the above-described embodiment has been described as a hardware configuration, the present disclosure is not limited to this. Any process of the present disclosure can be realized by causing a processor to execute a computer program.

[0076] 14 is a diagram showing an example of the configuration of a computer used as the patient terminal 100, the hospital terminal 200, or the server 300. The computer 1000 has a processor 1010, a storage unit 1020, a read only memory (ROM) 1030, a random access memory (RAM) 1040, a communication interface (IF: Interface) 1050, and a user interface 1060.

[0077] The communication interface 1050 is an interface for connecting the computer 1000 to a communication network via wired communication means or wireless communication means. The user interface 1060 includes a display unit such as a display. The user interface 1060 also includes an input unit such as a keyboard, a mouse, and a touch panel. Note that the user interface 1060 is not essential, particularly for the server 300.

[0078] The storage unit 1020 is an auxiliary storage device that can hold various data. The storage unit 1020 does not necessarily need to be a part of the computer 1000, and may be an external storage device or a cloud storage connected to the computer 1000 via a network.

[0079] The ROM 1030 is a non-volatile storage device. For example, a semiconductor storage device having a relatively small capacity, such as a flash memory, is used for the ROM 1030. The programs executed by the processor 1010 can be stored in the storage unit 1020 or the ROM 1030. The storage unit 1020 or the ROM 1030 stores various programs for implementing the functions of each unit in the server, for example.

[0080] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0081] The RAM 1040 is a volatile storage device. Various semiconductor memory devices such as a dynamic random access memory (DRAM) or a static random access memory (SRAM) are used for the RAM 1040. The RAM 1040 can be used as an internal buffer for temporarily storing data and the like. The processor 1010 loads a program stored in the storage unit 1020 or the ROM 1030 into the RAM 1040 and executes it. The processor 1010 may be a central processing unit (CPU) or a graphics processing unit (GPU). The processor 1010 executes a program, thereby realizing the functions of each unit in the server, for example. The processor 1010 may have an internal buffer for temporarily storing data and the like.

[0082] The above-mentioned computer is composed of a computer system including a personal computer, a word processor, etc. However, it is not limited to this, and a computer can also be composed of a server in a LAN (local area network), a host for computer (personal computer) communication, a computer system connected to the Internet, etc. It is also possible to distribute functions to each device on a network and configure a computer as a whole network.

[0083] The present disclosure is not limited to the above-mentioned embodiment, and can be modified as appropriate without departing from the spirit of the present disclosure. For example, the second embodiment and the third embodiment may be combined. That is, the model generating units 304a and 304b may generate a first health condition estimation model and a second health condition estimation model for each patient. The health condition information generating units 307a and 307b may generate health condition related information based on the estimated state information output from the first health condition estimation model and health condition related information output from the second health condition estimation model. The output control units 308a and 308b may output two types of health condition related information to the patient terminal 100. Furthermore, when the model generating units 304a and 304b generate a first and a second physical condition estimation model for each patient, the parameters of one physical condition estimation model may be used in the optimization process of the parameters of the other physical condition estimation model, thereby speeding up the optimization process.

[0084] In the above description, the output request process and the display process by the patient terminal 100 are performed on an application, but it is not essential that the above processes function on an application.

[0085] In the above description, the estimation model DB 305, 305b stores the first or second health condition estimation model for each patient ID, and the health condition information generating unit 307a, 307b reads out the first or second health condition estimation model corresponding to the target patient. However, instead of this, the estimation model DB 305, 305b may store parameters of the first or second health condition estimation model for each patient ID, and the health condition information generating unit 307a, 307b may read out parameters of the first or second health condition estimation model corresponding to the target patient. In this case, the health condition information generating unit 307a, 307b generates health condition-related information using the first or second health condition estimation model to which the read parameters have been applied. [Explanation of symbols]

[0086] 1,1a Information Processing System 10 Patient System 20 Hospital System 100 Patient terminals 110 Camera 200 Hospital terminals 210 Camera 300, 300b Information processing device (server) 301,301a Registration Department 302 Learning DB 3020 Patient ID 3021 Reference Image 3022 Status information 304, 304a, 304b Model generation section 305,305b Estimation model DB 3051 Patient ID 3052a First physical condition estimation model 3052b 2nd Physical Condition Estimation Model 306,306b Information acquisition department 307, 307a, 307b Physical condition information generation unit 308, 308a, 308b Output control section 1000 Computers 1010 Processor 1020 Storage section 1030 ROM 1040 RAM 1050 Communication Interface 1060 User Interface P patient

Claims

1. a registration means for acquiring, for each patient, a reference image of the patient and status information relating to at least one of the physical condition and progress of the patient at the time of image acquisition; a model generating means for generating, for each patient, a physical condition estimation model for estimating a physical condition of the patient based on the reference image and the condition information; a physical condition information generating means for generating information related to the physical condition of the target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; an output control means for outputting information related to the physical condition of the subject patient; Equipped with The status information includes information regarding a condition suggesting a progression of a disease. Information processing system.

2. The reference image includes an image area of ​​a target area used for case assessment of a patient. The information processing system according to claim 1 .

3. The registration means determines the target site for each patient based on the medical record information of the patient. The information processing system according to claim 2 .

4. the physical condition estimation model includes a first physical condition estimation model that receives a photographed image of the patient as an input and outputs estimated condition information of the patient; The physical condition information generating means generates information related to the physical condition of the target patient based on estimated state information when a photographed image of the target patient is input to the first physical condition estimation model of the target patient. The information processing system according to claim 1 .

5. the physical condition estimation model includes a second physical condition estimation model that receives predetermined condition information as an input and outputs a simulation image of the patient, The physical condition information generating means generates, using the second physical condition estimation model of the target patient, a simulation image of the target patient corresponding to the predetermined condition information as information related to the physical condition of the target patient. The information processing system according to claim 1 .

6. For each patient, a reference image of the patient and status information regarding the physical condition or progress of the patient at the time of the image acquisition are obtained; generating a physical condition estimation model for estimating a physical condition of each patient based on the reference image and the condition information; generating information related to the physical condition of the target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; Outputting information related to the physical condition of the target patient; The status information includes information regarding a condition suggesting a progression of a disease. Information processing methods.

7. A procedure for acquiring, for each patient, a reference image of the patient and status information relating to the physical condition or progress of the patient at the time of the image acquisition; generating, for each patient, a physical condition estimation model for estimating a physical condition of the patient based on the reference image and the condition information; A step of generating information related to the physical condition of a target patient by inputting a photographed image or predetermined condition information of the target patient into a physical condition estimation model of the target patient; outputting information related to the physical condition of the target patient; on the computer, The status information includes information regarding a condition suggesting a progression of a disease. program.

Images