Program, method, information processing system and defecation seat

Abstract

To provide a system allowing a medical worker to continuously assess disease activity with an extremely simple operation by a patient.SOLUTION: A program causing an information processing system including a server with a processor to execute processing, causes the processor to execute the steps of: acquiring a stool image of a patient captured with a terminal device used by the patient or a medical worker; estimating a disease activity score relating to a specific disease of the patient by image processing of the acquired stool image; and outputting the estimated disease activity score of the patient to the terminal device.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present disclosure relates to a program, a method, an information processing system, and a defecation sheet. [Background technology]

[0002] Conventionally, the current status of a particular disease is calculated using information on the health condition input by the user. Systems for assessing disease activity are known. For example, the following non-patent literature describes how multiple items related to health conditions are input by patients. It accepts daily input operations and uses the input information to generate scores according to its own evaluation criteria. A system is disclosed that performs screening to estimate disease activity of ulcerative colitis in a patient. It is being done. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] My IBD Care:Crohn's & Colitis(https: / / play.google.com / store / apps / details?id=nhs.ibd.com.nhsibd&hl=ja&gl=US&pli=1) Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional systems, patients are required to input multiple items of their daily health condition every day. This can make patients feel burdened by the daily input operations, leading to failure to continue to input information appropriately, and thus making it difficult to appropriately evaluate disease activity.

[0005] An object of the present disclosure is to provide a system that enables continuous evaluation of disease activity through extremely simple operations by patients. [Means for solving the problem]

[0006] One aspect of the present disclosure is a program to be executed by an information processing system including a server having a processor, the program causing the processor to execute the steps of acquiring a stool image of the patient taken by a terminal device used by the patient or a medical professional, estimating a disease activity index for a specific disease of the patient by image processing of the acquired stool image, and outputting the estimated disease activity index of the patient to the terminal device. Effect of the Invention

[0007] According to the present disclosure, disease activity can be continuously evaluated with extremely simple operations by the patient. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an outline of a disease activity evaluation system of the present invention. [Diagram 2] 2 is a diagram illustrating a hardware configuration of the terminal device illustrated in FIG. [Diagram 3] FIG. 3 is a diagram illustrating a functional configuration of the terminal device illustrated in FIG. [Figure 4] FIG. 2 is a diagram illustrating a hardware configuration of the evaluation server illustrated in FIG. [Diagram 5] FIG. 5 is a diagram showing a functional configuration of the evaluation server shown in FIG. [Figure 6] 4 is a diagram showing an example of the structure of each database stored in the evaluation server. FIG. [Figure 7] 2 is a diagram illustrating a hardware configuration of the medical information management server shown in FIG. 1. [Figure 8] 7 is a diagram illustrating a functional configuration of the medical information management server shown in FIG. 6. [Figure 9]4 is a diagram showing an example of a structure of each database stored in the medical information management server. FIG. [Figure 10] FIG. 1 is a diagram showing an overview of the present embodiment. [Figure 11] FIG. 1 is a diagram illustrating the flow of disease activity evaluation processing according to the present invention. [Figure 12] FIG. 2 is a diagram illustrating the flow of reservation processing according to the present invention. [Figure 13] FIG. 13 is a diagram illustrating a flow of a process for aggregating evaluation histories according to the present invention. [Figure 14] 11 is a diagram for explaining the flow of a process for generating an endoscope-equivalent image. FIG. [Figure 15] FIG. 2 is a diagram showing an example of a first screen of the system 1. [Figure 16] FIG. 2 is a diagram showing an example of a second screen of the system 1. [Figure 17] FIG. 13 is a diagram showing an example of a third screen of the system 1. [Figure 18] FIG. 13 is a diagram illustrating the flow of disease activity evaluation processing in the system 1 according to the first modified example. [Figure 19] FIG. 13 is a diagram illustrating the flow of disease activity evaluation processing in the system 1 according to the second modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0010] <1. System 1 Configuration> The configuration of the system 1 will now be described.

[0011] (1-1. Overall composition) The information processing system 1 (hereinafter, simply referred to as system 1) according to this embodiment is a system that evaluates disease activity of a specific disease in a user (patient or medical professional) using a stool image captured by the user. In the following description, a stool image refers to an image in which excrement containing stool is the subject. In the following description, inflammatory bowel disease is taken as an example of a specific disease.

[0012] Here, disease activity is an index that quantitatively indicates the state of disease. Generally, disease activity is scored according to the state for multiple evaluation items, and the total score is compared with a preset standard to assign a grade of disease activity. For example, the following grades can be set as disease activity in inflammatory bowel disease. Level 1: The disease is very good. Level 2: Mild lesions, but the disease is stable Level 3: Moderate lesions with mild to moderate symptoms Level 4: Severe lesions and moderate to severe symptoms Level 5: Very severe lesions and very severe symptoms In the following description, the above-mentioned grades are referred to as indexes of disease activity. The evaluation criteria for disease activity can be set arbitrarily.

[0013] FIG. 1 is a diagram showing the overall configuration of a system 1. As shown in FIG. 1, the system 1 includes a plurality of terminal devices 10 (FIG. 1 shows a patient terminal 10A and a medical terminal 10B. Hereinafter, they may be collectively referred to as “terminal devices 10”), an evaluation server 20, and a medical institution server 30.

[0014] The terminal device 10 is an information processing device operated by each user who uses the system 1. The terminal device 10 is realized by a desktop PC (Personal Computer), a laptop PC, or a mobile terminal such as a smartphone or tablet compatible with the system 1. The terminal device 10 is connected to a network 80.

[0015] Here, the users of the system 1 include the following: Patients with a history of a specific disease Medical professionals, including doctors, who provide medical care to patients -System administrator who maintains and operates System 1

[0016] In the following description, the terminal device 10 will be classified as follows according to the subject of operation. Patient terminal 10A: terminal device used by the patient Medical terminal 10B: Terminal device used by medical staff Administrator terminal: A terminal device used by a system administrator (not shown) Configurations common to these terminals will be collectively described as terminal device 10.

[0017] The evaluation server 20 is an information processing device that executes a process of evaluating the disease activity of the patient mainly by image processing using stool images transmitted from the patient terminal 10A. The assessment server 20 may include various computers such as a personal computer, a server computer (for example, a Web server, an application server, a database server, or a combination thereof), etc. In this embodiment, the assessment server 20 will be described taking a server computer as an example.

[0018] The medical institution server 30 is an information processing device that executes a process of managing information regarding daily medical treatment given to patients by medical staff, mainly in a medical institution such as a hospital. The medical institution server 30 may include various computers such as a personal computer, a server computer (for example, a Web server, an application server, a database server, or a combination thereof), etc. In this embodiment, the medical institution server 30 will be described taking a server computer as an example.

[0019] (1-2. Hardware Configuration of Terminal Device 10) FIG. 2 is a diagram showing a hardware configuration of the terminal device 10. As shown in FIG. As shown in FIG. 2, the terminal device 10 includes a communication IF (Interface) 12, an input device 13, an output device 14, a memory 15, a storage unit 16, and a processor 19.

[0020] The terminal device 10 is communicably connected to the evaluation server 20 and the medical institution server 30 via a network 80. The terminal device 10 is connected to the network 80 by communicating with communication devices such as a wireless base station 81 compatible with communication standards such as 5G and LTE (Long Term Evolution) and a wireless LAN router 82 compatible with wireless LAN (Local Area Network) standards such as IEEE (Institute of Electrical and Electronics Engineers) 802.11.

[0021] The communication IF 12 is an interface for inputting and outputting signals so that the terminal device 10 can communicate with an external device. The input device 13 is an input device (for example, a touch panel, a touch pad, a pointing device such as a mouse, a keyboard, etc.) for receiving an input operation from a user.

[0022] The output device 14 is an output device (such as a display and a speaker) for presenting information to a user. The memory 15 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0023] The storage unit 16 is a storage device for saving programs and data, and is, for example, a flash memory or a hard disk drive (HDD). The programs include, for example, the following programs: ·OS (Operating System) programs · Programs for applications that process information (e.g. web browsers) The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0024] The processor 19 is hardware for executing an instruction set written in a program stored in the storage unit 16, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

[0025] (1-3. Functional Configuration of Terminal Device 10) FIG. 3 is a diagram showing the functional configuration of the terminal device 10. As shown in FIG. As shown in FIG. 3, the terminal device 10 includes multiple antennas (antenna 111, antenna 112), wireless communication units corresponding to each antenna (first wireless communication unit 121, second wireless communication unit 122), an operation reception unit 130 (including a touch-sensitive device 131 and a display 132), a position information sensor 150, a camera 160, a memory unit 170, and a control unit 180.

[0026] The terminal device 10 also has functions and configurations (such as a battery for storing power, a power supply circuit for controlling the supply of power from the battery to each circuit, etc.) that are not specifically shown in Fig. 3. As shown in Fig. 3, each block included in the terminal device 10 is electrically connected by a bus or the like.

[0027] The antenna 111 emits a signal generated by the terminal device 10 as a radio wave. The antenna 111 also receives a radio wave from space and provides the received signal to the first wireless communication unit 121. The antenna 112 radiates a signal generated by the terminal device 10 as a radio wave. The antenna 112 also receives a radio wave from space and provides the received signal to the second radio communication unit 122.

[0028] The first wireless communication unit 121 performs modulation and demodulation processing for transmitting and receiving signals via the antenna 111 so that the terminal device 10 can communicate with other wireless devices. The second wireless communication unit 122 performs modulation and demodulation processing for transmitting and receiving signals via the antenna 112 so that the terminal device 10 can communicate with other wireless devices.

[0029] The first wireless communication unit 121 and the second wireless communication unit 122 are communication modules including a tuner, a received signal strength indicator (RSSI) calculation circuit, a cyclic redundancy check (CRC) calculation circuit, a high-frequency circuit, etc. The first wireless communication unit 121 and the second wireless communication unit 122 perform modulation / demodulation and frequency conversion of wireless signals transmitted and received by the terminal device 10, and provide the received signals to the control unit 180.

[0030] The operation reception unit 130 has a mechanism for receiving an input operation from a user. Specifically, the operation reception unit 130 is configured as a touch screen, and includes a touch-sensitive device 131 and a display 132.

[0031] Touch-sensitive device 131 accepts an input operation by a user of terminal device 10. Touch-sensitive device 131 detects a touch position of the user on the touch panel by using, for example, a capacitive touch panel. Touch-sensitive device 131 outputs a signal indicating the touch position of the user detected by the touch panel to control unit 180 as an input operation.

[0032] Display 132 displays data such as images, videos, and text under the control of control unit 180. Display 132 is realized by, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display.

[0033] The position information sensor 150 is a sensor that detects the position of the terminal device 10, and is, for example, a GPS (Global Positioning System) module. The GPS module is a receiving device used in a satellite positioning system. In the satellite positioning system, signals are received from at least three or four satellites, and the current position of the terminal device 10 equipped with the GPS module is detected based on the received signals. The camera 160 is a device that receives light using a light receiving element and outputs the received light as a captured image.

[0034] The storage unit 170 is configured with, for example, a flash memory, and stores data and programs used by the terminal device 10. The storage unit 170 stores at least user information 171, a detection model 172, and various programs (not shown).

[0035] The user information 171 is information about a user who uses the terminal device 10. The user information 171 includes account information (a user ID and a password for identifying the user) that is required to be input when the user logs in to the system 1 via the terminal device 10. In addition, the user information 171 may include various information about the attributes of the user that is registered when the user installs a predetermined application in the terminal device 10 to use the system 1.

[0036] The user ID may be a customer ID issued to the user. The user ID may be a user account for a service (which may include a corporate service) that provides various functions such as email, video calling, calendar, storage, document creation, spreadsheet creation, and news distribution in the form of SaaS (Software as a Service) or the like.

[0037] The detection model 172 is obtained by making a machine learning model perform machine learning according to a model learning program based on learning data. For example, in this embodiment, the detection model 172 is trained to output the position and size of excrement from a human body and the type of excrement for input image information. The types of excrement include solid stool, sludgy stool, watery stool, blood, mucus, contaminants, etc.

[0038] In this case, the learning data is, for example, image information of excrement from the human body as input data, and information regarding the position and size of the excrement and information regarding the type of excrement for the input image information are used as correct output data.

[0039] Specifically, the detection model 172 compares the feature amounts (reference feature amounts) of images of various types of excrement with the feature amounts of the excrement captured in the image to be evaluated to evaluate the degree of similarity, thereby determining whether any type of excrement is present. When evaluating the degree of similarity, if the feature amount of the image to be evaluated is within a preset threshold value range with respect to the reference feature amount, the presence of the excrement is detected. Note that reference feature amounts are respectively set for various types of excrement.

[0040] The detection model 172 according to the present embodiment is, for example, a parameterized composite function in which a plurality of functions are combined. The parameterized composite function is defined by a combination of a plurality of adjustable functions and parameters. The detection model 172 according to the present embodiment may be any parameterized composite function that satisfies the above requirements, but is assumed to be a multi-layered neural network model (hereinafter referred to as a "multi-layered network"). The detection model 172 using the multi-layered network has an input layer, an output layer, and at least one intermediate layer or hidden layer provided between the input layer and the output layer. The detection model 172 is assumed to be used as a program module that is a part of artificial intelligence software.

[0041] As the multi-layered network according to the present embodiment, for example, a deep neural network (DNN) that is a multi-layered neural network that is the subject of deep learning may be used. As the DNN, for example, a convolution neural network (CNN) that targets images may be used.

[0042] The control unit 180 reads a program stored in the storage unit 170 and executes instructions included in the program to control the operation of the terminal device 10. The control unit 180 is, for example, an application that is pre-installed in the terminal device 10. The control unit 180 operates according to the program to fulfill the functions of an input reception unit 181, a transmission / reception unit 182, an image capture unit 183, a detection unit 184, a data processing unit 185, and a display control unit 186.

[0043] The input reception unit 181 performs processing for receiving an input operation by a user to the input device 13 such as a touch-sensitive device 131 .

[0044] The transmission / reception unit 182 performs processing for the terminal device 10 to transmit and receive data to and from external devices such as the assessment server 20 in accordance with a communication protocol.

[0045] The photographing section 183 performs processing to start the camera 160 and photograph an image of the subject in response to an operation from the patient.

[0046] The detection unit 184 detects feces included in the image captured by the imaging unit 183. At this time, the detection unit 184 inputs image data to be evaluated to the detection model 172, and detects excrement such as feces included in the image based on information output from the detection model 172. This process will be described later.

[0047] The data processing unit 185 performs a process of performing calculations on the data inputted by the terminal device 10 according to a program and outputting the calculation results to a memory or the like. For example, the data processing unit 185 controls the storage process of the captured image in the storage unit 170. When feces is detected in the image, the data processing unit 185 transmits the image to the evaluation server 20 without storing it in the storage unit 170 of the terminal device 10.

[0048] The display control unit 186 performs a process of presenting information to a user by displaying the information on the display 132. The display control unit 186 has a function as a web browser, and accesses information output by the evaluation server 20 to a logical line (TCP connection) between the terminal device 10 and performs a process (rendering) of displaying the information on the display 132 of the terminal device 10.

[0049] (1-4. Hardware configuration of the evaluation server 20) FIG. 4 is a diagram showing a hardware configuration of the evaluation server 20. As shown in FIG. As shown in FIG. 4, the evaluation server 20 includes a communication IF 22, an input / output IF 23, a memory 25, a storage 26, and a processor 29.

[0050] The communication IF 22 is an interface for inputting and outputting signals so that the assessment server 20 can communicate with external devices. The input / output IF 23 functions as an interface with an input device for receiving an input operation from a user and an output device for presenting information to the user.

[0051] The memory 25 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0052] The storage 26 is a storage device for saving programs and data, such as a flash memory or a hard disk drive (HDD). The programs include, for example, the following programs: ·OS (Operating System) programs · Programs for applications that process information (e.g. web browsers) The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0053] The processor 29 is hardware for executing an instruction set written in a program stored in the storage 26, and is composed of an arithmetic unit, a register, peripheral circuits, and the like.

[0054] (1-5. Functional configuration of the evaluation server 20) FIG. 5 is a diagram showing the functional configuration of the evaluation server 20. As shown in FIG. As shown in FIG. 5, the evaluation server 20 functions as a communication unit 201, a storage unit 202, and a control unit 203.

[0055] The communication section 201 performs processing for the assessment server 20 to communicate with external devices.

[0056] The storage unit 202 stores data and programs used by the evaluation server 20. The storage unit 202 stores, for example, the following data: User Database (User DB) 2021 ·Image Database (Image DB) 2022 ·Object Database (Object DB) 2023 ·Tag Database (Tag DB) 2024 Comment Database (Comment DB) 2025 ·Evaluation History Database (Evaluation History DB) 2026 Activity Estimation Model 2027 Image synthesis model 2028

[0057] The user DB 2021 is a database for managing information on users who, as patients, use the system 1. The data structure of the user DB 2021 will be described in detail later.

[0058] The image DB 2022 is a database for managing information relating to images acquired by the terminal device 10. The data structure of the image DB 2022 will be described in detail later.

[0059] The object DB 2023 is a database for managing information on objects constituting excrement detected from an image. Objects constituting excrement include, for example, feces, blood, mucus, contaminants, etc. The data structure of the object DB 2023 will be described in detail later.

[0060] The tag DB 2024 is a database that manages information related to tags selected by the patient when photographing the stool. In the system 1, a number of items related to health conditions and excretory behavior history are set as tags, and the user can select a tag for the relevant item when photographing the stool. The data structure of the tag DB 2024 will be described in detail later.

[0061] The comment DB 2025 is a database for managing information related to comments arbitrarily input by the user when photographing a flight. The data structure of the comment DB 2025 will be described in detail later.

[0062] The evaluation history DB 2026 is a database that manages the history of the evaluation results of the disease activity by the system 1. The data structure of the evaluation history DB 2026 will be described in detail later.

[0063] The activity estimation model 2027 is obtained by making the machine learning model perform machine learning according to a model learning program based on learning data. For example, in this embodiment, the activity estimation model 2027 is trained to output, for an input stool image, an estimate of a disease activity index for inflammatory bowel disease in a person (patient) who excretes the stool. In addition, the activity estimation model 2027 compares the input stool image with the trained dataset, and outputs a likelihood indicating the likelihood of the estimate of the disease activity index.

[0064] In this case, the learning data is, for example, image information of excrement from the human body as input data, and the correct output data for the input image information is the disease activity index value for inflammatory bowel disease determined by a doctor at the same time for the person (patient) who excreted the stool.

[0065] The activity estimation model 2027 may further output an estimated position of the lesion range of the disease for the input stool image. In this case, the image information of the excrement is used as input data, and information on the lesion range of inflammatory bowel disease of the person (patient) who excreted the stool is used as correct output data.

[0066] The activity estimation model 2027 according to this embodiment is, for example, a parameterized composite function in which a plurality of functions are combined. The parameterized composite function is defined by a combination of a plurality of adjustable functions and parameters. The activity estimation model 2027 according to this embodiment may be any parameterized composite function that satisfies the above requirements, but is assumed to be a multi-layered neural network model (hereinafter referred to as a "multi-layered network"). The activity estimation model 2027 using the multi-layered network has an input layer, an output layer, and at least one intermediate layer or hidden layer provided between the input layer and the output layer. The activity estimation model 2027 is expected to be used as a program module that is a part of artificial intelligence software.

[0067] The image synthesis model 2028 is obtained by making a machine learning model perform machine learning according to a model learning program based on learning data. For example, in this embodiment, the image synthesis model 2028 is trained to output an endoscope-equivalent image that is estimated as the intestinal state of a person (patient) who excreted the stool in response to an input stool image.

[0068] In this case, the learning data is, for example, image information of excrement from the human body as input data, and the correct output data for the input image information is an endoscopic image of the inside of the intestines taken by endoscopic examination at the same time as the person (patient) who excreted the stool.

[0069] The image synthesis model 2028 according to this embodiment is, for example, a parameterized synthesis function in which a plurality of functions are synthesized. The parameterized synthesis function is defined by a combination of a plurality of adjustable functions and parameters. The image synthesis model 2028 according to this embodiment may be any parameterized synthesis function that satisfies the above requirements, but is assumed to be a multi-layered neural network model (hereinafter referred to as a "multi-layered network"). The image synthesis model 2028 using the multi-layered network has an input layer, an output layer, and at least one intermediate layer or hidden layer provided between the input layer and the output layer. The image synthesis model 2028 is assumed to be used as a program module that is a part of artificial intelligence software.

[0070] The control unit 203 performs functions shown as various modules by the processor 29 of the evaluation server 20 performing processing according to a program. The control unit 203 performs functions as a transmission / reception control module 2031, an acquisition module 2032, an evaluation module 2033, a judgment module 2034, a counting module 2035, an image generation module 2036, and an output module 2037.

[0071] The transmission / reception control module 2031 controls the process in which the assessment server 20 transmits and receives signals to and from external devices in accordance with a communication protocol.

[0072] The acquisition module 2032 acquires information about the stool image transmitted from the patient terminal 10A in response to an operation by the patient. The acquisition module 2032 stores the acquired information about the stool image in a predetermined storage area, and then records a new record for the database to be updated among the image DB 2022, the object DB 2023, and the comment DB 2025.

[0073] The evaluation module 2033 evaluates the disease activity of the patient by image processing using the acquired stool image. Specifically, the evaluation module 2033 inputs the stool image to the activity estimation model 2027 to obtain an estimated value of the disease activity index output from the activity estimation model 2027. The evaluation module 2033 records the obtained estimated value of the disease activity index as a new record in the evaluation history DB2026. The evaluation module 2033 may input data of excrement-related objects detected from the stool image to the activity estimation model 2027.

[0074] The determination module 2034 determines whether the patient is in a state in which he or she needs to be examined by a doctor, using preset determination conditions regarding the necessity of an appointment, based on the evaluation result of the disease activity performed by the evaluation module 2033. When the determination module 2034 determines that the patient needs to be examined by a doctor, it notifies the patient and the doctor in charge.

[0075] The tallying module 2035 tally up the evaluation history according to a preset frequency. The tallying module 2035 refers to the evaluation history DB 2026 and tally up, for example, the following information as the evaluation history of disease activity for the patient to be tallied: -Percentage of days during the period covered by the survey where stool images were taken - Trends in evaluation results during the period covered Comments from patients or doctors during the collection period

[0076] The image generation module 2036 performs image processing using the acquired stool image to generate an endoscopic image of the patient when the patient excretes the stool related to the stool image. The endoscopic image is information that expresses the state of the patient's intestines as an image equivalent to an endoscopic image that is assumed to be captured when actually photographed with an endoscope. The image generation module 2036 inputs a stool image to the image synthesis model 2028, thereby acquiring an endoscope-equivalent image output from the image synthesis model 2028. After the image generation module 2036 stores the endoscope-equivalent image in a predetermined storage area, it creates address information (e.g., URL information) indicating the storage area.

[0077] The output module 2037 outputs various information obtained by the executed processing in response to an operation by a patient or a medical professional. The output module 2037 outputs the following information, for example: Information on the evaluation result of disease activity for the patient terminal 10A and the medical terminal 10B Information on the compilation results of the evaluation history for the patient terminal 10A and the medical terminal 10B Information to suggest a medical appointment to the patient terminal 10A Information to the medical terminal 10B that the generated endoscope-equivalent image was created

[0078] (1-6. Data structure of each database managed by the evaluation server 20) Next, an example of the data structure of each database stored in the storage unit 202 of the assessment server 20 will be described with reference to FIG.

[0079] (1-6-1. User DB2021) FIG. 6A is a diagram showing an example of the data structure of the user DB 2021. As shown in FIG. 6A, the user DB 2021 stores information about a user who, as a patient, uses the system 1. A new record is recorded in the user DB 2021 by a user registration operation when the patient starts using the system 1.

[0080] The user DB 2021 includes an item "user ID", an item "name", an item "date of birth", an item "gender", and an item "patient ID".

[0081] The item "user ID" stores user identification information that can identify a user who uses the system 1 as a patient.

[0082] The item "name" stores information about the name of the user corresponding to the user ID.

[0083] The item "Date of Birth" stores information about the date of birth of the user corresponding to the user ID.

[0084] The item "gender" stores information about the gender of the user corresponding to the user ID.

[0085] The item "patient ID" stores identification information of the user corresponding to the user ID as a patient managed by the medical institution server 30. Note that instead of a patient ID managed only by a specific medical institution, the insured person number on a health insurance card or My Number may be used. It should be noted that the structure of the user DB 2021 shown in FIG. 6A is merely an example, and the user DB 2021 may include columns in which other data items are stored.

[0086] (1-6-2. Image DB2022) FIG. 6B is a diagram showing an example of the data structure of the image DB 2022. As shown in FIG. 6B, the image DB 2022 stores information about captured stool images. When a stool image is transmitted to the evaluation server 20 and acquired by the acquisition module 2032, a new record is recorded in the image DB 2022.

[0087] The image DB 2022 includes an item "image ID", an item "user ID", an item "photographing date and time", an item "photographing terminal", an item "photographing location", an item "tag ID", and an item "storage area".

[0088] The item "image ID" stores identification information of the stool image that can identify the stool image acquired by the evaluation server 20.

[0089] The item "user ID" stores the user ID of the patient who excreted the stool related to the stool image corresponding to the image ID.

[0090] The item "photographed date and time" stores information related to the date and time when the flight image corresponding to the image ID was photographed.

[0091] The item "photographing terminal" stores identification information of the terminal device 10 that can identify the terminal that photographed the flight image corresponding to the image ID.

[0092] The item "Photo location" stores information about the location where the flight image corresponding to the image ID was taken. Information about the photo location can be obtained by linking the location information acquired by the location information sensor 150 in the terminal device 10 that took the photo with the photographed image.

[0093] The item "tag ID" stores identification information of the tag that the user selected as the relevant item when photographing the flight image corresponding to the image ID.

[0094] The item "storage area" stores address information (for example, URL information) of a storage area in which a stool image corresponding to an image ID is stored. The system 1 may use an external storage area for storing image data. It should be noted that the structure of the image DB 2022 shown in FIG. 6B is merely an example, and the image DB 2022 may include columns in which other data items are stored.

[0095] (1-6-3. Object DB2023) FIG. 6C is a diagram showing an example of the data structure of the object DB 2023. As shown in FIG. 6C, the object DB 2023 stores information about objects constituting excrement contained in a stool image. A new record is recorded in the object DB 2023 when information about excrement detected by the detection unit 184 of the patient terminal 10A is transmitted to the evaluation server 20 together with the stool image and acquired by the acquisition module 2032.

[0096] The object DB 2023 includes an item "object ID", an item "image ID", an item "object type", and an item "position and range of object".

[0097] The item "object ID" stores identification information of an object that can specify an object constituting excrement detected to be included in a stool image.

[0098] The item "image ID" stores the image ID of the stool image in which the object corresponding to the object ID is detected.

[0099] The item "object type" stores the type of excrement to which the object corresponding to the object ID corresponds. Examples of excrement types include the following: ·Solid stool -Muddy stool Watery stools ·blood ·mucus ·Adulterants

[0100] The item "position and range of object" stores information capable of identifying the position and range of an object in an image in which the object corresponding to the object ID is detected. It should be noted that the structure of the object DB 2023 shown in FIG. 6C is merely an example, and the object DB 2023 may include columns in which other data items are stored.

[0101] (1-6-4. Tag DB2024) FIG. 6D is a diagram showing an example of the data structure of the tag DB 2024. As shown in FIG. 6D, the tag DB 2024 stores information about tags input by a patient when photographing a stool. A plurality of pieces of tag-related information are prepared in advance and are already stored when the system 1 starts to be used.

[0102] The tag DB 2024 includes an item "tag ID", an item "tag name", and an item "tag description".

[0103] The item "tag ID" stores tag identification information that can identify the tag.

[0104] The item "tag name" stores the name of the tag corresponding to the tag ID.

[0105] The item "tag description" stores a description of the tag corresponding to the tag ID. Specific examples of tag names and descriptions are given below. - Abdominal pain tag: Tag to select if you experience abdominal pain on the day of or the day before defecation Bleeding tag: Select this tag if bleeding occurs during excretion. Loss of appetite: Select this tag if you have a loss of appetite on the day of or before defecation. Fever: Select this tag if you have a fever on the day of or before excretion. Note that the structure of tag DB 2024 shown in FIG. 6D is merely an example, and tag DB 2024 may include columns in which other data items are stored.

[0106] (1-6-5. Comment DB2025) FIG. 6E is a diagram showing an example of the data structure of the comment DB 2025. As shown in FIG. 6E, the comment DB 2025 stores information about comments entered by the patient when photographing the stool. When the object DB 2023 is transmitted to the evaluation server 20 together with the stool image and acquired by the acquisition module 2032, a new record is recorded therein.

[0107] The comment DB 2025 includes an item "comment ID", an item "image ID", and an item "patient comment".

[0108] The item "comment ID" stores comment identification information that can identify a comment.

[0109] The item "Image" stores the image ID of the flight image to which the comment corresponding to the comment ID has been entered.

[0110] The item "Patient comment" stores the content of the comment corresponding to the comment ID. It should be noted that the structure of comment DB 2025 shown in FIG. 6E is merely an example, and comment DB 2025 may include columns in which other data items are stored.

[0111] (1-6―6. Evaluation History DB2026) FIG. 6F is a diagram showing an example of the data structure of the evaluation history DB 2026. As shown in FIG. 6F, the evaluation history DB2026 stores a history of the evaluation results of the disease activity by the system 1. A new code is recorded in the evaluation history DB2026 when the evaluation module 2033 evaluates the disease activity.

[0112] The evaluation history DB 2026 includes an item "evaluation history ID", an item "image ID", an item "evaluation date and time", an item "evaluation result", an item "likelihood", an item "comments from attending physician", and an item "necessity of examination".

[0113] The item "evaluation history ID" stores identification information of an evaluation history that can identify the evaluation result obtained by the processing of the system 1.

[0114] The item "image ID" stores the image ID of the stool image that was the subject of the evaluation corresponding to the evaluation history ID.

[0115] The item "evaluation date and time" stores information about the date and time when the evaluation corresponding to the evaluation history ID was performed. As the evaluation date and time, the date and time when the flight that was the subject of the evaluation was photographed may be managed.

[0116] The item "evaluation result" stores information about the evaluation result of the disease activity index in the evaluation history corresponding to the evaluation history ID.

[0117] The item "likelihood" stores the likelihood of the evaluation result of disease activity in the evaluation history corresponding to the evaluation history ID. The likelihood of the evaluation result is a value that quantitatively indicates the probability of the evaluation result and indicates the credibility of the evaluation result.

[0118] The item "Attending physician's comment" stores the evaluation history corresponding to the evaluation history ID and the comment input by the attending physician regarding the stool image that was the subject of the evaluation.

[0119] The item "examination required" stores the result of a judgment as to whether or not the patient who was the subject of the evaluation by system 1 needs to be examined by a doctor at a medical institution, based on the evaluation history corresponding to the evaluation history ID. It should be noted that the structure of the review history DB 2026 shown in FIG. 6F is merely an example, and the review history DB 2026 may include columns in which other data items are stored.

[0120] (1-7. Hardware configuration of medical institution server 30) FIG. 7 is a diagram showing a hardware configuration of the medical institution server 30. As shown in FIG. As shown in FIG. 7, the medical institution server 30 includes a communication IF 32, an input / output IF 33, a memory 35, a storage 36, and a processor 39.

[0121] The communication IF 32 is an interface for inputting and outputting signals so that the medical institution server 30 can communicate with external devices. The input / output IF 33 functions as an interface with an input device for receiving an input operation from a user and an output device for presenting information to the user.

[0122] The memory 35 is for temporarily storing programs and data to be processed by the programs, and is a volatile memory such as a DRAM (Dynamic Random Access Memory).

[0123] The storage 36 is a storage device for saving programs and data, such as a flash memory or a hard disk drive (HDD). The programs include, for example, the following programs: ·OS (Operating System) programs · Programs for applications that process information (e.g. web browsers) The data includes, for example, the following data: Databases referenced in information processing Data obtained by performing information processing (i.e., the results of performing information processing)

[0124] The processor 39 is hardware for executing an instruction set written in a program stored in the storage 36, and is composed of an arithmetic unit, a register, peripheral circuits, etc.

[0125] (1-8. Functional configuration of medical institution server 30) FIG. 8 is a diagram showing the functional configuration of the medical institution server 30. As shown in FIG. As shown in FIG. 5, the medical institution server 30 functions as a communication unit 301, a storage unit 302, and a control unit 303.

[0126] The communication unit 301 performs processing for the medical institution server 30 to communicate with external devices.

[0127] The storage unit 302 stores data and programs used by the medical institution server 30. The storage unit 302 stores, for example, the following data: · Patient Database (Patient DB3021) 3021 ·Doctor Database (Doctor DB3023) 3022 ·Medical history database (Medical history DB3022) 3023 ·Medical treatment time slot database (Medical treatment time slot DB3024) 3024 · Reservation database (Reservation DB3025) 3025

[0128] The patient DB 3021 is a database for managing information on patients who undergo medical treatment at a medical institution. The data structure of the patient DB 3021 will be described in detail later.

[0129] The doctor DB 3023 is a database for managing information on doctors who belong to medical institutions. The data structure of the doctor DB 3023 will be described in detail later.

[0130] The medical history DB 3022 is a database for managing information related to the history of medical treatments performed at medical institutions. The data structure of the medical history DB 3022 will be described in detail later.

[0131] The consultation time slot DB 3024 is a database for managing information related to consultation time slots scheduled at medical institutions. The data structure of the consultation time slot DB 3024 will be described in detail later.

[0132] The reservation DB 3025 is a database for managing information regarding reservations for medical treatment at medical institutions. The data structure of the reservation DB 3025 will be described in detail later.

[0133] The control unit 303 performs functions shown as various modules by the processor 39 of the medical institution server 30 performing processing according to a program. The control unit 303 performs functions as a transmission / reception control module 3031, an input reception module 3032, a query module 3033, a reservation module 3034, and an output module 3035.

[0134] The transmission / reception control module 2031 controls the process in which the medical institution server 30 transmits and receives signals to and from external devices in accordance with a communication protocol.

[0135] The input reception module 3032 performs a process of receiving input information in response to an operation from the terminal device 10. The input reception module 3032 receives, for example, a medical history input during daily medical treatment by a doctor at a medical institution, and records the medical history as a new record in the medical history DB.

[0136] The inquiry module 3033 refers to the consultation time slot DB 3024 and the reservation DB 3025 to inquire about available reservation slots for the desired reservation date included in the reservation request.

[0137] The reservation module 3034 accepts a reservation request for an available reservation slot in response to an operation from the terminal device 10, and records the reservation request as a new record in the reservation DB 3025, thereby making a reservation for a medical examination.

[0138] The output module 3035 outputs various information obtained by the executed processing in response to an operation by a patient or a medical professional. The output module 3035 outputs the following information, for example: Information regarding available reservation slots for the patient terminal 10A -Notify patient terminal 10A of reservation completion Past medical history of the patient to be treated on the medical terminal 10B

[0139] (1-9. Data Structure of Each Database Managed by Medical Institution Server 30) Next, an example of the data structure of each database stored in the storage unit 302 of the medical institution server 30 will be described with reference to FIG.

[0140] (1-9-1.Patient DB3021) FIG. 9A is a diagram showing an example of the data structure of the patient DB 3021. As shown in FIG. 9A, the patient DB 3021 stores information about patients who use a medical institution. A new record is recorded in the patient DB 3021 when the patient is registered at the medical institution for the first time.

[0141] The patient DB 3021 includes the following items: "Patient ID", "Name", "Date of birth", "Sex", "Address", "Telephone number", "Insured person information", and "Attending physician".

[0142] The item "patient ID" stores patient identification information that can identify the patient at the medical institution.

[0143] The item "name" stores information about the name of the patient corresponding to the patient ID.

[0144] The item "Date of Birth" stores information about the date of birth of a patient corresponding to a patient ID.

[0145] The item "gender" stores information regarding the gender of a patient corresponding to the patient ID.

[0146] The item "address" stores information about the address of the patient corresponding to the patient ID.

[0147] The item "phone number" stores information about the telephone number of the patient corresponding to the patient ID.

[0148] The item "insured person number" stores the insured person number on the health insurance card of the patient corresponding to the patient ID.

[0149] The item "attending physician" stores identification information for the attending physician in charge of the patient corresponding to the patient ID. It should be noted that the structure of the patient DB 3021 shown in FIG. 9A is merely an example, and the patient DB 3021 may include columns in which other data items are stored.

[0150] (1-9-2. Doctor DB3023) FIG. 9B is a diagram showing an example of the data structure of the doctor DB 3023. As shown in FIG. 9B, the doctor DB 3023 stores information about doctors who belong to medical institutions. A new record is recorded in the doctor DB 3023 when a doctor is assigned to a medical institution.

[0151] The doctor DB 3023 includes the following items: "Doctor ID", "Name", "Department", "Qualification", "Title", and "Contact information".

[0152] The item "Doctor ID" stores identification information of a doctor that can identify a doctor at a medical institution.

[0153] The item "Name" stores information about the name of the doctor corresponding to the doctor ID.

[0154] The item "medical department" stores information about the medical department to which the doctor corresponding to the doctor ID belongs.

[0155] The item "Qualification" stores information about qualifications, such as specialist qualifications, held by the doctor corresponding to the doctor ID.

[0156] The item "position" stores information about the position in the medical department of the doctor corresponding to the doctor ID.

[0157] The item "contact information" stores information such as a telephone number or email address as the contact information of the doctor corresponding to the doctor ID. It should be noted that the structure of the doctor DB 3023 shown in FIG. 9B is merely an example, and the doctor DB 3023 may include columns in which other data items are stored.

[0158] (1-9-3. Medical History DB3022) FIG. 9C is a diagram showing an example of the data structure of the medical history DB 3022. 9C, the medical history DB 3022 stores information about the history of medical treatment at a medical institution. A new record is recorded in the medical history DB 3022 when a medical professional, including a doctor, inputs the details of medical treatment performed at a medical institution.

[0159] The medical history DB3022 includes the items "medical history ID", "patient ID", "doctor", "date and time of medical treatment", "contents of medical treatment", "examination results", "prescription contents", and "treatment, surgery information".

[0160] The item "medical treatment history ID" stores identification information of a medical treatment history that can identify medical treatment performed at a medical institution.

[0161] The item "patient ID" stores identification information of a patient who underwent medical treatment corresponding to the medical treatment history ID.

[0162] The item "Doctor in Charge" stores identification information of the doctor in charge of the medical treatment corresponding to the medical treatment history ID.

[0163] The item "treatment date and time" stores information regarding the date and time when treatment corresponding to the treatment history ID was performed.

[0164] The item "medical treatment details" stores the details of medical treatment corresponding to the medical treatment history ID. Examples of medical treatment details include the following: Information about symptoms reported by the patient to the doctor - Medical findings made by the doctor during the medical examination (including the index of disease activity at the time of the examination)

[0165] The item "examination result" stores information regarding the results of examinations performed in the medical treatment corresponding to the medical treatment history ID.

[0166] The item "Prescription contents" stores information about medicines prescribed by a doctor to a patient in the medical treatment corresponding to the medical history ID.

[0167] The item "treatment, surgery information" stores information related to the treatment or surgery performed in the medical treatment corresponding to the medical treatment history ID. It should be noted that the structure of the medical history DB 3022 shown in FIG. 9C is merely an example, and the medical history DB 3022 may include columns in which other data items are stored.

[0168] (1-9-4. Consultation time slot DB3024) FIG. 9D is a diagram showing an example of the data structure of the consultation time slot DB 3024. As shown in FIG. 9D, the medical treatment time slot DB3024 stores information on the medical treatment schedule for each medical department in the medical institution. The medical treatment time slot DB3024 is set based on the medical treatment time slot set in the medical institution, and a new record is recorded when the work shift of the doctor in charge for the target period is set.

[0169] The consultation time slot DB 3024 includes an item "consultation time slot ID", an item "medical department", an item "doctor in charge", an item "consultation date and time", and an item "consultation time slot".

[0170] The item "treatment time slot ID" stores identification information of a treatment time slot that can specify a treatment time slot at a medical institution.

[0171] The item "medical department" stores information about the medical department corresponding to the consultation time slot corresponding to the consultation time slot ID.

[0172] The item "Doctor in Charge" stores identification information of the doctor in charge of the medical treatment corresponding to the medical treatment time slot ID.

[0173] The item "treatment date and time" stores information regarding the date and time of treatment corresponding to the treatment time slot ID.

[0174] The item "treatment time slot" stores information about the treatment time slot corresponding to the treatment time slot ID. It should be noted that the structure of the consultation time slot DB 3024 shown in FIG. 9D is merely an example, and the consultation time slot DB 3024 may include columns in which other data items are stored.

[0175] (1-9-5. Reservation DB3025) FIG. 9E is a diagram showing an example of the data structure of the reservation DB 3025. 9E, the reservation DB 3025 stores information regarding reservations for medical treatment at medical institutions. A new record is recorded in the reservation DB 3025 when a patient makes a reservation for medical treatment.

[0176] The reservation DB 3025 includes an item "reservation ID", an item "patient ID", an item "doctor in charge", and an item "reservation time slot".

[0177] The item "reservation ID" stores reservation identification information that can identify a medical appointment at a medical institution.

[0178] The item "patient ID" stores identification information of a patient who will receive medical treatment in accordance with a medical treatment appointment corresponding to the appointment ID.

[0179] The item "Doctor in Charge" stores identification information of the doctor in charge of the medical treatment for the medical treatment reservation corresponding to the reservation ID.

[0180] The item "reservation time slot" stores identification information of the treatment time slot to which the treatment reservation corresponding to the reservation ID applies. Note that the structure of reservation DB 3025 shown in FIG. 9E is merely an example, and reservation DB 3025 may include columns in which other data items are stored.

[0181] <2. Overview of the embodiment> Hereinafter, an outline of an embodiment of the present invention will be described with reference to Fig. 10. Fig. 10 is a diagram showing an outline of an embodiment of the present invention.

[0182] As shown in Fig. 10, in the system 1, stool is photographed by a patient. The stool is photographed periodically (for example, every day) at the patient's home. A terminal device 10 (patient terminal 10A), such as a smartphone used by the patient in daily life, is used to photograph the stool. It is preferable to photograph the first stool of the day.

[0183] Next, the acquired stool image is transmitted to the evaluation server 20, which judges the disease activity of the patient. The evaluation server 20 notifies the patient terminal 10A of the evaluation result. The user can grasp his / her own disease activity from the state of the stool of that day.

[0184] The evaluation server 20 also notifies the evaluation result to the medical terminal 10B used by the attending physician. The attending physician checks the disease activity of the patient as necessary. At this time, the evaluation server 20 may issue an alert to the attending physician for patients whose disease activity is worsening.

[0185] The evaluation server 20 also cooperates with the medical institution server 30 via API, and cooperates with the medical institution server 30 regarding medical appointments. When a predetermined judgment criterion is met, the evaluation server 20 notifies the patient that a medical examination by a doctor is necessary. Here, the judgment criterion for the necessity of medical examination may be, for example, the following patterns: - If the disease activity on that day exceeds the threshold index (determined by the evaluation results on that day) - When a clear deterioration trend is observed over a certain period of time

[0186] In this way, System 1 uses the act of excretion, a routine and ongoing activity, to evaluate disease activity in patients with a history of a specific disease, and supports the treatment of the disease by notifying the patient and medical institution of the result. The processing of the system 1 will be described in detail below.

[0187] <3. Operation of System 1> Various processes performed by the system 1 will now be described.

[0188] (3-1. Evaluation of disease activity) First, the disease activity evaluation process, which is the main function of the system 1, will be described. FIG. 11 is a flowchart showing the disease activity evaluation process performed by the system 1.

[0189] As shown in FIG. 11, in the disease activity evaluation process, first, the patient operates the patient terminal 10A to photograph feces excreted by the patient (step S101). Specifically, the photographing unit 183 of the patient terminal 10A starts the camera 160 in response to an operation from the user, and photographs the stool as a subject. An example of the screen at this time will be described later as a first screen example P1. In addition, the patient can select a tag and input a comment as necessary. The data processing unit 185 inputs the image data captured by the imaging unit 183 to the detection unit 184 .

[0190] After step S101, the patient terminal 10A detects feces from the captured image (step S102). Specifically, the detection unit 184 of the patient terminal 10A inputs the image input from the imaging unit 183 in step S101 to the detection model 172, and thereby acquires a detection result of an object related to excrement included in the image output from the detection model 172. The detection unit 184 inputs the acquired object detection result to the data processing unit 185.

[0191] After step S102, the patient terminal 10A transmits the stool image to the evaluation server 20 (step S103). Specifically, the data processing unit 185 of the patient terminal 10A transmits the following information to the evaluation server 20: · Mail image data Information about the detected object input from the detection unit 184 in step S102 Tag data and comment data entered when taking a flight image That is, the data processing unit 185 is triggered by the detection of an object related to excrement, and transmits the corresponding image (feces image) to the evaluation server 20 via the transmission / reception unit 182. Therefore, if no object related to excrement is detected from the captured image, the data processing unit 185 does not transmit the corresponding image to the evaluation server 20.

[0192] After step S103, the patient terminal 10A deletes the stool image (step S104). Specifically, the data processing unit 185 of the patient terminal 10A deletes the data of the image in which an object related to excrement was detected in step S102 without storing the data in the storage area of ​​the patient terminal 10A. The data processing unit 185 also deletes the data without storing the stool image in an external storage area in which captured images are stored, such as a cloud server that is routinely used by the patient.

[0193] After step S103, the evaluation server 20 acquires a stool image (step S201). Specifically, the transmission / reception control module 2031 of the evaluation server 20 receives various input data transmitted from the terminal device 10, and inputs the data to the acquisition module 2032. The acquisition module 2032 acquires the input data, stores the data in the respective storage areas, and records a new record in the image DB 2022.

[0194] Here, the type and storage area of ​​the data newly recorded in step S201 will be described. Mail image data: Image storage area (external server) Attribute information for flight images: Image DB2022 Information about detected object: Object DB2023 Tag data entered by the patient: Image DB2022 Comments entered by patients: Comment DB2025

[0195] After step S201, the assessment server 20 estimates a disease activity index for a particular disease in the patient (step S202). Specifically, the evaluation module 2033 of the evaluation server 20 inputs the image data newly recorded in step S201 to the activity estimation model 2027, and obtains the estimated value of the disease activity index together with its likelihood output from the activity estimation model 2027. The evaluation module 2033 uses the obtained estimated value of disease activity to record it in a new record in the evaluation result DB.

[0196] In step S202, the determination module 2034 refers to the new record in the evaluation result DB and determines whether the corresponding patient is in a condition that requires a doctor's examination. The determination module 2034 updates the item "need for medical treatment" in the evaluation result DB with the determination result.

[0197] After step S202, the evaluation server 20 outputs the evaluation result to the patient terminal 10A (step S203). Specifically, the output module 2037 of the evaluation server 20 outputs the following information to the patient terminal 10A. -Disease activity assessment results (estimated values) -Whether or not a doctor is required Address information for medical appointments

[0198] That is, when the determination module 2034 determines in step S202 that medical treatment by a doctor is necessary, the output module 2037 performs API linkage with the medical institution server 30 in step S203. The output module 2037 acquires address information (e.g., URL information) of a site related to consultation reservations provided by the medical institution server 30, and outputs it to the patient terminal 10A.

[0199] After step S203, the patient terminal 10A presents the evaluation result to the patient (step S105). Specifically, the display control unit 186 of the patient terminal 10A displays the information output from the evaluation server 20 received by the transmission / reception unit 182 on the display 132 of the patient terminal 10A. In this way, the evaluation result of disease activity using stool images is presented to the patient. The evaluation result includes address information related to medical appointments. An example of the screen at this time will be described later as a third screen example P3.

[0200] After step S203, the evaluation server 20 transmits the evaluation result to the medical terminal 10B used by the attending physician (step S204). Specifically, the output module 2037 of the evaluation server 20 queries the patient DB 3021 and doctor DB 3023 of the medical institution server 30, and transmits the following information to the doctor registered as the patient's primary doctor. Image data of the flights that were evaluated Attribute data on flight images (items included in Image DB2022) Tag data and comment data entered by the patient -Disease activity assessment results (estimated values)

[0201] At this time, the doctor can input comments for the patient before the actual examination at the medical institution. The comments input by the doctor through the medical terminal 10B are stored in the evaluation history DB2026 under the item “comments from the attending physician”.

[0202] In addition, the process of step S204 may notify the terminal used by the attending physician only when the disease activity index estimated in step S202 indicates a state of concern (in other words, when it is determined that the patient's condition is worsening). The following are examples of cases in which the physician is notified of the evaluation result of the disease activity. If the disease activity index is worse than a certain threshold If the disease activity index is worse than the previous evaluation result - If the disease activity index shows a worsening trend during the evaluation period When a physician is in charge of multiple patients, there is a concern that the volume of notifications would be enormous if the physician were to be notified of all evaluation results that are not problematic. By selecting who to notify, it is possible to send only important notifications to the physician. This completes the disease activity evaluation process by the system 1.

[0203] (3-2. Reservation Processing) Next, the reservation process by the system 1 will be described. FIG. 12 is a flowchart showing the reservation process by the system 1.

[0204] As shown in FIG. 12, in the reservation process, first, the patient terminal 10A makes a reservation request (step S111). Specifically, the input receiving unit 181 of the patient terminal 10A receives an operation in which the patient clicks on the address information related to the medical appointment sent from the evaluation server 20. Then, the patient terminal 10A accesses a site related to the medical appointment provided by the medical institution server 30, thereby transmitting a request related to the medical appointment to the medical institution server 30. The appointment request may include the patient ID of the patient at the medical institution.

[0205] After step S111, the medical institution server 30 receives the reservation request (step S311). Specifically, the input reception module 3032 of the medical institution server 30 receives the reservation request transmitted from the patient terminal 10A, and inputs the patient ID included in the reservation request to the inquiry module 3033. The inquiry module 3033 refers to the patient DB 3021 using the input patient ID, identifies the attending physician, and then refers to the consultation time slot DB 3024 and the reservation DB 3025 to extract available reservation slots.

[0206] After step S311, the medical institution server 30 outputs the available reservation slots to the patient terminal 10A (step S312). Specifically, the output module 3035 of the medical institution server 30 outputs the available appointment slots extracted in step S311 to the patient terminal 10A.

[0207] After step S312, the patient terminal 10A presents the available reservation slots to the patient (step S112). Specifically, the display control unit 186 of the patient terminal 10A displays the information regarding the available appointment slots received by the transmission / reception unit 182 on the display 132. In this way, the available appointment slots are presented to the patient. The patient considers the time slot during which he or she would like to be examined from the presented available appointment slots.

[0208] After step S112, the patient terminal 10A issues an instruction to select an appointment slot (step S113). Specifically, the input receiving section 181 of the patient terminal 10A receives the designation of the desired time period for medical treatment input by the patient, and transmits it to the evaluation server 20 via the transmitting / receiving section 182.

[0209] After step S113, the medical institution server 30 sets a reservation (step S313). Specifically, the appointment module 3034 of the medical institution server 30 records the desired time period for medical treatment received from the patient terminal 10A in step S113 as a new record in the appointment DB 3025. This makes it possible to make an appointment for medical treatment.

[0210] After step S313, the medical institution server 30 transmits a reservation completion notification to the patient terminal 10A (step S314). Specifically, the output module 3035 of the medical institution server 30 creates information regarding the completion of the reservation and transmits it to the patient terminal 10A.

[0211] After step S314, the patient terminal 10A receives a reservation completion notification (step S114). Specifically, the display control unit 186 of the patient terminal 10A displays information about the appointment completion notification received by the transmission / reception unit 182 on the display 132. This notifies the patient that the appointment has been completed. This completes the reservation process by the system 1.

[0212] (3-3. Aggregation of evaluation history) Next, the process of aggregating evaluation histories by the system 1 will be described. FIG. 13 is a flowchart showing the process of aggregating evaluation histories by the system 1.

[0213] As shown in FIG. 13, in the process of aggregating the evaluation history, first, the evaluation server 20 refers to the evaluation history DB 2026 and aggregates the evaluation history of the disease activity (step S221). Specifically, the aggregation module 2035 of the evaluation server 20 aggregates the evaluation history of the patient to be evaluated during a predetermined evaluation period. The aggregation module 2035 may execute the aggregation process at a preset frequency. The aggregation module 2035 may also aggregate the evaluation history in response to an operation by the patient or a medical professional including the patient's doctor.

[0214] After step S221, the assessment server 20 outputs the counting result (step S222). Specifically, the output module 2037 of the evaluation server 20 creates a summary report including the following information and outputs it to the patient terminal 10A. -Frequency of disease activity assessment during the evaluation period (frequency of stool image acquisition) - Time series of disease activity index as an evaluation result

[0215] After step S222, the patient terminal 10A presents the compilation result of the evaluation history to the patient (step S122). Specifically, the display control unit 186 of the patient terminal 10A displays the tally report received by the transmitting / receiving unit 182 on the display 132. In this way, the tally report of the evaluation results is presented to the patient. The tally report includes the following information: - Notification authorizing continued acquisition of stool images when the frequency of acquisition of stool images exceeds a certain threshold - Notifications to encourage continued acquisition of stool images when the frequency of acquisition of stool images falls below a certain threshold

[0216] In other words, if the patient is able to maintain a state of taking stool images every day and continually evaluating disease activity, System 1 will evaluate this and reward the patient for having an appropriate attitude toward health management. On the other hand, if the patient neglects to take daily stool images and is unable to continuously evaluate disease activity, System 1 will point this out and request the patient to improve their health management attitude, as it is inappropriate. With this, the reservation process by System 1 is completed.

[0217] (3-4. Endoscope-equivalent image generation process) Next, the process of generating an endoscope-equivalent image by the system 1 will be described. This process is a secondary process, and is selectively executed, for example, at the discretion of a doctor. Specifically, in step S204 shown in Fig. 11, this process is triggered by a doctor receiving a notification of the disease activity evaluation result inputting an instruction to generate an endoscope-equivalent image to the evaluation server 20 for the purpose of checking the state of the patient's intestines. FIG. 14 is a flowchart showing the process of generating an endoscope-equivalent image.

[0218] As shown in FIG. 14, in the process of generating an endoscope-equivalent image, first, the evaluation server 20 generates an endoscope-equivalent image (step S241). Specifically, the image generation module 2036 of the evaluation server 20 inputs a stool image specified by a doctor to the image synthesis model 2028, thereby acquiring a specified intestinal image output from the image synthesis model 2028. After the image corresponding to the endoscope is stored in a predetermined storage area, the image generation module 2036 creates address information (e.g., URL information) indicating the storage area.

[0219] After step S241, the output module 2037 outputs the endoscope-equivalent image to the medical terminal 10B (step S242). Specifically, the output module 2037 of the evaluation server 20 outputs address information for the endoscope-equivalent image generated by the image generation module 2036 in step S241 to the medical terminal 10B. The doctor can check the endoscope-equivalent image of the patient by accessing the output address information. This allows the doctor to estimate the state of the patient's intestines. This completes the process of generating an endoscope-equivalent image by the system 1.

[0220] <4. Screen examples> Below, examples of screens in the system 1 will be explained.

[0221] (4-1. First screen example P1) 15 is a diagram showing a first screen example P1. The first screen example P1 is a screen displayed on the patient terminal 10A in step S101 shown in FIG. 11, and shows a stool photographing operation.

[0222] As shown in Fig. 15, the first screen example P1 displays stool as a subject to be photographed. After defecation, the patient activates the camera 160 on the patient terminal 10A to photograph the stool. The camera is adjusted so that the toilet bowl fits within the angle of view, and the photographing button B1 is pressed to photograph the stool image.

[0223] In addition, before shooting, the user can select settings for the storage process of the captured image on the setting screen that is displayed by pressing the setting button B2. The storage process of the captured image includes the following patterns. Whether to save in the memory area of ​​the terminal device 10 (default setting: do not save) Whether or not to save in the storage area of ​​the cloud server used by the terminal device 10 (default setting is not to save) For example, when photographing is performed using a medical terminal 10B such as a tablet in a medical institution, the photographed stool image may be stored in the medical terminal 10B.

[0224] (4-2. Second screen example P2) Fig. 16 is a diagram showing a second screen example P2. The second screen example P2 is a screen displayed on the patient terminal 10A in step S101 shown in Fig. 11, and shows a registration operation of a stool image used for evaluation of disease activity. In other words, it shows an execution operation of evaluation of disease activity.

[0225] As shown in FIG. 16, in the second example screen P2, information on the attributes of the image (photographed date, photographed time, user ID) is displayed in an attribute column C1 together with the photographed flight image. Also, the second screen example P2 displays a tag input field C2 in which a tag corresponding to one's own physical condition or the like can be selected.

[0226] Also, the second screen example P2 displays a patient comment field C3 in which the patient can input any content he or she wishes to input as text data. When the patient presses the evaluation execution button B3, the process from step S102 onward shown in FIG. 11 is executed as an evaluation of disease activity using the content.

[0227] (4-3. Third screen example P3) Fig. 17 is a diagram showing a third screen example P3. The third screen example P3 is a screen displayed on the patient terminal 10A in step S105 shown in Fig. 11, and shows a state in which the evaluation result of the disease activity is presented to the patient.

[0228] 17, the third screen example P3 displays an evaluation result field C5 showing the evaluation result of disease activity. The following information is listed in the evaluation result field C5. Evaluation date: The date the evaluation was performed Patient name: Patient name (may be omitted when displayed on the patient terminal 10A) Disease activity (estimated value): Estimated value of the disease activity index that is the evaluation result Likelihood: An index showing the likelihood of the estimate · Necessity of medical treatment: Judgment result of whether medical treatment by a doctor is required Rating details: Information about tags and comments entered

[0229] Here, in the display of the necessity of medical treatment, if the evaluation result of the disease activity index indicates a state of concern regarding the condition, the deadline for the next medical treatment date may be notified. Examples of notifying the deadline for the medical treatment date include, for example, any of the following. If the disease activity index is worse than a certain threshold If the disease activity index is worse than the previous evaluation result - If the disease activity index shows a worsening trend during the evaluation period

[0230] In these cases, the patient is not only notified that medical treatment is necessary, but also prompted to visit a medical institution by setting a deadline for the next medical treatment. Also, the number of days until the deadline for the next medical treatment may be adjusted according to the evaluation result of the disease activity index, i.e., according to the estimated degree of the disease. For example, if the estimated disease condition is particularly severe, a shorter deadline is set than if the condition is only slightly severe.

[0231] In addition, a notification may be displayed to encourage the use of a previously prescribed strong drug according to the evaluation result of the disease activity index. Generally, strong drugs include those that have certain side effects or those that some patients hesitate to use, such as suppositories. Such strong drugs are prescribed by doctors in medical practice and may be used as an as-needed drug with a limited administration period according to changes in the condition in daily life. In such a case, if the condition estimated from the evaluation result of the disease activity index is a worrying state on the display screen of the evaluation result, the use of the strong drug can be encouraged so that the patient can use it in a timely manner.

[0232] Furthermore, if the estimated likelihood falls below a threshold, this may be displayed as an evaluation result, and the patient may be notified that the reliability of the estimation result is low. In addition, if the estimated likelihood is high and the disease activity index is high (disease condition is poor), the result may be displayed and a notice may be sent to strongly encourage the patient to seek medical treatment at a medical institution. In addition, a deadline until the next scheduled medical treatment date may be set according to the likelihood and the disease activity index.

[0233] The third screen example P3 also displays a medical appointment appointment section C6 for making an appointment. The medical appointment appointment section C6 displays address information for the patient that can access the appointment site. In other words, by accessing the appointment site from the address information through API cooperation between the evaluation server 20 and the medical institution server 30, it is possible to receive an inquiry about available appointment slots without having to enter a patient ID, etc.

[0234] In addition, the third screen example P3 displays an advice column C7. When the patient clicks on the advice column C7, general points that the patient should take note of in daily life in the current state of disease activity are displayed as advice to the patient.

[0235] Furthermore, the third screen example P3 displays a doctor's comment field C8. When the patient clicks on the doctor's comment field C8, the contents of the comment entered by the patient's doctor are displayed.

[0236] The following information may be displayed on the evaluation result display screen shown in FIG. Number of flights on the current day or past days Number of bloody stools on the current day or in the past days Number of days until next appointment

[0237] <5.Summary> As described above, in the system 1, a disease activity index for a specific disease of a patient is estimated by image processing of a stool image taken by the patient using the patient terminal 10A, and is output to the patient terminal 10A. As a result, unlike conventional systems, there is no need for patients to input multiple evaluation items, such as their health condition and the presence or absence of subjective symptoms, and disease activity can be continuously evaluated with extremely simple operations by patients.

[0238] Furthermore, in the system 1, when estimating the disease activity index, an estimated value of the disease activity index is output using an activity estimation model 2027 created by machine learning using learning data in which a stool image is input data and the disease activity index is output as correct answer data. Therefore, by performing machine learning using a large amount of learning data, it is possible to ensure the accuracy of the estimation of the disease activity index.

[0239] In addition, when estimating a disease activity index, the system 1 outputs an estimated location of the lesion range for the disease. Therefore, when the attending physician wants to check the evaluation results by the system 1, information that contributes to treatment can be provided to the physician.

[0240] Furthermore, in the system 1, when estimating a disease activity index, the likelihood of the estimated disease activity index is output. Therefore, by mentioning how likely the estimated value of disease activity, which is the evaluation result, is, it is possible to prompt a decision on how to handle the evaluation result.

[0241] Furthermore, in the system 1, when stool is detected in an image taken by a patient, the image is transmitted to the evaluation server 20 without being stored in the terminal device 10. Therefore, generally, images of stool that a patient does not want to store in his / her own terminal device 10 can be automatically deleted from the terminal device 10, thereby improving the convenience of the system 1 for patients.

[0242] Furthermore, in the system 1, when the estimated disease activity index exceeds a preset threshold, the system notifies the patient that medical treatment by the patient's doctor is necessary. This enables timely medical treatment by the doctor and supports medical treatment by the medical institution.

[0243] In addition, the system 1 counts the frequency of stool image acquisition by the patient, and when the counted acquisition frequency falls below a certain standard, a notification is sent to the patient terminal 10A to encourage the patient to continue acquiring stool images. Therefore, if the patient neglects to take stool images for continuous disease activity evaluation, this fact can be pointed out to the patient.

[0244] Furthermore, in the system 1, when the collected frequency of acquisition exceeds a certain criterion, a notification approving the continued acquisition of stool images is sent to the terminal device 10. Therefore, for patients who are continually taking stool images to adequately grasp their own health condition, their posture can be evaluated and they can be motivated to continue their efforts.

[0245] Furthermore, in the system 1, the evaluation result is notified to the terminal device 10 used by the patient's doctor. This allows them to smoothly share with the attending physician any concerns that the patient's condition may have worsened, which can be helpful in subsequent medical treatment at the medical institution.

[0246] Furthermore, the system 1 may be configured to notify the attending physician only when a preset threshold is exceeded. In this case, even if the attending physician is in charge of multiple patients, the evaluation results to be notified to the attending physician can be automatically selected, thereby preventing the attending physician from receiving an overwhelming amount of notifications.

[0247] In addition, the system 1 generates an endoscopic equivalent image that is estimated as the state of the patient's intestines when the patient excretes the stool related to the stool image by image processing using the acquired stool image, and notifies the terminal used by the attending physician of the result. In this way, when the physician wants to check the state of the patient's intestines due to a change in the patient's pathology, the physician can be provided with an image showing the estimated state of the patient's intestines before the endoscopic examination.

[0248] <6. Variations> A modification of this embodiment will now be described. In the following description, the description of the same configuration as in the above embodiment will be omitted.

[0249] (6-1) First modified example: Evaluation method via estimation of the content of specific proteins In the first modification of the system 1, the method of estimating the disease activity index is different from that of the above embodiment. That is, in this modification, the evaluation server 20 does not output an estimated value of the disease activity index in response to the input of a stool image, but estimates the content of a specific protein in the stool and then estimates the disease activity index from the estimated content. This process will be described with reference to FIG. 18.

[0250] Fig. 18 is a diagram for explaining the flow of the estimation process of the disease activity index according to the first modified example. This diagram explains only the process corresponding to step S202 shown in Fig. 11. The processes before and after that are the same as those shown in Fig. 11, and the explanation thereof will be omitted.

[0251] As shown in FIG. 18, in the first modified example, after step S201 shown in FIG. 11, the evaluation server 20 estimates the content of calprocin, which is a specific protein contained in the stool (step S2021). Specifically, the evaluation module 2033 inputs an image to the content estimation model, and obtains the content of calprocin output from the content estimation model.

[0252] Here, the system 1 according to the first modified example has a content estimation model instead of the above-mentioned activity estimation model 2027. The content estimation model is created by machine learning using learning data in which a stool image (or object data related to the detected excrement) is input data and the content of calprocin in the stool is output data. The evaluation module 2033 inputs the acquired content of calprocin to the determination module 2034.

[0253] After step S2021, the evaluation server 20 judges the disease activity from the estimated value of the calprocin content (step S2022). Specifically, the determination module 2034 uses the estimated value of the calprocin content input from the evaluation module 2033 in step S2021 to compare it with a pre-stored disease activity index determination criterion to determine the disease activity index of the patient. The determination criterion used at this time is set for each range of the calprocin content, with the range of the calprocin content and the disease activity index estimated within that range corresponding to each other. In this way, the estimated value of the disease activity index is determined.

[0254] In this way, in the system 1 according to the first modification, an acquired stool image is input to the content estimation model, which outputs an estimated value of the calprocin content in the stool. Then, in the system 1 according to the first modification, the estimated value of the content is used to determine the estimated value of the disease activity index based on a determination criterion that associates a preset content with an estimated value of the disease activity index. Therefore, the estimated value of the disease activity index can be evaluated through the estimation of the content of a specific protein (calprocin) that is said to be closely related to the disease activity of inflammatory bowel disease, and it is expected that the evaluation can be performed with high accuracy.

[0255] (6-2) Second Modification: Use of Rule-Based Judgment In the second modification of the system 1, the method of estimating the disease activity index is further different from the two patterns described above. That is, in this modification, first, when taking a stool image, at least one of the following behavioral history and biological information is received from the patient.

[0256] 1) Information regarding the history of excretory behavior (behavioral history) Number of flights on the previous or same day ·Defecation time Stool characteristics Degree of blood in stool Image data of fecal incontinence, etc.

[0257] 2) Information regarding the history of eating habits, including medication (behavioral history) -Medication history (whether or not antidiarrheal drugs or painkillers were used) -Meal contents from the previous day or the day (including meal images)

[0258] 3) Biometric information Height, weight, BMI Body temperature ·pulse · Severity of abdominal pain, etc.

[0259] In the system 1 according to the second modification, the disease activity index is estimated by using the stool image and also by determining the disease activity index using the behavior history and biological information input by the patient. This process will be described with reference to FIG.

[0260] Fig. 19 is a diagram for explaining the flow of the estimation process of the disease activity index according to the second modified example. This diagram explains only the process corresponding to step S202 shown in Fig. 11. The processes before and after that are the same as those shown in Fig. 11, and the explanation thereof will be omitted.

[0261] As shown in FIG. 10, in the third modified example, after step S201 shown in FIG. 11, the evaluation server 20 estimates a disease activity index using a stool image (step S2023). Specifically, the evaluation module 2033 of the evaluation server 20 inputs a stool image to the activity estimation model 2027, similar to step S202 shown in FIG. 11, to obtain an estimated value of the disease activity index output from the activity estimation model 2027.

[0262] After step S2023, the assessment server 20 performs a rule-based disease activity index determination using other inputs from the patient (step S2024). Specifically, the evaluation module 2034 of the evaluation server 20 compares the behavior history and biological information input by the patient with a preset evaluation criterion to determine the disease activity index of the patient. The evaluation criterion used at this time is set for each numerical range, in which the numerical ranges of the behavior history and biological information are associated with the disease activity index estimated within the range. In this way, the judgment value of the disease activity index is obtained.

[0263] After step S2024, the evaluation server 20 performs a comprehensive evaluation by comparing the first disease activity index estimated from the stool image with the second disease activity index determined from the behavior history and biological information (step S2025). Specifically, if the first disease activity index and the second disease activity index are of the same grade, the evaluation module 2033 adopts the grade.

[0264] On the other hand, when the first disease activity index and the second disease activity index are not of the same grade, the evaluation module 2033 adopts the worse value as the disease state. The judgment method is not necessarily limited to adopting the worse value. For example, when the first disease activity index and the second disease activity index are not of the same grade, any value between them (for example, the arithmetic mean value of both) may be adopted. The method of comprehensive evaluation using such two indexes can be set arbitrarily.

[0265] In this way, the system 1 according to the second modification acquires at least one of the behavior history and the biological information input by the patient, estimates the disease activity index using the stool image, and determines the disease activity index using the behavior history and the biological information. Then, the system 1 according to the second modification performs a comprehensive evaluation using the two disease activity indices obtained. Therefore, it is possible to perform a more reliable evaluation of disease activity by using knowledge of various judgment criteria accumulated in the past. Also, by comparing the estimation result by the activity estimation model 2027 with the rule-based judgment result, it is possible to obtain information that can be used for re-learning the activity estimation model 2027, which can lead to an improvement in the accuracy of the estimation process of disease activity from stool images.

[0266] In addition, the above-mentioned rule-based judgment may use the stool image and the colonization information input from the patient terminal 10A, as well as the medical history at the medical institution. For example, the following judgment rules may be used. 1) Number of bowel movements (user input information) Normal count: +0 points - 1-2 times / day more than normal: +1 point 3-4 times / day more than normal: +2 points - 5 times / day more than normal: +3 points

[0267] 2) Bloody stool (determined from user input information or stool images) No blood in stool: +0 points - Slight blood stains (streaks) during less than half of bowel movements: +1 point - Obvious blood in most bowel movements: +2 points Mostly blood: +3 points

[0268] 3) Mucosal findings (using the results of the most recent endoscopic examination) Normal or inactive findings: +0 points Mild (redness, decreased vascular visibility, mild weakness): +1 point Moderate (significant redness, loss of vascular visibility, fragility, erosion): +2 points ·Severe (spontaneous bleeding, ulcer): +3 points

[0269] 4) Overall evaluation by doctor (results of most recent doctor's examination) ·Normal: +0 points Mild: +1 point ·Moderate: +2 points ·Severe: +3 points

[0270] 5) Overall Judgment A second disease activity index is determined based on the total score of 1) to 4) above. ·0 points: Lv1 ·1~2 points: Lv2 ·3~5 points: Lv3 ·6~10 points: Lv4 ·11~12 points: Lv5

[0271] (6-3) Third Modification: Re-learning of Patient-Specific Activity Estimation Model 2027 The third modified example of the system 1 differs from the above-described embodiment in that the activity estimation model 2027 is re-learned for each patient. That is, in this modified example, the activity estimation model 2027 is customized for each patient.

[0272] Specifically, the administrator of the system 1 can use the patient's own stool image and the actual disease activity index at the time of excretion of stool corresponding to the stool image as learning data and cause the activity estimation model 2027 to relearn the learning data. Here, the actual disease activity index refers to the disease activity index obtained by medical examination by a doctor.

[0273] In this case, the activity estimation model 2027 is optimized uniquely for each patient, and the activity estimation model 2027 can learn the correspondence between the state of stool and the current disease activity for inflammatory bowel disease, a disease whose symptoms vary greatly from person to person. This allows for more accurate evaluation of disease activity.

[0274] <7. Other Modifications> Other modifications will be described below.

[0275] In the above embodiment, the configuration for photographing the feces discharged into the toilet has been shown, but this is not a limitation. For example, the system 1 may use a dedicated defecation sheet for photographing. The defecation sheet is used while floating in the sealed water in the toilet, and is a sheet that prevents feces from flowing into the sealed water during the time required for photographing. In other words, it is preferable that the defecation sheet has a certain degree of water resistance and is hydrophilic so that it can be washed away together with the excrement. In addition, it is preferable that at least a part of the surface of the sheet body constituting the defecation sheet is colored. By having a colored surface of the sheet body, it is easier to identify the white mucus contained in the excrement.

[0276] The sheet body of the defecation sheet may be marked with a color standard or a size standard. The color standard is a mark that serves as a color standard used in image processing of the stool image, and is used in image processing of the stool image. For example, the color standard can be used to detect blood in the stool, or to distinguish between mucus and watery stool.

[0277] In image processing using color criteria, feature quantities such as hue, saturation, and brightness of excrement-related objects contained in a stool image are extracted based on color criteria, and the type of object is determined. In addition, as a method using a color histogram, the type of object contained in a stool image may be determined by analyzing the color distribution of pixels in the stool image using color criteria.

[0278] The color of the color reference may be a color equivalent to the color of the excrement-related object, a color equivalent to its complementary color, or a color with contrasting brightness. Examples of detected objects and corresponding color reference candidates are shown below. When detecting blood (red) (reddish colors or complementary greenish colors) - When detecting stool (brown) (brownish color or its complementary blue-purple color) - When detecting mucus (white) (black is the contrast in brightness)

[0279] The size standard and color standard are indications that serve as size standards used in image processing of stool images, and are used in image processing of stool images. For example, the size standard can be used to detect the amount of stool and bleeding.

[0280] In image processing using a size standard, a trained model (detection model 172, activity estimation model 2027, or content estimation model) that has been trained on images of excrement with size standards in advance is used to accurately estimate the actual size and position of the object by comparing the size and position of the object contained in the stool image with the size and position of the size standard.

[0281] In other words, when performing image analysis using color or size standards written on such defecation sheets, learning may be performed by using images in which the color and size standards are depicted together with excrement as input data for the learning data of the trained model. In this way, by performing image processing on the stool image using the color and size standards written on the defecation sheet, it is expected that the accuracy of estimation by the activity estimation model 2027 can be improved.

[0282] In addition, in the system 1, the display mode of the evaluation result may be changed on the display screen of the evaluation result shown in Fig. 17 according to the value of the estimated disease activity index. That is, when the estimated disease activity index is worse than a threshold value, worse than the previous evaluation result, or a worsening tendency is observed during the evaluation period, a display mode that emphasizes this fact may be adopted. Examples of such change patterns of the display mode are listed below. Change the background color of the display screen Change the text color on the display screen Change the size of the text on the display screen - Emit an alert sound along with the display screen The display screen is displayed and the patient terminal 10A is vibrated.

[0283] Furthermore, the system 1 may receive input of information regarding the level of odor related to excrement from the patient. The odor information is subjectively determined by the patient on, for example, the following five-point scale, and input to the patient terminal 10A. Normal stool odor Strong (or strange) stool odor Extremely strong (or very strange) stool odor

[0284] When estimating a disease activity index using such stool odor, for example, a stool image and the degree of stool odor depicted in the stool image can be used as input data, and an estimation model obtained by machine learning using learning data can be used in which the value of the disease activity index is the correct output data for the input image information and odor degree information. In addition, a comprehensive assessment of the disease activity index may be made using a correction standard that corresponds the degree of odor to the expected disease activity index for the estimation results obtained by the above-mentioned activity estimation model 2027 or content estimation model.

[0285] In the above embodiment, the specific disease is described by taking inflammatory bowel disease as an example, but is not limited to this. The disease activity of the disease evaluated by the system 1 may be a disease other than inflammatory bowel disease.

[0286] In the above embodiment, the patient uses the patient terminal 10A to photograph the excrement excreted by the patient, but the present invention is not limited to this. For example, a medical professional at a medical institution or a staff member at a care facility may photograph the excrement excreted by the patient using the terminal device 10 that the medical professional uses to obtain a stool image.

[0287] For example, the disease activity of an inpatient can be evaluated by the system 1. In this case, a nurse or the patient who has been lent the medical terminal 10B by the nurse may take a stool image using the medical terminal 10B. In such an evaluation of disease activity for an inpatient, the doctor on duty may be notified of the evaluation result according to the evaluation result. Specifically, the evaluation result of disease activity is notified to the medical terminal 10B used by the doctor on duty or to the contact information of the doctor on duty.

[0288] For example, the doctor on duty may be informed of the disease activity assessment results in the following situations: 1) If the evaluation results of the disease activity index indicate a concern about the patient's condition If the disease activity index is worse than a certain threshold If the disease activity index is worse than the previous evaluation result - If the disease activity index shows a worsening trend during the evaluation period 2) When the estimated likelihood is high and the patient's condition is of concern

[0289] In addition, depending on the evaluation result of the disease activity by the system 1, that is, when the condition is of concern, a guide to the effect that it is necessary for a nurse to call a doctor on duty may be sent to the medical terminal 10B. This allows the nurse to smoothly make a decision on whether or not to immediately seek the doctor's opinion based on the evaluation result of the disease activity by the system 1 by checking the guide displayed on the medical terminal 10B.

[0290] Furthermore, the system 1 can automatically record the number of stool episodes and the time of defecation. That is, when a patient obtains a stool image, the time of photographing the stool image is regarded as the time of defecation, and the following information can be recorded over time simply by photographing an image each time the patient defecates. Number of flights per day Number of flights during the day Number of nighttime flights

[0291] In addition, taking into account the above-mentioned results of detection of excrement, the following information can be further recorded over time. · Number of regular flights Frequency and specific characteristics of diarrhea Frequency of bloody stool 100 bloody stools Number of times that blood in stools is more than 50% Fewer than 50% of stools showed blood Doctors can refer to the patient's bowel history, which is accumulated over time, and use it to determine the actual condition of the patient in their medical practice.

[0292] Furthermore, the medical terminal 10B may be able to access information on medical history recorded in the medical history DB 3022 shown in Fig. 9C from the evaluation result screen shown in Fig. 17. That is, according to the needs of a medical worker who has confirmed the evaluation result of disease activity by the system 1, past medical results and the results of various tests such as blood sampling actually performed in the most recent medical treatment can be displayed on the medical terminal 10B.

[0293] The evaluation result of the disease activity index by the system 1 stored in the evaluation history DB2026 shown in Fig. 6F may be accessible from the electronic medical record of the patient displayed on the medical terminal 10B. In this case, the electronic medical record mainly displays information on the medical history recorded in the medical history DB3022 shown in Fig. 9C. Then, the evaluation history of the disease activity index by the system 1 can be displayed on the medical terminal 10B according to the needs of a medical worker who has checked the contents of the electronic medical record.

[0294] In addition, in the system 1, an endoscope-equivalent image may be output to the patient terminal 10A. In general, the treatment of inflammatory bowel disease tends to take a long time. In order to continue the treatment, it is effective for the patient to intuitively understand the current condition. Therefore, by presenting the patient with an endoscope-equivalent image that allows the patient to visually understand the condition in addition to the estimated disease activity index, the patient can reconcile his / her subjective symptoms related to the disease with the estimated actual condition, and the patient can be motivated to continue the treatment.

[0295] In addition, the system 1 may generate an endoscope-equivalent video that allows confirmation of the chronological progression of the intestinal condition by synthesizing the endoscope-equivalent images generated at each of a plurality of evaluation periods in a chronological order and generating a video in which the images change over time in response to a display instruction. This allows medical personnel or patients to grasp the change in the condition due to treatment activities.

[0296] The system 1 may also provide information that gives certain suggestions to the medical treatment by the doctor. Specifically, the system 1 may use information on prescriptions and procedures recorded in the medical history to chronologically analyze the evaluation results of disease activity estimated, for example, before and after a change in medication or a surgery. This makes it possible to provide information that is useful for evaluating the non-inferiority of a newly prescribed medication and determining the timing of changing the treatment plan.

[0297] Although the embodiment of the present invention has been described in detail above, the scope of the present invention is not limited to the above embodiment. Furthermore, the above embodiment can be improved or modified in various ways without departing from the spirit of the present invention. Furthermore, the above embodiment and modified examples can be combined, or a part of them can be omitted.

[0298] <8. Notes> The matters described in the embodiment and the modified examples are supplemented below.

[0299] (Appendix 1) A program to be executed in an information processing system including a server having a processor, the program causing the processor to: A step of acquiring a stool image of the patient taken by a terminal device used by the patient or a medical worker (step S201); A step of estimating a disease activity index for a specific disease of the patient by image processing of the acquired stool image (step S202); and outputting the estimated disease activity index to a terminal device (step S203).

[0300] (Appendix 2) In the step of estimating a disease activity index (step S202), the processor The program described in Appendix 1 executes a step of inputting the acquired stool image into an activity estimation model created by machine learning using learning data in which the stool image is used as input data and the disease activity index is used as correct output data, thereby outputting an estimate of the disease activity index for a specific disease of the patient.

[0301] (Appendix 3) In the step of estimating a disease activity index (step S202), the processor A step of inputting the acquired stool image into an activity estimation model created by machine learning using learning data in which the stool image is used as input data and the calprocin content in the stool is used as correct output data, thereby outputting an estimate of the calprocin content in the stool; The program described in Appendix 1 executes a step of outputting an estimated value of the disease activity index using the estimated value of the content based on a determination criterion that corresponds a predetermined content to an estimated value of the disease activity index.

[0302] (Appendix 4) In the step of estimating the disease activity index (step S202), 2. The program of claim 1, which outputs an estimated location of the lesion area in the disease.

[0303] (Appendix 5) In the step of estimating the disease activity index (step S202), 2. The program of claim 1, which outputs the likelihood of the estimated disease activity index.

[0304] (Appendix 6) The processor further comprises: The program described in Appendix 1 executes a step of re-learning an activity estimation model used to estimate the disease activity index using stool images of a patient and the actual disease activity index at the time of excretion of stool corresponding to the stool images as learning data.

[0305] (Appendix 7) The processor further comprises: Executing a step of acquiring at least one of a behavior history indicating a history of excretory behavior and eating behavior including medication, and biological information inputted into the terminal device; The program described in Appendix 1, in which in the step of estimating a disease activity index (step S202), a disease activity index is estimated based on predetermined evaluation criteria using the acquired behavioral history and biometric information, along with image processing of the stool image.

[0306] (Appendix 8) A program described in Appendix 1, in which in the step of estimating a disease activity index (step S202), image processing is performed on a stool image using color or size standards written on the defecation sheet used by the patient when defecation.

[0307] (Appendix 9) In the step of acquiring a stool image (step S201), A step of detecting excrement contained in an image captured by the terminal device (step S102); and if excrement is detected in the image, transmitting the image to a server without storing it in the terminal device (step S103).

[0308] (Appendix 10) The processor further comprises: The program of claim 1, further comprising: a step of notifying a patient that medical treatment by a primary care physician is required when the estimated disease activity index exceeds a predetermined threshold.

[0309] (Appendix 11) The processor further comprises: A step of counting the frequency of stool image acquisition by patients (step S221); The program according to claim 1, further comprising: a step (step S222) of notifying the terminal device of a notification encouraging the continued acquisition of stool images when the collected acquisition frequency falls below a certain standard.

[0310] (Appendix 12) The processor further comprises: A step of counting the frequency of stool image acquisition by patients (step S221); The program according to claim 1, further comprising: a step (step S222) of notifying the terminal device of a notification authorizing continued acquisition of stool images when the collected acquisition frequency exceeds a certain standard.

[0311] (Appendix 13) The processor further comprises: A program as described in Appendix 1, which executes a step (step S204) of notifying a terminal used by the patient's physician when the estimated disease activity index exceeds a predetermined threshold value set in advance.

[0312] (Appendix 14) The processor further comprises: A step of generating an endoscope-equivalent image by image processing using the acquired stool image, the endoscope-equivalent image being estimated as a state of the patient's intestines when the patient excretes the stool related to the stool image (step S241); The program according to claim 1, which causes the computer to execute a step (step S242) of notifying a terminal used by the patient's doctor that an endoscopic equivalent image has been created.

[0313] (Appendix 15) A method executed by an information processing system including a server having a processor, the method comprising the steps of: A step of acquiring a stool image of the patient taken by a terminal device used by the patient or a medical worker (step S201); A step of estimating a disease activity index for a specific disease of the patient by image processing of the acquired stool image (step S202); A method which executes a step of outputting the estimated disease activity index to a terminal device (step S203).

[0314] (Appendix 16) An information processing system including a server having a processor, the information processing system including the processor: A means for acquiring a stool image of a patient taken by a terminal device used by the patient or a medical professional; A means for estimating a disease activity index for a specific disease of a patient by image processing of the acquired stool image; and means for outputting the estimated disease activity index to a terminal device.

[0315] (Appendix 17) A defecation sheet used for capturing a stool image when executing the program according to any one of appendixes 1 to 14, The defecation sheet is used to cover the sealed water in the toilet where the patient defecates, and has a water-resistant sheet body to prevent feces from flowing into the sealed water during the time required for imaging.

[0316] (Appendix 18) The defecation sheet according to claim 17, wherein at least a portion of the sheet body is colored and at least one of color standards and size standards used in image processing of the stool image is displayed on the surface of the sheet body. [Explanation of symbols]

[0317] 1. System 1 10 Terminal Equipment 12 Communication Interface 13 Input Devices 14 Output Devices 15 Memory 16 Memory section 19 Processors 20 Rating Server 20 20 Terminal Equipment 22 Communication Interface 23 Input Devices 24 Output Devices 25 Memory 26 Memory section 29 Processors 80 Network

Claims

[Claim 1] A program to be executed by an information processing system having a server with a processor, wherein the program is to be executed by the processor, The steps include: obtaining an image of the patient's stool taken by a terminal device used by the patient or healthcare worker; The steps include: estimating a disease activity index for a specific disease of the patient by image processing of the acquired stool image; A program that performs the steps of outputting the estimated disease activity index to the terminal device.

Images