Inspection guide device and computer-aided inspection guide method

Abstract

Provided are an examination guidance device and an examination guidance method with which it is possible to receive necessary advice for the sake of properly receiving an endoscopic examination, an examination that occurs in conjuction with medical practice, a clinical examination, or similar. This examination guidance method estimates (S39-S43) an endoscopic examination recipient's degree of progress through a preparatory step for the endoscopic examination, on the basis of temporal variation in condition information about the recipient, and generates (S49) information regarding the completion timing of the preparatory step on the basis of the estimated degree of progress. Here, the degree of progress through the preparatory step is the degree of progress of a bowel cleansing condition of the endoscopic examination recipient, and the timing at which the bowel cleansing condition is predicted to reach a fixed degree is treated as the completion timing of the preparatory step.

Description

[Technical field]

[0001] The present invention relates to an examination guide device that can provide appropriate guidance to subjects (who are to be distinguished from subjects who are to participate in research, etc., and are to be referred to as subjects as those who undergo the examination) who undergo examinations involving medical procedures such as endoscopic examinations or clinical examinations, until the actual examination. Computer-based The present invention relates to an examination guide method. [Background technology]

[0002] In medical facilities, a number of examination devices such as endoscopes are prepared, and an appropriate endoscope is prepared according to the examination items of the patient or examinee, and this endoscope is used for the examination. In this case, a schedule must be set to use the endoscope in good condition, taking into consideration the cleaning condition of the endoscope, as well as the state of wear and tear and deterioration. Therefore, Patent Document 1 proposes an endoscopic examination work support system that appropriately schedules endoscopic examination work. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2017-113082 A Summary of the Invention [Problem to be solved by the invention]

[0004] The endoscopic examination work support system of the above-mentioned Patent Document 1 describes a technology for appropriately scheduling endoscopic examination work in a medical facility, but does not describe support such as advice for subjects to undergo an appropriate examination in a medical facility. In actual medical settings, there are various things that subjects must do when undergoing an endoscopic examination, such as taking a laxative and performing digestive tract cleaning before undergoing an endoscopic examination, and the necessary treatment, the time to take the treatment such as a laxative, the examination time, etc. vary depending on the health condition and individual differences of the subjects. Therefore, it is difficult for subjects to undergo an examination with sufficient peace of mind unless appropriate advice is given, including the taking of treatment drugs including laxatives before the examination, physical condition management during the examination, the health condition and individual differences of the subjects, and other conditions of the subjects.

[0005] The present invention has been made in consideration of the above circumstances, and provides an examination guide device that allows a patient to receive necessary advice for appropriately undergoing an examination involving medical procedures such as an endoscopic examination or a clinical examination, and Computer-based An object of the present invention is to provide an inspection guidance method. [Means for solving the problem]

[0006] In order to achieve the above object, the first aspect of the present invention Computer-based The examination guide method includes: determining a progress level of a process related to preparation for an endoscopy examination of an endoscopy examinee based on a temporal change in condition information of the endoscopy examinee; Automatically by computer Estimation Steps to Based on the estimated progress, information regarding the completion timing of the preparation process is provided. Automatically by computer Generate Includes steps , Computer-based The examination guide method includes: a time change in the state information being a time change in an excretion image; and a time change in a degree of cleansing of the intestine in the future from a time when the excretion image was captured is calculated based on the time change in the excretion image. By computer The timing at which the estimated change in cleaning degree over time reaches a certain level is determined as By computer This is the timing at which intestinal cleansing is completed.

[0007] The second invention relates to Computer-based The examination guide method includes: determining a progress level of a process related to preparation for an endoscopy examination of an endoscopy examinee based on a temporal change in condition information of the endoscopy examinee; Automatically by computer Estimation Steps to Based on the estimated progress, information regarding the completion timing of the preparation process is provided. Automatically by computer Generate Includes steps , Computer-based A method for guiding an endoscopy, comprising: a step of guiding an endoscopy patient in a step of preparing for an endoscopy; a step of guiding an endoscopy patient in a step of preparing for an endoscopy; By computer The predicted timing is set as the end timing of the preparation process, and the excretion image in which the state in which the endoscopy examinee excretes and the multiple excretion images are recorded, and the excretion progress model and feature quantities are calculated. By computer By comparing, the change in the degree of cleaning over time is estimated, By computer The timing of completion of bowel cleansing is estimated.

[0008] Third of Related to the invention Computer-based The examination guide method according to the second aspect of the present invention further includes taking into account the time from the start of intestinal cleaning of the patient undergoing the endoscopy examination, and calculating the change in cleansing degree over time. By computer Estimate. The fourth aspect of the present invention Computer-based The examination guide method according to the second aspect of the present invention further comprises: Amount of of By computer Estimate, and this estimate The intestinal cleansing agent The amount is further taken into consideration to estimate the change in cleanliness over time.

[0010] The fifth invention relates to Computer-based The examination guide method, in the second aspect of the present invention, includes determining the end timing of the intestinal cleaning state by using an AI model trained using an excretion image in which the excretion state is captured, and a plurality of excretion images in which the state in which the endoscopy examination subject is excreted is captured. By computer Estimate. The sixth invention relates to Computer-basedThe examination guide method according to the second aspect of the present invention includes inputting an image of a state in which a subject is excreting into an AI model that has been trained using at least one of the time to completion of cleaning and the amount of cleaning agent in addition to the image of the excretion, By computer The timing of completion of bowel cleansing is estimated.

[0011] The seventh aspect of the present invention Computer-based The examination guidance method according to the second aspect of the present invention includes inputting an excretion image of an endoscopy patient excreting into an AI model trained using a plurality of learning data sets in which an excretion image of an endoscopy patient excreting is annotated with a cleanliness level. By computer The degree of cleansing is estimated, and the timing for completion of the intestinal cleansing state is estimated. The eighth aspect of the present invention Computer-based The examination guide method according to the second aspect of the present invention includes inputting a plurality of images of a subject excreting into an AI model trained using a plurality of learning data sets in which a plurality of images of the subject excreting are annotated with the cleaning time or the amount of detergent taken until the subject reaches an examination-ready state, By computer The timing for completing the cleaning is estimated.

[0013] An inspection guide device according to a ninth aspect of the present invention comprises: A status acquisition unit that acquires status information of an endoscopy patient; , a progress estimation unit that estimates a progress status of the examination preparation of the endoscopy examination examinee based on a temporal change in the state information, and an advice generation unit that displays advice regarding the examination preparation based on the progress status, The progress estimation unit estimates the progress of digestive tract cleansing and the time required to reach a certain level of cleansing, or the number and amount of cleaning agent intake, and the advice generation unit generates information based on the estimation results of the progress estimation unit. Effect of the Invention

[0014] According to the present invention, an examination guide device that allows a patient to receive necessary advice for appropriately undergoing an examination involving medical procedures such as an endoscopic examination or a clinical examination, and Computer-based A method for guiding an examination may be provided. [Brief description of the drawings]

[0015] [Figure 1]1 is a block diagram showing a configuration of an endoscopic examination support system according to a first embodiment of the present invention. [Diagram 2] 5 is a flowchart showing a main operation of a service server of the endoscopic examination support system according to the first embodiment of the present invention. [Figure 3A] 5 is a flowchart showing the operation of displaying an examination time guide in the service server of the endoscopic examination support system according to the first embodiment of the present invention. [Figure 3B] 13 is a flowchart showing a modified example of the operation of displaying a guide to an examination period in the service server of the endoscopic examination support system according to the first embodiment of the present invention. [Figure 4] 4 is a graph showing changes in cleanliness in the endoscopic examination support system according to the first embodiment of the present invention. [Diagram 5] 1A to 1C are diagrams showing a reference image sequence of defecation in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 6] FIG. 1 is a diagram for explaining a method for calculating the cleanliness level by AI in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 7] 4 is a graph showing a cleanliness curve when the cleanliness is predicted by an algorithm in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 8] FIG. 1 is a diagram showing an algorithm for predicting cleanliness by AI in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 9A] FIG. 13 is a diagram showing an alarm display screen when a subject starts to drink a cleaning agent in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 9B] FIG. 13 is a diagram showing an advice screen for a subject to take a cleaning agent in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 9C] FIG. 13 is a diagram showing an advice screen displayed from when a subject leaves his / her home until the examination is completed at a medical facility in the endoscopic examination support system according to the first embodiment of the present invention. [Figure 10]FIG. 11 is a block diagram showing the configuration of an endoscopic examination support system according to a second embodiment of the present invention. [Figure 11] 10 is a flowchart showing a main operation of a service server of the endoscopic examination support system according to the second embodiment of the present invention. [Figure 12] FIG. 13 is a diagram showing an advice screen displayed from when a subject arrives at a medical facility to when the examination is completed, in the endoscopic examination support system according to the second embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, as an embodiment of the present invention, an example in which the present invention is applied to an endoscopic examination support system will be described. In a colonoscopic examination, in order to examine the large intestine, a cleansing agent for the intestine is drunk and the large intestine is cleaned before the examination, so in this embodiment, a case in which the intestine is cleaned will be mainly described. Of course, the present invention is not limited to endoscopic examination, and can also be applied to other clinical examinations and examinations involving medical procedures. First, an endoscopic examination support system according to the first embodiment will be described with reference to Figs. 1 to 9C. Fig. 1 is a block diagram showing the overall configuration of the endoscopic examination support system according to the first embodiment. This endoscopic examination support system is composed of a service server 10, a user terminal 20, and in-hospital systems 30 and 34.

[0017] The service server 10 can be connected to a user terminal 20 used by a user or the like, and to in-hospital systems 30, 34 used by medical staff or the like in a medical facility and for acquiring information in the medical facility through a communication network such as the Internet, and can provide various services to the user or the like. The service server 10 has a processing device such as a CPU (Central Processing Unit), a memory storing a program, and other peripheral circuits, and also has a control unit 11, a communication unit 12, a schedule management unit 13, a cleanliness evaluation unit 14, a time prediction unit 15, an instruction transmission unit 16, and an examination result recording unit 17. The schedule management unit 13, the cleanliness evaluation unit 14, the time prediction unit 15, the instruction transmission unit 16, and the examination result recording unit 17 may be realized by a hardware circuit or the like, or may be realized by the control unit 11 executing a program stored in the memory. In this case, one or more processors constituting the control unit 11 perform the functions of the schedule management unit 13, the cleanliness evaluation unit 14, the time prediction unit 15, and the instruction transmission unit 16.

[0018] The control unit 11 controls the entire service server 10. The control unit 11 is composed of one or more processors having a processing device such as a CPU, a memory storing a program, etc., and can control each unit in the service server 10 by executing the program. The control unit 11 also functions as a usage status acquisition unit that acquires the usage status of the medical institution where the patient undergoes the endoscopic examination from the waiting room information acquisition unit 37, the examination room information acquisition unit 38, and the recovery room information acquisition unit 39 of the in-hospital system 34 (see S15, S17, etc. in FIG. 2). The usage status is the usage status (crowding status) of the space in the medical institution where each action related to at least one of waiting, pre-treatment, examination, recovery, and payment is performed.

[0019] The communication unit 12 has a communication circuit provided in the peripheral circuit of the service server 10, and can communicate with each communication unit in the user terminal 20 and the in-hospital system 30 and 34. For example, when the instruction issuing unit 16 creates advice, the instruction issuing unit 16 transmits the advice to the user terminal 20. In addition, the subject's request can be input through the communication unit 12. For example, when the subject wishes to be examined using a painkiller or a sedative, the subject inputs this to the user terminal 20 and transmits it to the service server 10 through the communication unit 12. In addition, the subject can input requests regarding transportation means and the like when moving from home to the medical facility. When the service server 10 functions as an examination guide device, the communication unit 12 functions as a patient request input unit that inputs requests from endoscopy patients.

[0020] The schedule management unit 13 performs various time management when a user (subject) undergoes an endoscopic examination. For example, the schedule management unit 13 performs time management such as management of the timing at which the user starts taking a cleaning agent for intestinal cleaning when undergoing an examination (see S3 in FIG. 2), the departure time when the user leaves for a medical facility such as a hospital to undergo an examination (see S11 and S13 in FIG. 2), and the predicted time of all steps of the examination (S19 in FIG. 2). In addition, the schedule management unit 13 performs schedule management taking into account the examination time predicted by the time prediction unit 15, etc. (see S15, S17, and S19 in FIG. 2).

[0021] The cleanliness determination unit 14 determines the cleanliness of the intestinal tract of the subject based on images of the subject's defecation, etc. The subject drinks the cleansing agent multiple times, and each time he or she defecates, the intestinal tract is gradually cleansed. The cleanliness determination unit 14 determines the degree of this cleanliness. The determination of the cleanliness based on defecation images will be described later with reference to Figures 4 to 8. When the service server 10 functions as an examination guide device, the cleanliness determination unit 14 functions as a status acquisition unit that acquires status information of the person undergoing an endoscopic examination (see, for example, S5 in Figure 2, Figures 3A and 3B, S14a in Figure 11, etc.). The status acquisition unit acquires information related to the excretion of the person undergoing an endoscopic examination.

[0022] The time prediction unit 15 predicts the time when the subject will undergo the test. For example, when the subject takes a cleansing agent at home to cleanse the inside of the intestine, the time when the cleansing will end is predicted based on the time change of the result of judging the cleansing level based on the image of the subject's defecation (see, for example, Figs. 4 and 8). In addition, the time prediction unit 15 performs time prediction based on the time change of the judgment result of the intestinal cleansing level performed by the cleansing judgment unit 14 (for example, the time change of the excretion image, see the reference image sequence in Fig. 5). In addition, the timing when the intestinal cleansing level is predicted to reach a certain level is set as the end timing (see, for example, S49 in Figs. 3A and 3B, and the testable level Lev in Fig. 4). Based on this prediction result, the amount of cleansing agent and the number of times the subject will drink until the cleansing is completed are also predicted.

[0023] In addition to cleaning, the time prediction unit 15 may also predict the time when preparation for the test will end, the time when the test will start, the time when the test will be completed, etc. For example, the time to start drinking and the time to leave may be predicted by calculating backwards from the time required for the test (see, for example, S3 and S11 in FIG. 2), and in a medical facility, the time required for the test, recovery, etc., the time required to hear the test results after the test is completed, the time required to wait for payment, etc. may be considered to predict the time required for the entire process of the test (see, for example, S19 in FIG. 2).

[0024] When the service server 10 functions as an examination guide device, the time prediction unit 15 functions as a progress estimation unit that estimates the progress of the examinee's examination preparation based on the temporal change in the status information (see, for example, S49 in Figs. 3A and 3B, etc.). The examination preparation is preparation related to digestive tract cleansing. In this embodiment, this corresponds to the examinee drinking a cleansing agent to cleanse the intestines. The progress estimation unit estimates the examinee's progress related to digestive tract cleansing as the progress. The progress estimation unit estimates the progress of digestive tract cleansing and the time required to reach a certain degree of cleansing, or the number and amount of times the cleansing agent is ingested (see, for example, S49 in Figs. 3A and 3B, etc.).

[0025] The time prediction unit 15 functions as a time estimation unit that estimates the time related to the endoscopic examination based on the status information and the usage situation (for example, see S19 in FIG. 2). The time estimation unit predicts the start or end time of each action related to at least one of waiting, pre-treatment, examination, recovery, and payment in the medical institution.

[0026] The instruction transmission unit 16 creates advice for the subject and transmits it from the service server 10 to the user terminal 20 or the like through the communication unit 12. The advice includes, for example, the time when cleaning will end and the amount and number of times of cleaning agent remaining based on the subject's defecation (see, for example, S49 in Fig. 3A and Fig. 3B). In addition to the above, there is also advice on departure time and cleaning time (see Fig. 9A to Fig. 9C). When the service server 10 functions as an examination guide device, the instruction transmission unit 16 functions as an advice display unit that displays advice related to examination preparation based on the progress status (see S49 in Fig. 3A and Fig. 3B, Fig. 9A to Fig. 9C, etc.). The advice display unit displays information based on the estimation result of the progress estimation unit (cleanliness, number and amount of cleaning agent intake, expected time of completion of cleaning, etc.). The advice display unit displays advice taking into account the request of the endoscopy examination patient.

[0027] When the service server 10 functions as an examination guide device, the instruction transmission unit 16 functions as an advice display unit that displays advice related to an endoscopic examination based on estimation. The advice display unit displays advice related to the start of travel to a medical institution, the start or end of each action, and the completion of all actions at the medical institution. The instruction transmission unit 16 functions as an advice generation unit (see, for example, S49 in Figs. 3A and 3B) that displays advice related to examination preparation based on the progress status. The advice generation unit generates information based on the estimation result of the progress estimation unit.

[0028] When an examination such as an endoscopic examination on a subject is completed at a hospital, the examination result recording unit 17 records the examination results in the recording unit.

[0029] In this embodiment, the in-hospital system 30 and the in-hospital system 34 are provided in the same hospital, and multiple in-hospital systems are also provided in other hospitals. One of the in-hospital systems 30 and 34 provided in the same hospital is a system connected to mobile terminals and PCs (personal computers) used by doctors, nurses, and staff in the administrative department and dispensing department, etc., for exchanging various information, and the other is a system connected to devices for acquiring information in patient waiting rooms, examination rooms, recovery rooms, etc., such as monitoring devices having imaging units, and mobile terminals and PCs used by staff in medical facilities, for exchanging various information. If there are three or more systems in the same hospital, it is of course possible to provide three or more in-hospital systems, or to combine them into one.

[0030] The control units 31, 35 in the in-hospital systems 30, 34 control the entire in-hospital systems 30, 34. The control units 31, 35 have a processing device such as a CPU, a memory storing a program, etc., and can execute the program and control each unit in each in-hospital system. The control units 31, 35 may be configured so that the in-hospital systems 30, 34 in the same hospital operate in cooperation with each other.

[0031] The communication units 32 and 36 have communication circuits and can communicate with the service server 10 and each communication unit in the other in-hospital systems 30 and 34 .

[0032] The schedule management unit 33 manages the schedules of users (doctors, nurses, pharmacists, laboratory technicians, office staff, etc.) of mobile terminals and PCs in the hospital system 30. This schedule management is performed in cooperation with the schedule management unit 13 in the service server 10, and in conjunction with the examination schedule of the subject (including before the examination, at the time of the examination, and after the examination).

[0033] The waiting room is a room where the subject waits until undergoing an endoscopic examination. The waiting room information acquisition unit 37 has an imaging unit such as a camera and an audio collection unit such as a microphone arranged in the waiting room, and acquires various information about the waiting room. The waiting room information acquisition unit 37 acquires information about the waiting room based on information collected by the imaging unit and audio collection unit. If the waiting room information acquisition unit 37 can directly communicate with the user terminal 20, it can detect that the subject having the user terminal 20 is in the waiting room. Furthermore, it can acquire various information such as what kind of examination the subject has come to take, what the schedule is, and the subject's profile. In addition, a nurse or the like may directly input information into the in-hospital system 34 as text or images from an in-hospital terminal.

[0034] The examination room is a room where the subject undergoes an endoscopic examination. The examination room information acquisition unit 38 has an imaging unit such as a camera and an audio collection unit such as a microphone arranged in the examination room, and acquires various information related to the endoscopic examination that the subject undergoes in the examination room. The examination room information acquisition unit 38 acquires information on the examination room based on the information collected by the imaging unit and the audio collection unit. If the examination room information acquisition unit 38 can directly communicate with the user terminal 20, it can detect that the subject having the user terminal 20 is in the examination room. Furthermore, it is also possible to acquire information such as the progress of the subject's examination and the examination results. In addition, a nurse or the like may directly input information to the in-hospital system 34, or input text or images from an in-hospital terminal. In many cases, multiple examination rooms are installed. Therefore, it is preferable that the examination room information acquisition unit 38 acquires the information for each examination room.

[0035] The recovery room is a room where the subject rests after the endoscopy of the subject is completed until the effects of the sedative or other drugs wear off and the subject returns to a normal state. The recovery room information acquisition unit 39 has an imaging unit such as a camera and an audio collection unit such as a microphone arranged in the recovery room, and acquires various information regarding the subject's condition in the recovery room. Based on the information collected by the imaging unit and the audio collection unit, the recovery room information acquisition unit 39 acquires information in the recovery room. If the recovery room information acquisition unit 39 can also directly communicate with the user terminal 20, it can detect that the subject having the user terminal 20 is in the recovery room. Furthermore, information regarding the subject's resting state can be acquired. Also, a nurse or the like may directly input information into the in-hospital system 34 by text input or image input from an in-hospital terminal.

[0036] The user terminal 20 may be a PC used by the subject, but in this embodiment, a mobile terminal such as a smartphone is assumed for explanation. Since the subject carries a mobile terminal, it is easy to collect information on lifestyle habits and to photograph the state of the subject's defecation. The user terminal 20 has a processing device such as a CPU (Central Processing Unit), a memory storing a program, and other peripheral circuits, and the user terminal 20 is provided with a control unit 21, a communication unit 22, an imaging unit 23, and a UI (User Interface) unit 24. Each unit in the user terminal 20 may be realized by a hardware circuit or the like, and some functions may be realized by the control unit 21 executing a program stored in the memory.

[0037] The control unit 21 controls the entire user terminal 20. The control unit 21 is composed of one or more processors having a processing device such as a CPU, a memory storing a program, and the like, and can execute a program and control each unit in the user terminal 20. In this embodiment, schedule management for an endoscopic examination, judgment of the cleanliness of defecation, time prediction, and issuance of advice to the subject are performed by the schedule management unit 13, cleanliness judgment unit 14, time prediction unit 15, and instruction transmission unit 16 in the service server 10.

[0038] However, the control unit 21 may execute these functions that are performed in the service server 10. In this case, an inference engine may be provided in the control unit 21 to perform inferences regarding the cleansing level, etc. In this case, the user terminal 20 functions as an examination guide device, and the control unit 21 functions as a progress estimation unit that estimates the progress of the examinee's examination preparation based on temporal changes in the status information. The progress estimation unit estimates the progress of the digestive tract cleansing and the time required to reach a certain cleansing level, or the number and amount of times the cleansing agent is ingested (for example, see S49 in Figs. 3A and 3B).

[0039] When the user terminal 20 functions as an examination guide device, the control unit 21 functions as a usage status acquisition unit that acquires usage status at a medical institution where a patient undergoes an endoscopic examination. When the user terminal 20 functions as an examination guide device, the control unit 21 functions as a time estimation unit that estimates a time related to an endoscopic examination based on state information and usage status.

[0040] The communication unit 22 has a communication circuit provided in the peripheral circuit of the user terminal 20, and can communicate with the communication unit 12 in the service server 10. Through the communication unit 22, it is possible to exchange various information with the service server 10, such as schedule management, cleaning degree judgment, time prediction, and advice to the subject.

[0041] The photographing unit 23 has an image sensor, a photographing lens, an image sensor, an image sensor circuit, etc., and converts an object into image data and outputs it. In this embodiment, the photographing unit 23 is used to acquire an image of the subject's defecation. When the mobile terminal 20 functions as an examination guide device, the photographing unit 23 functions as a status acquisition unit that acquires status information of the subject undergoing an endoscopic examination (for example, see S39 in Figs. 3A and 3B). The status acquisition unit acquires information related to the subject's excretion.

[0042] The UI unit 24 is a user interface for inputting information to the user terminal 20 and outputting information. The UI unit 24 has a visual (including auditory, etc.) display unit for conveying information to the subject, and an input unit (including, for example, a text input unit, a voice input unit, etc.) for the subject to input information to the user terminal 20. This input unit can also input requests from the subject undergoing an endoscopic examination, such as whether or not the subject wishes to undergo an examination using a sedative. In addition, the subject may be able to input requests regarding transportation, etc., when traveling from his or her home to a medical facility.

[0043] When the user terminal 20 functions as an examination guide device, the UI unit 24 functions as an advice display unit that displays advice related to examination preparation based on the progress status. The advice display unit displays information based on the estimation result of the progress estimation unit (cleanliness, number of times and amount of detergent intake, expected time of cleaning completion, etc.). When the user terminal 20 functions as an examination guide device, the UI unit 24 functions as a patient request input unit that inputs requests of an endoscopy examination patient. The advice display unit displays advice taking into consideration the requests of an endoscopy examination patient. When the user terminal 20 functions as an examination guide device, the UI unit 24 functions as an advice display unit that displays advice related to an endoscopy examination based on estimation.

[0044] By constructing such an endoscopic examination support system, a user (subject) can undergo an endoscopic examination with peace of mind. For example, during an endoscopic examination, the subject may have many questions, such as whether or not further doses are necessary after starting to take a cleaning agent, what time to go to the hospital, what time the examination will start, and what time the examination will end. In fact, some people finish cleaning quickly after taking a cleaning agent, while others take a long time to clean. In such a case, the subject can undergo the examination with peace of mind by using a "peace of mind app" (an application executed in the user terminal 20 in parallel with the main flow shown in FIG. 2 described later) installed in the user terminal 20. In other words, by using this peace of mind app, the subject can receive necessary advice for undergoing the examination again.

[0045] Next, the main operation of the application software for the user terminal 20 and the service server 10 to cooperate with each other to assist the subject (user) in the endoscopic examination will be described with reference to the flowchart shown in FIG. 2. The main operation of this application is realized by the control unit 11 in the service server 10 cooperating with the control unit 21 in the user terminal 20, controlling each unit in the service server 10, and further cooperating with the in-hospital systems 20 and 35. The flow shown in FIG. 2 is described on the premise that, when a safety application for examination assistance is installed in the user terminal 20, the service server 10 takes the lead in providing advice to the user terminal 20 for the subject. In this case, the service server 10 functions as an examination guide device. However, it is of course possible for the user terminal 20 to take the lead in providing advice to the subject with the support of the service server 10. In this case, the user terminal 20 functions as an examination guide device.

[0046] When the main operation shown in FIG. 2 starts, first, the expected examination start time is acquired (S1). Here, the control unit 11 inquires of the schedule management unit 13, acquires information on the reservation date and time of the endoscopic examination at the medical facility by the subject, and the expected examination start time is known based on this reservation date and time. If there is no reservation at the medical facility, a reservation is made and the expected examination start time is also acquired. This expected examination start time is acquired when the subject starts the safety application on the user terminal 20 and notifies the service server 10 that the examination will be performed. Note that, when the application software of FIG. 2 is executed independently on the user terminal 20, in this step, the user terminal 20 inquires of the schedule management unit 13 in the service server 10 about the expected examination start time and acquires the expected examination start time.

[0047] Next, a start-drinking alarm is issued (S3). Here, the control unit 11 judges whether it is time to start taking the detergent based on the expected test start time acquired in step S1, and when it is time to start taking the detergent, it instructs the user terminal 20 from the instruction transmission unit 16 to display an alarm to start taking the detergent on the UI unit 24 for the subject. The detergent is taken multiple times, and the start-drinking time is determined taking into consideration the standard time until the degree of cleanliness that can be tested is reached. However, since the time until the degree of cleanliness that can be tested varies depending on the subject, a little leeway may be provided from the standard time, and if there is a past record, the information may be taken into consideration when determining the time (see FIG. 4, FIG. 8, etc.). An example of the display of this start-drinking alarm will be described later with reference to FIG. 9A. Note that, in the case where the application software in FIG. 2 is executed proactively by the user terminal 20, in this step, the control unit 21 of the user terminal 20 notifies the subject of the start-drinking alarm when the start-drinking time acquired from the schedule management unit 13 is reached.

[0048] Next, the cleanliness is checked (S5). Here, the control unit 11 instructs the user terminal 20 through the instruction transmission unit 16 to acquire an image of the subject's defecation by the photographing unit 23. When the subject photographs the defecation image, it is transmitted to the service server 10 through the communication unit 22. The cleanliness determination unit 14 in the service server 10 determines the cleanliness based on the defecation image of the subject and determines whether or not a sufficient cleanliness level has been reached. By determining the cleanliness, it is possible to predict when the subject will complete the cleanliness (see FIG. 4, FIG. 8, etc.). The operation of this cleanliness check will be described later with reference to FIG. 3A or FIG. 3B. Also, during the cleansing, for example, an advice display as shown in FIG. 9B is displayed on the UI unit 24 of the user terminal 20. This display will be described later. Note that, when the application software of FIG. 2 is executed proactively on the user terminal 20, in this step, the defecation image of the subject is transmitted to the service server 10 and a determination of the cleanliness is requested.

[0049] When a sufficient cleansing level is reached, the current location is then acquired (S7). In this embodiment, it is assumed that the subject takes a cleansing agent at home or the like (a place other than a medical facility) to perform intestinal cleansing. Therefore, after the intestinal cleansing is completed, the subject moves to a medical facility such as a hospital. In this step, in calculating the time required to move to the medical facility, the control unit 21 first instructs the user terminal 20 to transmit the current location. Information regarding the current location is acquired by a position detection unit such as a GPS in the user terminal 20. For this reason, the user terminal 20 has a position detection unit such as a GPS. Note that, when the application software in FIG. 2 is executed proactively by the user terminal 20, in this step, information regarding the current location is acquired by a position detection unit in the user terminal 20.

[0050] After acquiring the current location information, the control unit 11 next acquires the estimated examination start time (S9). Although the estimated examination start time is acquired in step S1 as well, the control unit 11 again inquires of the schedule management unit 33 in the hospital system 30 about the estimated examination start time, taking into consideration subsequent changes in the situation. That is, in the medical facility, the examination may not proceed as scheduled or may proceed earlier than scheduled, and the examination start time is predicted taking into consideration the progress of the examination in the medical facility. Note that, in the case where the application software in FIG. 2 is executed independently in the user terminal 20, in this step, the control unit 21 acquires the estimated examination start time again through the service server 10 and the hospital system 30. The control unit 21 may collect necessary information from the service server 10 and predict the examination start time.

[0051] When the estimated examination start time is acquired, it is next determined whether it is the departure time (S11). Here, the control unit 11 calculates the movement start time to the medical facility based on the estimated examination start time acquired in step S9 and the information on the current location acquired in step S7, and determines whether the current time has reached the calculated movement start time (departure time). The movement start time is calculated taking into consideration the movement time from the current location to the medical facility. If the subject has a specified means of transportation, the movement time for this means of transportation may be calculated, and multiple means of transportation may be selected. Furthermore, the movement time may be calculated taking into consideration the traffic congestion situation on the day (for example, road congestion situation, train operation situation). The calculation of the movement time may be linked to another service that provides guidance on travel routes and means of transportation.

[0052] If the result of the determination in step S11 is that it is not time to depart, the process returns to step S7 and the above-mentioned operations are repeated. The progress of the examination in the medical facility, the situation of the means of transportation, etc. may change, and the departure time is predicted taking these changes into consideration. When the application software in FIG. 2 is executed proactively by the user terminal 20, in this step the control unit 21 inquires of the schedule management unit 13 in the service server 10 about the departure time and determines whether the current time is the departure time or not. The control unit 21 may collect necessary information from the service server 10 and the in-hospital systems 30, 34 and determine whether it is time to depart or not.

[0053] If the result of the determination in step S11 is that it is time to leave, a departure alarm is notified (S13). When the departure time arrives, the instruction transmission unit 16 in the service server 10 instructs the user terminal 20 to display a departure alarm on the UI unit 24. In this case, the instruction transmission unit 16 may notify the user terminal 20 of the departure time in advance, and when the user terminal 20 detects that the departure time has arrived, the departure alarm may be notified. The departure alarm may be made visually or audibly. In addition, a recommended means of transportation and a travel route may be displayed in accordance with the departure alarm. For example, in a case where a person travels by train immediately after the completion of intestinal cleansing, a route with many stations close to the platform and the toilet may be guided. In addition, since a subject may feel more comfortable if a train with relatively many stops (such as a local train) is guided rather than a train with few stops, an appropriate travel route may be guided after confirming whether the subject wants to go to the toilet. When the departure alarm is notified, the subject starts traveling from his / her home to a medical facility. It should be noted that, in the case where the application software of FIG. 2 is executed proactively on the user terminal 20, in this step, when the control unit 21 determines that it is time to depart, a departure alarm is displayed on the UI unit 24.

[0054] After the departure alarm is notified to the subject, the expected examination end time is obtained (S15). When the subject arrives at the medical facility, the endoscopic examination begins as soon as preparations are complete. The subject may arrive at the medical facility earlier than expected, and the examination may not start as scheduled because the examination of the subject in front of the subject takes time. Also, when the subject is examined, if insertion is difficult, the examination may take longer than scheduled. Also, if a polyp is found during the examination, the examination time may be extended due to polyp removal. Therefore, in this step, the time prediction unit 15 predicts the end time of the examination taking into account various situation changes. The time prediction unit 15 predicts the time when the endoscopic examination of the subject will end while cooperating with the in-hospital systems 30 and 34.

[0055] In step S15, the time prediction unit 15 may infer the possibility of the presence of polyps based on the subject's profile, lifestyle, previous examination results, etc. Also, the time prediction unit 15 may infer the possibility of lesions that require treatment such as resection or biopsy other than polyps. That is, here, the possibility of treatment related to the risk of fluctuating the examination time in the endoscopic examination is predicted, and the expected examination completion time is estimated based on the result.

[0056] 2 is executed proactively on the user terminal 20, the control unit 21 inquires of and acquires the predicted examination end time from the time prediction unit 15 in the service server 10 in step S15. The control unit 21 may collect necessary information from the service server 10 and predict the examination end time.

[0057] Next, the recovery completion expected time is acquired (S17). If a sedative is used in the examination, when the examination is completed, the subject rests in the recovery room until the sedative wears off. Information such as the congestion state of the recovery room can be acquired by the recovery room information acquisition unit 39. In this step, the control unit 11 queries the recovery room information acquisition unit 39 to acquire information such as the congestion state of the recovery room, and the time prediction unit 15 predicts the recovery completion time based on the information and the subject's physical condition, etc., and acquires the recovery completion expected time. Note that, in the case where the application software in FIG. 2 is executed independently on the user terminal 20, in this step, the control unit 21 acquires the recovery completion expected time from the time prediction unit 15 in the service server 10. The control unit 21 may collect necessary information from the service server 10 and predict the recovery completion time.

[0058] Next, the estimated time of completion of the entire process of the examination is displayed (S19). Here, the control unit 11 makes the time prediction unit 15 predict the time when the entire process of the examination will be completed based on the estimated time acquired in steps S15 and S17, and acquires this estimated time. Since it takes time for waiting time, examination time, recovery time, explanation time of the examination result, time to receive medicine if any, and accounting time until the examination is completed, the time prediction unit 15 predicts the completion time of the entire process of the examination taking these times into consideration. The completion time of the entire process of the examination may be predicted taking into consideration the condition information of the subject of the endoscopic examination, the congestion state of the hospital where the endoscopic examination is performed, and the operating status of the staff. When the time of each process of the examination is predicted, it is transmitted to the user terminal 20 via the estimated time instruction transmission unit 16 and displayed on the UI unit 24. An example of the display of the estimated time of the entire process will be described later with reference to FIG. 9C. 2 is executed proactively on the user terminal 20, in this step the control unit 21 causes the time prediction unit 15 in the service server 10 to predict the completion time, obtains this predicted time, and displays it on the UI unit 24. The control unit 21 may also collect necessary information from the service server 10 to predict the completion time.

[0059] After displaying the estimated time of completion of the entire examination process, it is next determined whether the subject has left the hospital (S21). Here, the user terminal 20 determines whether the subject has left the medical facility using a positioning system such as GPS. If the result of this determination is that the subject is still in the medical facility, the process returns to step S15, and the above-mentioned operation is performed to update the estimated time display of the entire examination process. That is, since the end time of the examination and the estimated recovery completion time at the medical facility change depending on the situation, in this step, the completion time of the entire examination process is predicted and displayed taking into account the change in situation. On the other hand, if the result of the determination in step S21 is that the subject has left the hospital, this main operation flow is terminated.

[0060] In step S21, other than leaving the hospital, it may be determined that the subject has gone to a parking lot, gotten in a car, started using public transportation, returned home, etc., as long as it is possible to determine the end of support by the peace of mind app. In addition, although the process returns to step S15 in FIG. 2, if the examination has already ended, the process may return to step S17, and if recovery has already been completed, the process may return to step S19.

[0061] In addition, when the application software in FIG. 2 is executed independently by the service server 10, in step S21, the control unit 11 in the service server 10 requests the user terminal 20 to transmit the current location obtained by a positioning system such as a GPS in the user terminal 20 to the service server 10.

[0062] In this way, in the flow of the main operation shown in FIG. 2, the expected start time of the examination such as an endoscopy is acquired, and based on this start time, the subject can be advised on the timing to start taking the cleaning agent (see S1, S3). In addition, after the cleaning is completed, the expected start time of the examination according to the subsequent change in the situation is acquired, and the departure time is calculated taking into consideration the time required for travel from the current location to the medical facility, and a departure alarm is notified when this departure time is reached (see S7 to S13). In addition, since it takes time such as waiting time, examination time, recovery time, explanation time of the examination results, and accounting time until the examination is completed at the medical facility, the completion time of the entire examination process is predicted and displayed taking these times into consideration (see S15 to S19). Therefore, the subject can predict how long the examination will take, and can undergo the examination with peace of mind.

[0063] In FIG. 2, the examination end time and recovery completion time are predicted based on information from the examination room information acquisition unit 37 and the recovery room information acquisition unit 39 (see S15, S17). However, without being limited to this, the start and end times of each procedure may be predicted and advice displayed using information on rooms (not limited to rooms, but may also be places (spaces)) within the medical facility, such as waiting rooms, pre-treatment rooms, rooms for notifying examination results (examination rooms, etc.), accounting rooms, etc., as well as the schedule of the endoscopist performing the examination, and other information.

[0064] Next, the operation of the cleanliness check in step S5 (see FIG. 2) will be described with reference to the flow chart shown in FIG. 3A. As described above, in this embodiment, the subject takes the cleansing agent at home or the like (excluding medical facilities) and performs intestinal cleansing. In this cleanliness check flow, advice necessary for the subject to take the cleansing agent is provided, and further, based on the defecation image, it is determined whether or not sufficient cleansing has been performed.

[0065] When the cleanliness check flow shown in FIG. 3A starts, first, it is determined whether or not cleaning is necessary and it is time to drink (S21). After the start drinking alarm is displayed in step S3 (FIG. 2), the subject must take the cleanser at a predetermined time interval. In this step, the control unit 11 determines whether or not the cleanser must be taken and a predetermined time has passed since the last time it was taken. To this end, the control unit 11 obtains information on the subject's cleanser intake status from the user terminal 20 and makes a determination. Note that, when the application software in FIG. 2 is executed independently in the service server 10, the control unit 21 determines in step S21 whether or not the subject has taken the cleanser based on input information, etc., to the UI unit 24 of the subject.

[0066] Next, the ease of cleaning is inferred based on the subject's profile and lifestyle (S23). When the subject drinks a cleaning agent to cleanse the intestinal tract, the ease of cleaning varies from person to person, and it is preferable to display an appropriate guide when drinking the cleaning agent according to the ease of cleaning. Therefore, in this step, the ease of cleaning of the subject is inferred. The user's profile is recorded in the recording unit in the user terminal 20 or the test result recording unit 17 of the service server 10. In addition, since the user usually carries the user terminal 20, the user's behavior, for example, daily habits such as what time the user wakes up, what time the user eats breakfast, lunch, and dinner, and what time the user goes to bed, can be known, and information such as whether the user walked, jogged, or exercised at the gym can be obtained. Furthermore, if the contents of meals are photographed, it is also possible to know what the user ate. Furthermore, the test result recording unit 17 also records the results of various medical examinations of the subject. Using this information, an inference engine provided in the service server 10 infers the ease of cleaning by inputting the above information.

[0067] If it is determined in step S21 that it is time to wash and drink, the amount of detergent is displayed (S25). Since the subject may not know how much detergent to drink, in this step, the control unit 11 transmits a previously stored amount of detergent to the user terminal 20 through the instruction transmission unit 16 according to the detergent used by the subject for washing, and causes the UI unit 24 to display it. At this time, the amount of detergent to be drunk in a certain period of time may be adjusted according to the ease of washing. This display will be described later with reference to FIG. 9B. Note that, when the application software in FIG. 2 is executed independently by the service server 10, the control unit 21 causes the UI unit 24 to display the amount of detergent transmitted from the instruction transmission unit 16. Alternatively, the control unit 21 may store in advance the amount of detergent to be drunk each time, and display the amount of detergent on the UI unit 24 based on the stored amount of detergent.

[0068] 2 is executed autonomously by the service server 10, the control unit 11 may collect the above-mentioned information in steps S23 and S25 and infer the ease of cleaning based on this information. Also, the possibility of polyps (possibility of treatment related to the risk of fluctuating the examination time) may be inferred. Also, if an inference engine is provided in the user terminal 20, this inference engine may be used in step S23. In this embodiment, the inference engine is used to infer the ease of cleaning (which may be the presence or absence of polyps), but the ease of cleaning (which may be the presence or absence of polyps) may also be predicted (determined) logically.

[0069] Next, it is determined whether or not the detergent is difficult to drink (S27). As described above, the ease of drinking a detergent varies from person to person, and some subjects complain that the detergent is difficult to drink. If the subject finds the detergent difficult to drink, the subject can express his / her intention by clicking a give-up icon on the screen of the UI unit 24 of the user terminal 20. In this step, the control unit 11 confirms the subject's intention expressed on the user terminal 20. Note that, when the application software of FIG. 2 is executed independently on the service server 10, the control unit 11 may confirm the subject's intention expressed on the UI unit 24. The user terminal 20 may be provided with a give-up button or the like in addition to the give-up icon, so that the subject can operate the give-up button or the like.

[0070] If the result of the determination in step S27 is that it is difficult to drink, the hospital checks the image via the server and inputs whether or not it is necessary to drink any more cleansing agent (S29). To perform an endoscopic examination, it is necessary to make sure that there is no stool remaining in the intestines, and this can be seen by looking at the image of the subject's defecation. In this step, the subject's defecation is photographed by the photographing unit 23 of the user terminal 20, and this image is sent to the in-hospital system 30 or 34 via the service server 10 (see screen D4 in FIG. 9B). The doctor in charge, etc., looks at the transmitted image and determines whether or not the subject has been sufficiently cleansed, and judges whether or not it is necessary to drink any more cleansing agent. At this time, it is possible to have a dialogue by voice or the like so that the subject and the hospital can communicate with each other.

[0071] Next, based on the image judgment in step S29, it is judged whether the hospital judged that the endoscopy was sufficiently cleaned (S31). In this step, the judgment is made based on the judgment of the doctor in charge of the endoscopy. The result of this judgment is transmitted to the control unit 21, and if it is judged that the endoscopy was sufficiently cleaned, the process proceeds to step S47, where the completion of cleaning is displayed. On the other hand, if it is judged that the endoscopy was not sufficiently cleaned, the hospital's judgment result is displayed (S33). Here, the control unit 11 transmits the judgment result from the in-hospital system 30 or 34 to the user terminal 20, and causes the UI unit 24 of the user terminal 20 to display the hospital's judgment result, that is, the need to drink a cleaning agent. In addition, if the hospital judges that an abnormal situation has occurred, it may display a message to visit the hospital immediately. Note that, when the application software in FIG. 2 is executed independently on the user terminal 20, the control unit 11 obtains the judgment result of the doctor in charge of the hospital through the service server 10, and displays the judgment result on the UI unit 24.

[0072] If it is not difficult to drink in step S27, or if the hospital's decision is displayed in step S33, the process waits until the cleansing agent is completely drunk (S35). Here, the process waits until the subject drinks the cleansing agent amount displayed in step S23. When the subject drinks the cleansing agent, this can be input to the user terminal 20. During the waiting period until the cleansing agent is completely drunk, the subject may defecate. In this case, although not shown in FIG. 3A, the process proceeds to step S39 and the process from S39 onwards is executed. When the cleansing agent is completely drunk, the process returns to step S21 and a determination is made as to whether it is time to drink the next cleansing agent. When it is time to drink the cleansing agent, the above-mentioned steps S23 and onwards are executed.

[0073] If the result of the determination in step S21 is that a cleanser is necessary but it is not time to drink it, then it is determined whether or not a bowel movement has occurred (S37). Drinking the cleanser may cause the subject to defecate. If a bowel movement has occurred, this is input to the UI unit 24 of the user terminal 20 (see screen D4 in FIG. 9B). If no defecation has occurred, the system is in standby mode and the process returns to step S21. Here, the control unit 11 makes the determination based on information from the user terminal 20.

[0074] On the other hand, if defecation has occurred, an image is captured (S39). In this case, the control unit 11 instructs the subject to capture an image of the defecation using the imaging unit 23 of the user terminal 20. At this time, the control unit 21 displays an advice display on the UI unit 24 for the subject to capture an image (see screen D4 in FIG. 9B). Note that when the application software in FIG. 2 is executed proactively on the user terminal 20, the control unit 11 instructs the subject to capture an image of the defecation.

[0075] The image capturing in step S39 is not limited to capturing an image, and if a sensor such as a camera is connected to the toilet bowl, the image and information related to defecation may be obtained from the sensor. Also, various information other than optical physical quantities, such as the liquid property of the water in the flush toilet, electrical resistance, and ultrasonic reflectance, may be obtained in step S39, and the degree of cleansing of the intestinal tract may be determined using the changes over time.

[0076] Once an image is captured, it is transmitted (S41). Here, in order to determine the level of cleanliness, the control unit 11 instructs the user terminal 20 to transmit the captured image to the service server 10. If the application software in FIG. 2 is executed autonomously on the user terminal 20, then once a defecation image is captured, the control unit 11 transmits the image. If the user terminal is capable of performing a cleanliness check, the image may be determined within the user terminal 20 without being transmitted to the service server 10.

[0077] Next, the cleansing level is obtained (S43). In the service server 10 that has obtained the image of the defecation, the cleansing level determination section 14 determines the cleansing level of the intestinal tract of the subject. As the cleansing of the intestinal tract progresses, the defecation image becomes clearer, so the cleansing level can be obtained by analyzing the image. How to determine the cleansing level will be described later with reference to Figs. 4 to 8. When the application software in Fig. 2 is executed proactively by the user terminal 20, the control section 21 obtains the cleansing level determination result from the cleansing level determination section 14.

[0078] Once the cleaning level has been acquired, it is next determined whether or not sufficient cleaning has been achieved (S45). Here, the control unit 11 acquires the latest cleaning level judgment result from the cleanliness judgment unit 14 and makes a judgment based on this. Note that, when the application software of Fig. 2 is executed independently on the user terminal 20, the control unit 21 may make a judgment based on the latest cleaning level judgment result from the cleanliness judgment unit 14, or may acquire the judgment level in the cleanliness judgment unit 14.

[0079] If the result of the determination in step S45 is that the cleaning level is not sufficient, the expected time to finish cleaning and the remaining amount (number of times) to be drunk are displayed (S49). Here, the time prediction unit 15 predicts the amount (number of times) of cleaning agent required to reach the level at which cleaning is completed and the end time. Alternatively, the time prediction unit 15 queries the schedule management unit 13 or the cleanliness prediction unit 14 in the service server 10 to obtain the prediction result. The UI unit 24 is configured to display the amount of cleaning agent that the subject drinks each time (see, for example, screens D2a to D2c in FIG. 9B), and considering the measurement result of the cleanliness of the subject, it is possible to estimate the timing at which the cleaning level of the subject reaches the testable level Lev (see, for example, FIG. 4). If the end timing can be predicted, the amount (number of times) of cleaning agent to be drunk by then can be predicted. This prediction of the end timing may be a classical logical method (see, for example, FIG. 4, FIG. 7, etc.), or may be a method of inference using artificial intelligence AI (see, for example, FIG. 6, FIG. 8, etc.).

[0080] When the predicted results of the amount (number of times) of detergent and the end time are acquired, the instruction transmission unit 16 transmits the predicted results to the user terminal 20 and causes the UI unit 24 to display them. When this display is performed, a message of encouragement to the subject, such as "Good luck," may also be displayed. After the predicted results of the amount (number of times) of detergent and the end time are displayed on the UI unit 24, the process returns to step S21.

[0081] In addition, in step S49, when the application software of FIG. 2 is executed proactively on the user terminal 20, the control unit 21 collects necessary information, predicts the amount of cleaning agent (number of times) required to reach the cleaning completion level and the end time, and displays this prediction result on the UI unit 24.

[0082] On the other hand, if it is determined in step S45 that sufficient cleaning has been performed, or if the hospital determines in step S31 that sufficient cleaning has been performed, a cleaning completion display is performed (S47). Here, the instruction transmission unit 16 causes the UI unit 24 of the user terminal 20 to display that cleaning has been completed (see screen D6 in FIG. 9B). Since cleaning has been completed, the subject can move to the hospital and undergo an endoscopic examination there (see S11 and S13 in FIG. 2). When the cleaning completion display is performed, the cleanliness check flow ends and the original flow is returned to.

[0083] Next, a modified example of the cleanliness check will be described with reference to the flowchart shown in FIG. 3B. In the flowchart shown in FIG. 3A, if the result of the determination in step S21 is that it is time to drink the cleanser, a series of steps from step S23 onwards are executed, and if it is not time to drink, it is determined whether or not there has been a defecation. In contrast, in the flow shown in FIG. 3B, if the result of the determination in step S21 is that it is time to drink the cleanser, the amount of cleanser is displayed and then the process returns to step S21, while if it is not time to drink the cleanser, a series of steps from step S27 onwards are executed. That is, the flow is the same as that of FIG. 3A, except that the destination when the result of the determination in step S21 is No, the destination after executing step S23, and the destination after executing step S33 are different. The operation in each step is the same as that of FIG. 3A, and therefore a detailed description will be omitted.

[0084] 3A and 3B, when the subject drinks the cleansing agent and performs intestinal cleansing, a defecation image is acquired (see S39, S41), the cleansing level is determined based on the defecation image (see S43), and advice is given to the subject based on the change in the cleansing level over time (see S47, S49). This makes it possible to give advice tailored to the condition of each individual subject.

[0085] In the flow of the cleansing check shown in FIG. 3A and FIG. 3B, when it is time for the subject to take the cleansing agent, the subject is advised to that effect (see S21, S23). The cleansing agent must be taken at a specified time interval, and some subjects find time management troublesome, but even such subjects can take the cleansing agent without any problems in this flow. In addition, if the subject feels that it is too hard to take the cleansing agent any more and operates the give up icon (S27 Yes), an image of the defecation is sent to the hospital, and the degree of cleansing is judged at the hospital (see S29, S31). Although the degree of cleansing of the image may be judged using AI, in this embodiment, the person in charge at the medical institution judges it, so that the subject can be reassured. In this case, if the result of the intestinal cleansing indicates that the intestines are clean, "cleaning completed" is displayed (S47), and on the other hand, if the intestines are not clean, an encouraging message such as "keep up the good work" may be displayed.

[0086] Next, the acquisition of the cleaning level in step S49 and the prediction of the cleaning degree in step S49 will be described with reference to FIGS.

[0087] First, the prediction of cleanliness will be explained using FIG. 4. FIG. 4 is a graph showing the change in cleanliness, with the horizontal axis showing the time since the start of taking the cleansing agent, and the vertical axis showing the cleanliness. The vertical axis shows the cleanliness at the top, low, and low, high, and if the cleanliness is below the level Lev, the test is possible. The solid line is the standard cleanliness curve Lst showing the standard change in cleanliness since the start of taking the cleansing agent. It is considered that the change in cleanliness has an approximate tendency depending on the movement of the intestines. Therefore, the meal contents, time, amount of stool, and time are collected, the cleanliness is predicted, and the standard cleanliness curve Lst is created. This standard cleanliness curve Lst may be created using AI, or may be created using a classical method (logical method, etc.). It may also be changed based on the previous performance, gender, age, race, region, etc. It may also be changed according to the cleansing agent used. The creation of this standard cleanliness change curve will be described later using FIG. 5 to FIG. 8.

[0088] In the graph of FIG. 4, Ra1 to Ra5 ​​are the measured values ​​of the cleanliness of the subject a at each time, and the dashed line is a predicted change curve Lesa of the cleanliness of the subject a, which is created by considering the measured values ​​Ra1 to Ra5 ​​of the subject a with respect to the standard cleanliness curve Lst. Using this predicted change curve Lesa, the cleanliness of the subject a at each time can be predicted, and Ra6 and Ra7 are the predicted values. In this example, the amount of detergent cannot be reduced, but a sense of security can be provided by displaying how much more detergent is required to drink before the specified amount is consumed.

[0089] The dashed line is an alert line La for judging whether or not the degree of cleansing is insufficient when the subject drinks the cleansing agent to cleanse the intestinal tract. If the subject's measurement value is to the right of the alert line La, that is, if the degree of cleansing is insufficient, a warning is issued to the subject. In the example shown in Fig. 4, the measurement value Ra5 is to the right of the alert line La, exceeding the alert line La and indicating an insufficient degree of cleansing, so the service server 10 issues an alert to the hospital.

[0090] In the graph of FIG. 4, Rb1 and Rb2 are the measured values ​​of the cleanliness of the subject b at each time. As in the case of the subject a, a predicted change curve Lesb of the cleanliness can be created by considering the measured values ​​Rb1 and Rb2 of the subject b with respect to the standard cleanliness curve Lst. Using this predicted change curve, the cleanliness of the subject b at each time can be predicted, and Rb3, Rb4, and Rb5 are the predicted values. In the case of the subject b, the cleanliness is more advanced than the standard cleanliness curve Lst, so it is predicted that the cleanliness will be completed even if the amount is less than the specified amount. In other words, it is possible to roughly estimate from the actual cleanliness curve and display how much should be drunk.

[0091] Thus, the time required for the test level Lev to be reached varies from subject to subject, and in the example shown in FIG. 4, there is a difference in the time difference Tdif. In general, a prescribed amount of intestinal cleansing agent is taken at regular intervals, so the amount of cleansing agent to be taken can be reduced in approximately proportion to the time difference Tdif. In this embodiment, the cleansing level is measured, and the time required for the test level to be reached is predicted in consideration of individual differences, and is presented to the subject. That is, it is possible to predict how much time will be required and how many times the cleansing agent will be taken from the standard cleansing level curve Lst and the actual measured value. Based on this prediction, the UI unit 24 of the user terminal 20 may display a message with a sense of watching over the subject, such as "You're doing well" or "It's a little late, but do you want to talk?".

[0092] Next, the cleanliness prediction algorithm will be described. First, the creation of the standard cleanliness curve Lst will be described. To create the standard cleanliness curve, the following data is obtained from N subjects. - Basic information of the subject (age, gender, weight, height, presence of constipation, etc.) - Images and time of defecation after drinking detergent

[0093] Once the above data has been obtained from the N subjects, the cleanliness is then quantified. To quantify the cleanliness, an image feature F is extracted from the defecation image. The image feature F may be a vector value in the RGB color space or an average value of RBG. Alternatively, a feature using another color space may be calculated. Furthermore, solid feces may be extracted, and the occupancy rate of the solid matter in the image may be used as the feature.

[0094] A sequence of defecation images up to the point where general inspection is possible will be called a reference image sequence. Figure 5 shows an example of a reference image sequence. In this example, image P1 of the subject's initial defecation shows a solid state, subsequent image P2 shows a muddy state, and subsequent image P3 shows a state with residue. Image P4 in an inspectable state is a transparent liquid. In the reference image sequence, the image feature amount in image P4 in an inspectable state is F_ok. The feature amount F_j of each defecation image and its distance in the feature amount space from F_ok are denoted as D_j, and this is defined as the cleanliness level. In other words, the smaller the value of D, the closer the state is to being inspectable.

[0095] Once the reference image sequence has been created and the features of each image have been calculated, a standard cleanliness curve is then created. First, image data of N subjects are plotted in the space of cleanliness vs. time. Patterns with clearly different decreasing trends are recorded as different standard patterns (see, for example, the standard cleanliness curve Lst in FIG. 4). At this time, basic information about the subjects that defines the groups is identified based on the groups of patterns of the standard cleanliness curve. By carrying out these processes, a standard cleanliness curve Lst is created. It is also possible to create different standard cleanliness curves Lst for each group of the population, based on their age, sex, etc.

[0096] The above-mentioned cleaning curve may vary greatly at first depending on the mass being excreted, and then change to a mode in which the substances adhering to the surface of the intestine are washed away by the cleaning agent, so that the cleaning curve may be determined by the attenuation of the first mode determined by the cross-sectional area of ​​the intestine in the luminal direction, and the attenuation of the second mode determined by the surface area of ​​the inner surface of the intestine. The shape and size of the intestine may be classified and determined based on the profile of the subject.

[0097] It is also known that whether or not the examinee has paid attention to their diet up until the day of the examination affects whether or not residues are likely to remain. For example, in terms of noodles, udon, somen, and hiyamugi are easy to digest and do not leave residues, while soba, ramen, and pasta are known to be more likely to be affected. It is known that people who have consumed these recommended foods can take the examination with less intestinal cleansing agent than people who have eaten other foods. In other words, information regarding the end timing of the process related to the preparation for the endoscopic examination may be generated according to the dietary history information of the examinee or subject. Also, according to the dietary history information, the threshold level for the examination being OK may be corrected, or the expected time or amount of intestinal cleansing agent may be corrected to provide a guide for the end of the cleansing. The above-mentioned history information may be self-reported by inputting it into a smartphone or terminal, or may be input into the system by a medical professional through oral report, or may be determined from a menu image taken with a smartphone.

[0098] The above-mentioned flushing curve depends on the lumps and liquid color in the bowel movement, and after solid excrement is released, only liquid remains, and finally only residue is visible. Although it is called liquid, it is not a solution, so it eventually turns muddy after visible solids are released. In this way, the flushing curve can be classified into classes where there is solid (Class 1), where it is muddy (Class 2), and where there is liquid + residue (Class 3). A curve based on the correlation between the duration of Class 1 (or Class 2 as well) and the time change of Class 3 can be used as a substitute for the flushing curve, or to predict the end of flushing.

[0099] In addition to the method of defining the cleanliness from the image feature amount F and creating the standard cleanliness curve Lst as described above, the cleanliness may be defined using deep learning. This method will be described with reference to FIG. 6.

[0100] FIG. 6(a) shows a method for collecting defecation image data and generating an inference model. First, a large number of defecation image data are collected in association with the subject's basic information and the time since they started taking the cleanser. Next, an annotator (a person who adds annotations) refers to the reference image sequence (see FIG. 5 and FIG. 6(b)) for the collected image data, determines which class the image belongs to, adds the determined class as an annotation, and creates training data. To explain using the example of FIG. 6, the annotator looks at defecation image P11, refers to the reference image sequence (images P1 to P4) shown in FIG. 6(b), and determines which image (class) defecation image P11 is closest to.

[0101] In the example shown in Fig. 6(b), the cleanliness levels of images P1 to P4 are 100, 66, 33, and 0, respectively. In Fig. 6(a), the defecation image P11 is annotated A11 with "No. 2" since it is close to "2. Muddy state" in image P2, and teacher data T11 is created. Teacher data is created by annotating all of the collected image data.

[0102] Once training data is created from N subjects, an inference model that predicts stool images and classes is created using this training data. This may be replaced with the distance D in the feature space of the classical method. Once this inference model is created, a standard cleanliness curve Lst can be obtained from the stool images of many subjects. In other words, a standard cleanliness curve Lst (see Figure 4) can be obtained that shows how the cleanliness changes over time when a standard subject drinks a cleanser. A standard cleanliness curve Lst may be created for each subject profile, such as the gender and age of the subjects who are the parent set of training data.

[0103] When the standard cleanliness curve Lst is created by quantifying the cleanliness from the image feature value F as described above, or when the standard cleanliness curve Lst is created using deep learning as described with reference to FIG. 6, the system then selects a standard cleanliness curve Lst suitable for the user based on the subject's (user's) basic information (e.g., gender, age group, etc.). FIG. 7(a) shows an example of a standard cleanliness curve Lstc suitable for a user c. In FIG. 7(a), as in the graph of FIG. 4, the vertical axis indicates the cleanliness and the horizontal axis indicates the elapsed time since the subject started taking the cleaner. Note that, from the standard cleanliness curve Lst of M patterns created based on basic information such as representative age and gender, a standard cleanliness curve Lst of N patterns (N≦M) close to the subject's basic information may be selected, and the standard cleanliness curve Lstc of the subject may be created by performing interpolation or the like.

[0104] After the standard cleanliness curve Lstc suitable for the subject is selected, the subject then drinks the cleansing agent and sends a defecation image to the service server 10 every time he or she defecates (see S37 to S41 in Fig. 3A and Fig. 3B). When the service server 10 receives the defecation image, the cleanliness determination unit 14 calculates the cleanliness from the defecation image. This calculation may be performed by the cleanliness determination unit 14 comparing the defecation image with the reference image sequence shown in Fig. 5, or may be performed using an inference model for inference. The cleanliness D may also be calculated using image features.

[0105] After the cleanliness assessment unit 14 calculates the cleanliness, it plots the cleanliness on a graph. After multiple cleanliness levels have been plotted, a predicted curve is calculated and displayed as time-series data. In the example shown in FIG. 7(b), the cleanliness levels Rc1, Rc2, and Rc3 are calculated for each defecation image of subject c. Next, a predicted curve Lc of the cleanliness level of subject c is calculated based on the cleanliness levels Rc1, Rc2, and Rc3.

[0106] The algorithm for calculating the predicted curve of the subject's cleanliness may be an AR model or a Kalman filter. Alternatively, the predicted curve may be calculated by fitting a standard cleanliness curve to the plotted data of the subject. When the predicted curve is calculated, it is possible to calculate how many more times the subject needs to drink the cleaner before the test can be performed, and the number of times to drink the cleaner can be notified to the subject (see S49 in Figures 3A and 3B).

[0107] Next, with reference to FIG. 8, an example in which prediction of cleanliness is performed by deep learning instead of a classical method will be described. To infer cleanliness using deep learning, first, an inference model is created by deep learning. FIG. 8(a) shows the deep learning algorithm. First, images of defecation of N people who have drunk a cleanser are acquired (#1). For example, the service server 10 acquires images of defecation of subjects through multiple user terminals. With the consent of the subjects, the images acquired in S41 of FIG. 3A and FIG. 3B may be used.

[0108] Once the defecation images have been collected, information is then added to each image until it becomes suitable for examination, and learning is performed (#3). Information until it becomes suitable for examination includes the amount of cleanser taken, the number of times, the time, etc. until it becomes suitable for examination. Figure 8(c) shows an example of defecation images of subject A. Images PA1, PA2, ... are defecation images of subject A, with image PA1 on the left being the first image, changing to images PA2, PA3, ... as time passes, and in the final image PAN, the defecation has been sufficiently cleaned to perform an endoscopic examination. Similarly, images PB1, PB2, PB3, PBN are defecation images of subject B, and in the final image PBN, the defecation has been sufficiently cleaned.

[0109] In step #3, subject A's test is OK at the image PAN stage, and subject B's test is OK at the image PBN stage, so the information that is available for the test includes the amount of cleanser that subject A and B drank, the number of times, the time, etc., until the test is OK. For example, at the defecation image PA1 stage, subject A's information that is available for the test, such as another 1500 ml, another 10 times, or another 2 hours, is available.

[0110] Once information such as the amount of detergent required (cleaning time) until the image is ready for testing is obtained, the information required until the image is ready for testing is annotated onto each stool image to create training data. Since images of stool from N people are collected, N pieces of training data can be created. Once the training data is ready, the stool image is input into the inference engine, and an inference model is generated by changing the weighting of the connections for each input of the inference engine's neural network so that the information required until the image is ready for testing can be output. When the inference model is generated, the learning process in Figure 8(a) ends.

[0111] When the inference model is generated, the expected time of the end of the cleaning in step S49 in FIG. 3A and FIG. 3B and the amount (number of times) of the cleanser to be drunk until then can be inferred. The algorithm of this inference will be explained with reference to FIG. 8(b). First, the subject (user) undergoing an endoscopic examination transmits a defecation image to the system (#11). As explained in the flow of FIG. 3A and FIG. 3B, when the subject drinks the cleanser and defecates, the subject is advised to take an image of the defecation and transmit the image (see S37, S39, S41 in FIG. 3A and FIG. 3B, and screen D4 in FIG. 9B). This defecation image is transmitted to the service server 10.

[0112] Next, based on the received image, the amount of detergent, the number of times, and / or the time until the test becomes possible are estimated (#13). When the service server 10 receives the defecation image data in step S41 of FIG. 3A and FIG. 3B, the cleanliness determination unit 14 inputs the received defecation image data to the inference engine. For this purpose, the inference model described with reference to FIG. 8(a) is set in the inference engine. When the defecation image data is input to the inference engine, the amount of detergent, the number of times, and / or the time until the test becomes possible can be estimated. When the inference result is output, the expected time when the cleaning will end and the amount (number of times) of detergent to be drunk until then can be displayed on the display unit of the UI unit 24 or the like (for example, see screen D5 of FIG. 9B).

[0113] Here, we will explain the deep learning performed in Figure 8(a) and other figures. "Deep learning" is a multi-layered structure of the "machine learning" process using neural networks. A typical example is the "forward propagation neural network," which sends information from front to back and makes a judgment. The simplest forward propagation neural network has three layers: an input layer consisting of N1 neurons, a middle layer consisting of N2 neurons given by parameters, and an output layer consisting of N3 neurons corresponding to the number of classes to be discriminated. Each neuron in the input layer and middle layer, and the middle layer and output layer are connected by a connection weight, and a bias value is added between the middle layer and output layer, making it easy to form a logic gate.

[0114] A neural network can have three layers if it is to perform simple discrimination, but by having many intermediate layers, it is possible to learn how to combine multiple features in the machine learning process. In recent years, neural networks with 9 to 152 layers have become practical in terms of the time required for learning, judgment accuracy, and energy consumption. In addition, a "convolutional neural network" that compresses image features, operates with minimal processing, and is strong in pattern recognition, can be used. In addition, a "recurrent neural network" (fully connected recurrent neural network) that can handle more complex information and flow information in both directions can be used to handle information analysis in which the meaning changes depending on the order or sequence.

[0115] To realize these technologies, conventional general-purpose arithmetic processing circuits such as CPUs and FPGAs (Field Programmable Gate Arrays) may be used. However, this is not the only option. Since most neural network processing is matrix multiplication, processors specialized for matrix calculations such as GPUs (Graphic Processing Units) and Tensor Processing Units (TPUs) may also be used. In recent years, such artificial intelligence (AI) dedicated hardware "neural network processing units (NPUs)" have been designed to be integrated and embedded with CPUs and other circuits, and may become part of the processing circuit.

[0116] Other machine learning methods include, for example, support vector machines and support vector regression. The learning here involves calculating the weights, filter coefficients, and offsets of a classifier, and there are also methods that utilize logistic regression processing. When making a machine make a judgment, a human needs to teach the machine how to make the judgment. In this embodiment, a method is adopted in which an image judgment is derived by machine learning, but a rule-based method that applies rules acquired by humans through experience and heuristics may also be used.

[0117] Next, with reference to Figs. 9A to 9C, advice provided to the subject on the UI unit 24 of the user terminal 20 will be described. Fig. 9A is an example of an alarm to start drinking, which is performed in step S3 of Fig. 2. As described above, the flow shown in Fig. 2 shows the operation when the subject drinks a cleaning agent at home or the like to undergo an endoscopic examination, cleans the intestinal tract, and then goes to the hospital to undergo the examination. In this flow, advice to start drinking is presented to the subject when it is time to start drinking, calculated backwards from the start time of the examination (see S3 in Fig. 2). Screen D1 in Fig. 9A shows an advice image at this time, and in this example, "Please start drinking. It will take more than two hours to finish drinking. Please start drinking soon" is displayed on the UI unit 24. At this time, the required time may be changed and displayed using past cleaning time results, etc. For example, if the degree of cleaning was sufficient for the examination in one and a half hours in the past, it may be displayed that about one hour and 45 minutes is required.

[0118] 9B shows advice displayed on the UI unit 24 of the user terminal 20 from the start of taking the detergent to the completion of cleaning. Screens D2a, D2b, and D2c are screens that display the amount of detergent. Since there are several types of detergent, the display is made according to the detergent given to the subject. Screen D2a displays how to take the detergent when the detergent is MoviPrep (registered trademark), screen D2b displays how to take the detergent when the detergent is Moven (registered trademark), and screen D2c displays how to take the detergent when the detergent is VisiClear (registered trademark).

[0119] In addition, the icon Iph is displayed at the bottom left of screens D2a, D2b, and D2c for making voice calls to the hospital where the subject will be examined. If the subject feels that the cleansing agent is too hard to swallow while drinking, he or she can touch this icon Iph to call the hospital and talk to the person in charge. Screen D3 shown in FIG. 9B is the screen displayed when the hospital is called. When this screen D3 is displayed, the subject can talk to the person in charge and have them show an image of the defecation, and then decide whether to continue drinking the cleansing agent, stop, or stop and go to the hospital (see S29 in FIG. 3A and FIG. 3B).

[0120] In addition, a toilet icon Ito is displayed at the bottom right of the screens D2a, D2b, and D2c, allowing the subject to receive advice when he or she goes to the toilet to defecate. At this time, when the subject touches the toilet icon Ito, a screen D4 is displayed, prompting the subject to take a picture of the defecation (see S39 in Fig. 3A and Fig. 3B).

[0121] When the subject transmits a defecation image to the service server 10, the cleanliness assessment unit 14 assesses the cleanliness level based on the received image. The cleanliness assessment unit 14 also predicts the amount (number of times) of cleanser to be consumed until the intestines are sufficiently cleansed to undergo an endoscopic examination, and the end time. The instruction transmission unit 16 in the service server 10 transmits the prediction result to the user terminal 20, and causes the UI unit 24 to display it as shown on screen D5. In this example, it is predicted that 33% of the required amount will be consumed, 8 cups of cleanser will remain, and the cleansing will end around 10:55. At this time, the service server 10 may also notify the in-hospital system 31 or 34 of the hospital of the degree of cleansing of the intestines of the subject.

[0122] Furthermore, when the subject transmits an image of defecation to the service server 10 and the cleansing degree assessment unit 14 determines that the subject has been sufficiently cleansed (see S47 Yes in Fig. 3A and Fig. 3B), the instruction transmission unit 16 causes the user terminal 20 to display that the cleansing has been completed. In the example shown in Fig. 9B, the UI unit 24 displays the completion as shown in screen D6. At this time, the service server 10 may also notify the hospital's in-house system 31 or 34 that the subject's intestinal cleansing has been completed.

[0123] FIG. 9C shows advice displayed on the UI unit 24 of the user terminal 20 when the subject leaves his / her home or the like to travel to the hospital after intestinal cleansing with a cleansing agent is completed. When intestinal cleansing with a cleansing agent is completed, a display such as that shown in screen D6 of FIG. 9B is displayed. Thereafter, when it is time to depart taking into consideration the travel time to the hospital and the start time of the examination, a departure alarm is displayed on the user terminal 20 (see S11 and S13 of FIG. 2). Screen D7 of FIG. 9C shows an example of the departure alarm display. The departure alarm not only notifies the subject that it is time to depart, but may also acquire and display reference information for travel, such as traffic conditions from home or the like to the hospital.

[0124] When the subject arrives at the hospital, the UI unit 24 displays the examination schedule as shown in screen D8 (see S19 in FIG. 2). For this display, the schedule management unit 13 and the time prediction unit 15 predict all steps of the subject's examination, and the instruction transmission unit 16 causes the UI unit 24 to display the examination schedule. For steps that have already been completed, the screen D8 may display the time based on the results by changing the character size, color, or font (displaying in such a way that the predicted time and the time based on the results can be distinguished). For example, when the subject arrives at the hospital, if the scheduled start time of the examination is 13:00 and the scheduled end time of the examination is 13:30, they are displayed as shown in screen D8. After that, if the time prediction unit 15 predicts the scheduled start time of the examination to be 13:05 based on the congestion status of the examination room, the predicted time is displayed. Also, if the examination actually starts at 13:04 and ends at 13:20, the screen D8 displays those times. Also, when the examination is to be completed 10 minutes earlier than the initial examination completion time, the time prediction unit 15 predicts the time forward taking this time into consideration, and the time predicted forward by the time prediction unit 15 is displayed.

[0125] As described above, in the first embodiment of the present invention, the subject takes a cleaning agent at home or the like other than a medical facility, cleans the intestines, and then moves to a medical facility to undergo an endoscopic examination. When undergoing this examination, status information such as where the subject is (S5 in FIG. 2) and what the cleaning state is (see S7 in FIG. 2, S43, S45 in FIG. 3A and FIG. 3B, etc.) is obtained, and the progress of the examination is estimated based on this status information (see, for example, S9, S15, S17, S19 in FIG. 2, S49 in FIG. 3A and FIG. 3B, etc.), and advice is provided based on this progress (see S13 in FIG. 2, S49 in FIG. 3A and FIG. 3B, etc.). Therefore, the subject can receive the advice necessary to undergo the endoscopic examination appropriately, and can undergo the examination with peace of mind.

[0126] Next, a second embodiment of the present invention will be described with reference to Figures 10 to 12. In the first embodiment, the subject drinks the cleaning agent not at a hospital but at home, etc., and after intestinal cleaning is completed, the subject moves to the hospital and undergoes an endoscopic examination. In contrast, in the second embodiment, the subject moves to the hospital, drinks the cleaning agent there, and after intestinal cleaning is completed, the subject undergoes an endoscopic examination. Therefore, the first and second embodiments differ in where the subject undergoes intestinal cleaning.

[0127] Fig. 10 is a block diagram showing the overall configuration of an endoscopic examination support system according to the second embodiment. As in the first embodiment, this endoscopic examination support system is composed of a service server 10, a user terminal 20, and in-hospital systems 30 and 34. The internal configurations of the service server 10, the user terminal 20, and the in-hospital system 30 are similar to those of the first embodiment shown in Fig. 1, and therefore will not be described.

[0128] The in-hospital system 34 has a control unit 35, a communication unit 36, a waiting room information acquisition unit 37, an examination room information acquisition unit 38, and a recovery room information acquisition unit 39, and each of these units has the same configuration as in the first embodiment shown in Fig. 1, so detailed explanations will be omitted. In addition to the above-mentioned units, the in-hospital system 34 has a pretreatment room information acquisition unit 40. In the second embodiment, the subject is transported to the hospital and then undergoes pretreatment such as drinking a cleaning agent.

[0129] The pretreatment room is a room where the subject takes a cleaning agent and performs other pretreatments before an endoscopic examination. The pretreatment room information acquisition unit 40 acquires information about the pretreatment performed by the subject. The pretreatment room information acquisition unit 40 has an imaging unit such as a camera and an audio collection unit such as a microphone arranged in the pretreatment room, and acquires various information about the pretreatment performed by the subject in the pretreatment room. Information in the pretreatment room is acquired based on the information collected by the imaging unit and audio collection unit. If the pretreatment information acquisition unit 40 can directly communicate with the user terminal 20, it can detect that the subject having the user terminal 20 is in the pretreatment room. Furthermore, it is possible to acquire various information such as what kind of examination the subject has come to receive, what kind of pretreatment is scheduled to be performed, and the subject's profile. In addition, a nurse or the like may directly input information as text or images from an in-hospital terminal. Instead of the user terminal 20, the hospital may lend a dedicated information terminal to the subject to collect information about the subject.

[0130] Next, the main operation of the application software for supporting an endoscopic examination for a subject (user) in cooperation with the user terminal 20 and the service server 10 in the second embodiment will be described with reference to the flowchart shown in FIG. 11. As in the first embodiment, the main operation of this application is realized by the control unit 11 in the service server 10 cooperating with the control unit 21 in the user terminal 20 to control each unit in the service server 10, and further cooperating with the in-hospital systems 20 and 35. The flow shown in FIG. 11 is described on the premise that, when a safety application for examination assistance is installed in the user terminal 20, the service server 10 takes the lead in providing advice to the user terminal 20 for the subject. In this case, the service server 10 functions as an examination guide device. However, it is of course possible for the user terminal 20 to take the lead in providing advice to the subject with the support of the service server 10. In this case, the user terminal 20 functions as an examination guide device.

[0131] As described above, the second embodiment is significantly different in that the subject drinks the cleaning agent at the hospital. Therefore, compared with the flow of the main operation in FIG. 2 according to the first embodiment, steps S1 to S5 and S9 performed by the subject at home are omitted, and step S10 is added instead. In addition, steps S14a to S14c are added as processing after the subject arrives at the hospital (note that S14c in FIG. 11 is the same as S5 in FIG. 2), steps S15 and S17 are omitted, and step S23 is added. Therefore, the flowchart in FIG. 11 will be described focusing on the differences from the first embodiment.

[0132] When the main operation shown in FIG. 11 starts, first, the current location is acquired (S7). As described above, in this embodiment, it is assumed that the subject moves to a medical facility such as a hospital, takes a cleaning agent there, and undergoes intestinal cleaning. In this step, in order to calculate the time required to move to the medical facility, first, the current location of the subject is acquired. For this purpose, the control unit 11 instructs the service server 10 to transmit information regarding the current location of the subject acquired by a position detection unit such as GPS in the user terminal 20, and acquires the current location of the subject. Note that, when the application software in FIG. 11 is executed proactively by the user terminal 20, in this step, the control unit 21 acquires current location information by a position detection unit in the user terminal 20.

[0133] After acquiring the current location information, the estimated pre-treatment start time is acquired (S10). Here, in order to calculate the time when the subject leaves his / her home, etc., the time when the pre-treatment, such as taking a cleaning agent in a hospital, is to start is acquired. The pre-treatment start time can be acquired by the control unit 11 inquiring of the schedule management unit 33 of the hospital or the like where the endoscopic examination is performed. The estimated pre-treatment start time may be acquired based on the reference time managed by the schedule management unit 33 and the time predicted by the time prediction unit 15 based on the time required for the process to be performed in the hospital, such as by the control unit 11 inquiring of the scheduled examination start time to the schedule management unit 33, and the like, such as by the time prediction unit 15 predicting the estimated pre-treatment start time by calculating backwards from the scheduled examination start time. Note that, when the application software of FIG. 11 is executed proactively by the user terminal 20, in this step, the control unit 21 inquires of the schedule management unit 33 through the service server 10 to acquire the estimated pre-treatment start time.

[0134] Next, it is determined whether the departure time has arrived (S11). Here, the control unit 21 calculates the start time of movement to the medical facility based on the current location information acquired in step S7 and the information on the predicted pre-treatment start time acquired in step S10. In order to calculate the departure time, the subject's preference for the means of transportation from home to the hospital may be set and the departure time may be calculated. In addition, the calculation may take into account the traffic conditions in front of the medical facility. After calculating the departure time, it is determined whether the current time is the calculated start time of movement (departure time). If the result of this determination is that it is not the departure time, the process returns to step S7. Note that, in the case where the application software in FIG. 11 is executed proactively by the user terminal 20, in this step, the control unit 21 calculates the departure time and determines whether the current time is the departure time.

[0135] If the result of the determination in step S11 is that it is time to depart, a departure alarm is notified to the subject (S13). As it is time to depart, the control unit 11 causes the UI unit 24 of the user terminal 20 to display a departure alarm. The departure alarm may be generated by a sound-generating member. When the departure alarm is notified, the examiner starts moving from his / her home or the like to the medical facility. Note that, in the case where the application software in FIG. 11 is executed proactively on the user terminal 20, in this step, the control unit 21 may cause the UI unit 24 to notify the departure alarm.

[0136] Next, it is determined whether or not the subject has arrived at the hospital (S14a). Here, it is determined whether or not the subject has arrived at the destination hospital, etc., based on the detection result of the position detection unit of the user terminal 20. For this purpose, the control unit 11 acquires information on the current location from the user terminal 20. If the result of this determination is that the subject has not arrived at the hospital, etc., the control unit 21 goes into a standby state. Note that, in the case where the application software in FIG. 11 is executed autonomously by the user terminal 20, in this step the control unit 21 makes the determination based on the detection result of the position detection unit in the user terminal 20.

[0137] If the result of the determination in step S14a is that the user has arrived at the hospital or the like, acquisition of in-hospital task predicted times begins (S14b). In the hospital, there are in-hospital tasks such as pretreatment (cleaning), examination, rest (recovery), explanation of examination results, and accounting (settlement). In this step, the control unit 11 acquires information on the predicted times of these tasks from the schedule management unit 33 in the in-hospital system 30. Note that, in the case where the application software in FIG. 11 is executed proactively by the user terminal 20, in this step the control unit 21 acquires the information from the schedule management unit 33 in the in-hospital system 30 via the service server 10.

[0138] Next, the cleanliness is checked (S14c). This cleanliness check is performed in the same manner as step S5 in FIG. 2. The subject drinks the cleanser in a hospital or the like, and thoroughly cleanses the intestinal tract. At this time, the control unit 11 causes the image of the subject's defecation to be acquired by the imaging unit 23 through the user terminal 20, and transmits the image to the service server 10 through the communication unit 22. The cleanliness determination unit 14 in the service server 10 determines the cleanliness based on the defecation image of the subject, and determines whether or not a sufficient cleanliness level has been reached. The operation of this cleanliness check is the same as the flow of FIG. 3A or FIG. 3B described above, so a detailed description will be omitted. During cleaning, the UI unit 24 of the user terminal 20 displays, for example, advice as shown in FIG. 9B described above. When the application software of FIG. 11 is executed proactively on the user terminal 20, in this step, the control unit 21 cooperates with the cleanliness determination unit 14 in the service server 10 to determine the cleanliness.

[0139] After checking the cleanliness, the estimated time for the entire inspection process to be completed is displayed (S19). The control unit 11 estimates the estimated time for all the inspection processes while checking the progress of the inspection and the schedule in the schedule management unit 33, and causes the UI unit 24 of the user terminal 20 to display this estimated time. This display will be described later with reference to FIG. 12. Note that, in this step, when the application software in FIG. 11 is executed independently on the user terminal 20, the control unit 21 cooperates with the service server 10, the schedule management unit 33, etc. to estimate the estimated time for all the inspection processes, and displays this estimated time on the UI unit 24.

[0140] Next, it is determined whether the subject has left the hospital or the like (S21). When all tasks in the hospital, such as intestinal cleansing, examination, recovery, explanation of the examination results, and accounting (settlement), the subject leaves the hospital or the like. Here, the control unit 11 instructs the service server 10 to transmit information on the subject's current location acquired by a position detection unit such as a GPS in the user terminal 20, acquires the subject's current location, and makes a determination. If the subject's location information can be acquired by the waiting room information acquisition unit 37, examination room information acquisition unit 38, and recovery room information acquisition unit 39 in the in-hospital system 34 in addition to the detection result by the position detection unit in the user terminal 20 of the subject, this information may be used. If the result of the determination in step S21 is that the subject has not left the hospital or the like, the process returns to step 19 and the above-mentioned operation is repeated. Instead of returning to step S19, the process may return to step S14c and check the cleansing degree. In this case, if the cleansing has already been completed, step S14c may be skipped. In addition, when the application software in FIG. 11 is executed proactively on the user terminal 20, in this step, the control unit 21 acquires current location information by the location detection unit in the user terminal 20 or information from the in-hospital system 34 and makes a judgment.

[0141] If the result of the determination in step S21 is that the subject has left the hospital, acquisition of the in-hospital task predicted time is terminated (S23). The acquisition of the predicted in-hospital task times was started in step S14b and was displayed on the UI unit 24 of the user terminal 20 (see FIG. 12). Since the result of the determination in step S21 is that the subject has completed all tasks for the examination, acquisition of the predicted times is terminated and the main operation flow is terminated.

[0142] Next, the display of the expected time of completion of the entire examination process will be described with reference to FIG. 12. This display is performed in step S19 of FIG. 11. In the second embodiment, as in the first embodiment, the time prediction unit 15 predicts the start time of the in-hospital task, etc. (see S14b, S23 of FIG. 11). Based on the start time of the in-hospital task, etc., the in-hospital schedule can be displayed on the UI unit 24 of the user terminal 20. FIG. 12 is an example of displaying the in-hospital task schedule, similar to the screen D8 of FIG. 9C, where CT indicates the current time. Note that, unlike the screen D8 of FIG. 9C, in the second embodiment, cleaning is being performed at a medical facility. In this way, the schedule of the in-hospital task is displayed on the UI unit 24 of the user terminal 20, so that the subject can easily know what to do next and what time it will be completed.

[0143] As described above, in the second embodiment of the present invention, after the subject moves to a medical facility, he or she takes a cleaning agent to cleanse the intestines and undergoes an endoscopic examination. When undergoing this examination, status information such as the state of cleaning of the subject during the endoscopic examination (see S14c in FIG. 11, S43, S45 in FIG. 3A and FIG. 3B, etc.) is acquired, and the progress of the examination is estimated based on this status information (see, for example, S11, S14a in FIG. 11, S49 in FIG. 3A and FIG. 3B, etc.), and advice is provided based on this progress (see S13 in FIG. 11, S49 in FIG. 3A and FIG. 3B, etc.). Therefore, the subject can receive the advice necessary to undergo the endoscopic examination appropriately, and can undergo the examination with peace of mind.

[0144] As described above, in each embodiment of the present invention, when undergoing an endoscopic examination, status information such as the cleaning status of the subject of the endoscopic examination (see S5 in FIG. 2, S43, S45 in FIG. 3A and FIG. 3B, S14c in FIG. 11, etc.) is acquired, and the progress of the examination is estimated based on this status information (see, for example, S5 in FIG. 2, S14a in FIG. 11, S49 in FIG. 3A and FIG. 3B, etc.), and advice is provided based on this progress (see S49 in FIG. 3A and FIG. 3B, etc.). Therefore, the subject can receive the advice necessary to undergo an endoscopic examination appropriately, and can undergo the examination with peace of mind.

[0145] In each embodiment of the present invention, a method for estimating the progress of a process related to an endoscopic examination of an examinee based on condition information of the examinee and displaying information related to the progress is provided. Also, a method for estimating the progress of a process related to an endoscopic examination of an examinee based on condition information of the examinee and displaying information related to the estimated progress is provided. Also, a prediction method is provided in which the same thing (excrement of the same examinee, water containing excrement, etc.) is repeatedly acquired, and based on the temporal change in the transparency (cleanliness) of the acquired thing, the time until the same thing reaches a specific transparency is predicted. According to this prediction method, the end time of cleaning can be known by multiple measurements (analysis of the image capture results, etc.) spanning a specific time in the first half of the start of cleaning. The end time of cleaning may be determined by referring to a specific cleaning progress model, or may be determined by multiple patterns, such as a first model if the change is fast and a second model if the change is slow, based on multiple measurements.

[0146] In this specification, the example of using images of excrement etc. has been described, but it is also possible to use optical sensors to see transmission, reflection, etc. instead of images. Instead of images, the pH of the water in the flush toilet, resistance, or reagent reaction may be used. This is a method of investigating the percentage of excrement that is made up of cleaning fluid, and can be used because the cleaning fluid is a chemical that contains electrolytes such as sodium. In addition, in the case of a flush toilet, it is assumed that not only excrement is photographed, but also water accumulated in the flush toilet bowl is mixed in. This is also true for photographic images, and it is better to photograph the water of the flush toilet containing excrement and eliminate the influence of the original flush water on the transparency and cleanliness, but this may be considered to be within the range of error.

[0147] In each embodiment of the present invention, when undergoing an endoscopic examination, status information such as the cleaning status of the subject of the endoscopic examination (see S5 in FIG. 2, S43, S45 in FIG. 3A and FIG. 3B, S14c in FIG. 11, etc.) is obtained, and the usage status (usage status of the place where at least one of the actions related to waiting, pre-treatment, examination, recovery, and payment is performed) in the medical facility where the subject undergoes the endoscopic examination is obtained (for example, S15, S17 in FIG. 2), and the time related to the endoscopic examination is estimated based on the cleaning status information and the usage status information, and advice related to the clinical examination is displayed based on this estimation (for example, see S19 in FIG. 2). Therefore, since the time of each action can be predicted for the endoscopic examination, which consists of each action, the subject can easily make plans and can undergo the examination with peace of mind.

[0148] In addition, the examination guide method in each embodiment of the present invention estimates the progress of the process related to the preparation for the endoscopic examination of the examinee based on the change over time of the condition information of the examinee for the endoscopic examination (for example, see S37 to S43 in Fig. 3A and Fig. 3B), and generates information on the end timing of the preparation process based on the estimated progress (for example, see S49 in Fig. 3A and Fig. 3B). Therefore, according to the examination guide method in this embodiment, it is possible to receive necessary advice for appropriately undergoing an examination involving medical practice such as an endoscopic examination or a clinical examination. That is, based on the change in the condition information of the examinee, the progress of the preparation process for the examination (for example, the progress of the degree of cleansing when an intestinal irrigation is performed) is judged, and based on the result of this judgment, advice on the timing of the end of the preparation process can be received.

[0149] In addition, in the examination guide method in each embodiment of the present invention, the degree of progress of the preparation steps is the degree of progress of the intestinal cleansing state of the patient undergoing an endoscopy examination (for example, FIG. 4), and the timing at which it is predicted that the intestinal cleansing state will reach a certain degree is set as the timing to end the preparation steps (for example, see S43 and S49 in FIG. 3A and FIG. 3B, the examination possible level Lev in FIG. 4, and FIG. 7 and FIG. 8). That is, in each embodiment, the timing to end the preparation is predicted based on the degree of progress of the intestinal cleansing state, and the patient can receive appropriate advice regarding the timing to end the preparation.

[0150] In the examination guide method in each embodiment of the present invention, the change in state information over time is the change in excretion image over time (see, for example, FIG. 5 and FIG. 6(b)), and based on this change in excretion image over time, the change in the cleansing degree of the intestine over time from the time the excretion image was captured is estimated (see, for example, FIG. 4 and FIG. 7(b)), and the timing at which the estimated change in cleansing degree over time reaches a certain level is set as the end timing of the intestine cleansing state (see, for example, S49 in FIG. 3A and FIG. 3B, the testable level Lev in FIG. 4, FIG. 7, FIG. 8, etc.). That is, in each embodiment, the cleansing state of the intestine cleansing is repeatedly acquired (see, for example, S37 to S41 in FIG. 3A and FIG. 3B), and based on the change in the cleansing state over time, the future cleansing degree is predicted (see, for example, S43 in FIG. 3A and FIG. 3B, the testable level Lev in FIG. 4), and the timing at which the predicted cleansing degree reaches a testable level is set as the end timing of the cleansing (see, for example, S49 in FIG. 3A and FIG. 3B). This allows the patient to receive appropriate advice regarding the timing for completing preparations.

[0151] Furthermore, in the examination guide method in each embodiment of the present invention, the time from the start of intestinal cleansing of the patient undergoing endoscopy is further taken into consideration to estimate the change in cleansing level over time (see, for example, S3 and S5 in FIG. 2, screen D8 in FIG. 9C, S19 in FIG. 11, FIG. 12, etc.). Therefore, the patient can receive advice on the appropriate timing to stop taking the cleansing agent after starting it.

[0152] Furthermore, in the examination guide method in each embodiment of the present invention, the intestinal cleansing agent taken by the endoscopy examinee is estimated, and this estimated amount (which is roughly proportional to the time since the start of cleansing) is further taken into consideration to estimate the change in cleansing level over time. For example, in the example shown in Figures 3A and 3B, the amount of cleansing agent is displayed in step S25 (for example, screens D2a to D2c in Figure 9B), and the examinee can estimate the future change in cleansing level over time based on this amount of cleansing agent and the number of times the examinee has taken it.

[0153] In addition, in the examination guide method in each embodiment of the present invention, a guide on how to take medicine for intestinal cleansing for an endoscopy examinee is displayed (for example, see S49 in Fig. 3A and Fig. 3B, screens D2a to D2c in Fig. 9B, etc.). Examinees often do not know how to take medicine for intestinal cleansing, and by being shown how to take it, they can undergo the examination appropriately.

[0154] In addition, in the examination guidance method according to each embodiment of the present invention, the guide display is changed according to the medicine that the patient takes for intestinal cleansing (see, for example, screens D2a to D2c in FIG. 9B). Since the guide display is switched according to the medicine, it is possible to prevent the patient from taking the medicine in the wrong way.

[0155] In addition, in the examination guidance method in each embodiment of the present invention, the time to take the medicine, the amount of the medicine, or the amount of other medicine to be taken together with the medicine is displayed as a guide on how to take the medicine (see, for example, screens D2a to D2c in FIG. 9B). Specifically, the time to take the medicine, the amount to take, etc. are displayed, so that it is possible to prevent the examinee from taking the medicine in an incorrect way.

[0156] In the examination guide method according to each embodiment of the present invention, an excretion image of an endoscopy examinee excreting and a plurality of excretion images are compared with the excretion progress model to estimate the time change in cleansing level and estimate the end timing of the intestinal cleansing state (see, for example, S39 to S43 in Fig. 3A and Fig. 3B, Fig. 5, etc.). That is, the excretion image of the examinee is compared with the excretion progress model (see, for example, the reference image sequence in Fig. 5), and the cleansing level is estimated using the feature amount of the excretion image of the examinee and the feature amount of the excretion progress model. In addition, an excretion image is taken every time the examinee defecates (see, for example, S37 in Fig. 3A and Fig. 3B, screen D4 in Fig. 9B, etc.), so that the cleansing level can be obtained for a plurality of excretion images. Based on the time change in the plurality of cleansing levels, the end timing of the intestinal cleansing can be predicted.

[0157] In the examination guide method in each embodiment of the present invention, the end timing of the intestinal cleansing state is estimated using an AI model trained using an excretion image in which the excretion state is captured, and multiple excretion images in which the state of the endoscopy examinee is captured (for example, see S45, S49 in Fig. 3A and Fig. 3B, Fig. 6, Fig. 8, etc.). In the examination guide method in each embodiment of the present invention, the end timing of the intestinal cleansing state is estimated by inputting an excretion image in which the subject excretes into an AI model trained using at least one of the time until the completion of cleaning and the amount of cleaning agent in addition to the excretion image in which the excretion state is captured (for example, see Fig. 6, Fig. 8). An AI model is generated using big data, etc., and the end timing of the intestinal cleansing state can be estimated using this AI model.

[0158] In the examination guide method in each embodiment of the present invention, an AI model trained using multiple learning data sets in which images of the excretion taken of an endoscopy patient excreting are annotated with the degree of cleansing is inputted into the AI ​​model to estimate the degree of cleansing and estimate the end timing of the intestinal cleansing state (see, for example, Figures 6 and 8). By using the AI ​​model, it is possible to make predictions using big data, etc.

[0159] In addition, in the examination guide method in each embodiment of the present invention, the washing end timing is estimated by inputting a plurality of images of the subject's excretion into an AI model trained using a plurality of learning data sets in which a plurality of images of the excretion are annotated with the washing time or the amount of detergent taken until the subject reaches an examination-ready state (see, for example, FIG. 8, etc.). Therefore, if big data is available, the washing end timing can be easily estimated.

[0160] In addition, in the examination guide method in each embodiment of the present invention, the end timing of the preparation process and the predicted examination start time determined by the examination time deviation situation of the examinee (including cleaning) before the endoscopic examination examinee are guided (for example, see S15 to S19 in FIG. 2, FIG. 9C, S19 in FIG. 11, FIG. 12, etc.). Since an endoscopic examination has various steps such as cleaning, examination, recovery (rest), result explanation, and payment, and moreover, since multiple examinees are undergoing an examination, if the previous examinee deviates from the schedule, the examination start time of the examinee who is receiving assistance for the examination will change. Even in this case, according to this embodiment, the fact that the examination start time has changed according to the deviation of the previous examinee is notified, which is convenient. In addition, in the examination guide method in each embodiment of the present invention, information regarding the end timing of the preparation process for the endoscopic examination is generated according to the examinee's meal history information. Since the difficulty of residue remaining varies depending on the examinee's meal history, information regarding the end timing of the preparation process can be generated with high accuracy by using this history information.

[0161] In each embodiment of the present invention, the second timing is estimated according to the treatment after the first timing. That is, since there are many steps in an endoscopic examination, and the time required for each step affects the start timing of the next step, the next timing is estimated according to each step. The end timing of cleaning may be estimated by appropriately combining the time from the start of cleaning, the amount of cleaning agent consumed, the progress of the cleaning state, the classical method or AI, the classical method and AI, etc.

[0162] Furthermore, in the examination guide method in each embodiment of the present invention, the progress of the process related to the endoscopic examination is estimated according to the condition information of the endoscopic examination patient and the congestion status or the work status of the staff at the hospital where the endoscopic examination is performed, and a guide is displayed according to this estimated progress (for example, see S15 to S19 in FIG. 2 and S19 in FIG. 11). That is, in each embodiment, the end time is estimated taking into account the in-hospital status where the endoscopic examination is performed, and this time is displayed.

[0163] In the examination guide method in each embodiment of the present invention, the condition information of the patient undergoing an endoscopic examination is the progress of the process related to the endoscopic examination (actual value, how far it has progressed) and / or the risk indicating the possibility that the patient will undergo treatment, and the start time or end time of the endoscopic examination is estimated according to the operating status of the endoscopy room and / or the endoscopy staff. In the examination guide method in each embodiment of the present invention, if the endoscopy room and / or the endoscopy staff performing the endoscopic examination to be performed by the patient is used / engaged in a second endoscopic examination on another patient, and the second endoscopic examination deviates (shortens or extends) from the time originally assumed (assumed before the second endoscopic examination) due to difficult insertion, detection of a lesion, biopsy, treatment, normality, etc., the estimated start time or estimated end time of the endoscopic examination to be performed by the patient is changed (advanced or delayed). In the examination guidance method according to each embodiment of the present invention, the congestion state of the hospital where the endoscopic examination is performed refers to the congestion state of any of the waiting area, pre-treatment, examination, recovery, notification of examination results, and accounting, or a state in which the time required to perform a process in that space deviates from the standard time (standard time at that facility).

[0164] Moreover, the examination guide device in each embodiment of the present invention has a status acquisition unit that acquires status information of an endoscopic examination examinee, a progress estimation unit that estimates the progress of the examination preparation of the endoscopic examination examinee based on the time change of the acquired status information, and an advice generation unit that displays advice related to the examination preparation based on the estimated progress. In the endoscopic examination support system shown in Figs. 1 and 10, the estimation of the progress of the examination preparation of the examinee and the generation of advice based on this estimation are mainly performed in the service server 10. However, these processes may be performed not only in the service server 10 but also in the user terminal 20 or the in-hospital system 30, 34. For this reason, it is sufficient if some of the functions in the service server 10 can be executed in the user terminal 20 or the in-hospital system 30, 34.

[0165] The above-mentioned preparation for examination is preparation for digestive tract cleaning, and the state acquisition unit acquires information related to excretion of the endoscopy patient (see, for example, S39, 41, etc. in Fig. 3A and Fig. 3B), and the progress estimation unit estimates the progress of the digestive tract cleaning of the endoscopy patient as the progress status (see, for example, S43, etc. in Fig. 3A and Fig. 3B). The progress estimation unit also estimates the progress of the digestive tract cleaning and the time required to reach a certain degree of cleaning or the number and amount of cleaning agent intake (see, for example, S49 in Fig. 3A and Fig. 3B, Fig. 4 to Fig. 8, etc.), and the advice generation unit generates information based on the estimation result of the progress estimation unit (see, for example, the instruction transmission unit 16 in Fig. 1, S49 in Fig. 3A and Fig. 3B, etc.).

[0166] The examination guide device in each embodiment of the present invention has a receiver request input unit for inputting requests of an endoscopy examinee, and the advice display unit displays advice taking into consideration the requests of the endoscopy examinee. The examination guide method acquires condition information of a clinical examination examinee, estimates a progress status of the examinee's examination preparation based on a temporal change in the condition information, and displays advice related to preparation for the clinical examination based on the progress status.

[0167] Furthermore, the examination guidance method in each embodiment of the present invention predicts (estimates) a first timing (e.g., start timing) (including cleaning) and a second timing (e.g., end timing) determined according to treatment after the first timing, depending on the congestion situation in the hospital. That is, in a hospital or the like where an examination is performed, there are multiple examinees, and the congestion situation in the hospital is not constant. Therefore, when performing examination guidance, in each embodiment, prediction (estimation) is made taking into account the congestion situation in the hospital.

[0168] Moreover, the examination guide device in each embodiment of the present invention has a status acquisition unit (e.g., see S5 in FIG. 2, FIGS. 3A and 3B, S14a in FIG. 11, etc.) that acquires status information of a patient undergoing an endoscopic examination, a usage status acquisition unit (e.g., see S15, S17 in FIG. 2, etc.) that acquires usage status at a medical institution where the patient undergoes an endoscopic examination, and a time estimation unit (e.g., see S19 in FIG. 2 and FIG. 11, screen D8 in FIG. 9C, FIG. 12, etc.) that estimates a time related to the endoscopic examination based on the status information and the usage status. Furthermore, the examination guide device may have an advice display unit that displays advice related to the endoscopic examination based on the estimation. That is, the examination guide device estimates a time related to the examination, such as a start time and an end time of the endoscopic examination, based on the endoscope used in the endoscopic examination, the usage status of the examination room, etc. In addition, the above-mentioned usage status is the usage status in the space where at least one of the actions related to waiting, pre-treatment, examination, treatment, recovery, and payment is performed at the medical institution, and the time estimation unit predicts the start or end time of each action, and the advice display unit displays advice related to the start of travel to the medical institution, the start or end of each action, and the completion of all actions at the medical institution.

[0169] Moreover, the examination guidance method in each embodiment of the present invention includes a progress estimation unit (e.g., see S49 in Figs. 3A and 3B ) that acquires status information of a subject of a clinical examination and estimates a progress status of the subject's examination preparation based on a temporal change in the status information, and an advice generation unit (e.g., see S49 in Figs. 3A and 3B ) that displays advice regarding examination preparation based on the progress status.

[0170] Furthermore, the endoscopic examination support method in each embodiment of the present invention acquires status information of a patient undergoing a clinical examination, acquires usage status at the medical institution where the patient undergoes the clinical examination, estimates a time related to the clinical examination based on the status information and usage status, and displays advice related to the clinical examination based on the estimation. Although each embodiment has been described mainly in relation to an endoscopic examination, the present invention can be applied to clinical examinations that involve various preparations and pre-processing before the examination, in addition to endoscopic examinations.

[0171] In each embodiment of the present invention, the service server 10 is described as providing advice to the subject when undergoing an endoscopic examination, but the service may be provided by a plurality of service servers, or may be provided only within the user terminal 20. In this case, the user terminal 20 may have functions such as a schedule management unit 13, a cleanliness evaluation unit 14, and a time prediction unit 15, and may be linked to the in-hospital systems 30, 34. In addition, although the description has been given mainly of undergoing an endoscopic examination, various preparations and pre-processing are required when undergoing any clinical examination, not limited to endoscopic examination, and the present embodiment can be applied to such cases as well.

[0172] In addition, in each embodiment of the present invention, logic-based judgment has been mainly described, and judgment by inference using machine learning has been performed in part. In this embodiment, whether to perform logic-based judgment or inference judgment may be appropriately selected and used. In addition, in the judgment process, a hybrid judgment may be performed by partially utilizing the advantages of each.

[0173] In each embodiment of the present invention, the control units 11, 21, 31, and 35 have been described as devices that are configured with a CPU, memory, and the like. However, in addition to being configured in a software manner using a CPU and a program, a part or all of each unit may be configured with a hardware circuit, or may be configured with a hardware configuration such as a gate circuit generated based on a program language described in Verilog or VHDL (Verilog Hardware Description Language), or may be configured with a hardware configuration using software such as a DSP (Digital Signal Processor). Of course, these may be combined as appropriate.

[0174] Furthermore, the control units 11, 21, 31, and 35 are not limited to CPUs, and may be any element that functions as a controller, and the processing of each of the above-mentioned units may be performed by one or more processors configured as hardware. For example, each unit may be a processor configured as an electronic circuit, or each circuit unit in a processor configured as an integrated circuit such as an FPGA (Field Programmable Gate Array). Alternatively, a processor configured as one or more CPUs may execute the function of each unit by reading and executing a computer program recorded on a recording medium.

[0175] In addition, in each embodiment of the present invention, the service server 10 has been described as having a control unit 11, a communication unit 12, a schedule management unit 13, a cleanliness evaluation unit 14, a time prediction unit 15, an instruction transmission unit 16, and an examination result recording unit 17. However, these do not need to be provided in a single device, and the above-mentioned units may be distributed, for example, as long as they are connected by a communication network such as the Internet. Similarly, the user terminal 20 has been described as having a control unit 21, a communication unit 22, an imaging unit 23, and a UI unit 24. However, these do not need to be provided in a single device, and the above-mentioned units may be distributed, for example, as long as they are connected by a communication network such as the Internet. The same applies to the in-hospital systems 30, 34, etc.

[0176] Furthermore, in recent years, artificial intelligence capable of making judgments based on various criteria has come to be used in many cases, and it goes without saying that improvements such as collectively performing each branch of the flowchart shown here also fall within the scope of the present invention. If the user can input the pros and cons of such control, the embodiment shown in this application can be customized to learn the user's preferences and to suit the user.

[0177] Furthermore, components from different embodiments may be appropriately combined. In particular, operations utilizing biological reactions, including voice recognition, require appropriate sensors, interfaces, and determination circuits, which are not specifically described to avoid complication. However, it should be noted that the present invention can also be achieved by various improved and alternative technologies that can substitute for manual operations by the user.

[0178] In addition, among the technologies described in this specification, the controls mainly described in the flowcharts can often be set by a program, and may be stored in a recording medium or a recording unit. The method of recording on this recording medium or recording unit may be recording at the time of product shipment, using a distributed recording medium, or downloading via the Internet.

[0179] In addition, in one embodiment of the present invention, the operation of this embodiment is explained using a flowchart, but the order of the processing procedure may be changed, any step may be omitted, steps may be added, and the specific processing content within each step may be changed.

[0180] Furthermore, even if the operational flow in the claims, specification, and drawings is described using words expressing an order such as "first" and "next" for convenience, this does not mean that it is necessary to carry out the operations in this order unless specifically explained.

[0181] The present invention is not limited to the above-described embodiment, and can be embodied by modifying the components without departing from the gist of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining the multiple components disclosed in the above-described embodiment. For example, some of the components shown in the embodiment may be deleted. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0182] 10 service server, 11 control unit, 12 communication unit, 13 schedule management unit, 14 cleaning degree judgment unit, 15 time prediction unit, 16 instruction transmission unit, 17 examination result recording unit, 20 user terminal, 21 control unit, 22 communication unit, 23 imaging unit, 24 UI unit, 30 in-hospital system, 31 control unit, 32 communication unit, 33 schedule management unit, 34 in-hospital system, 35 control unit, 36 communication unit, 37 waiting room information acquisition unit, 38 examination room information acquisition unit, 39 recovery room information acquisition unit, 40 pretreatment room information acquisition unit

Claims

1. A step of automatically estimating a progress level of a process related to preparation for an endoscopic examination of the endoscopic examination patient by a computer based on a temporal change in condition information of the endoscopic examination patient; A computer-based inspection guidance method, comprising: automatically generating information regarding an end timing of a preparation process based on the estimated progress level, The change in the state information over time is a change in the excretion image over time, A computer is used to estimate a time change in the cleansing degree of the intestine in the future from the time when the excretion image was captured based on the time change in the excretion image; The timing when the estimated change in cleansing degree reaches a certain level is determined by the computer as the end timing of the intestinal cleansing state. A computer-aided inspection guiding method comprising:

2. A step of automatically estimating a progress level of a process related to preparation for an endoscopic examination of the endoscopic examination patient by a computer based on a temporal change in condition information of the endoscopic examination patient; A computer-based inspection guidance method, comprising: automatically generating information regarding an end timing of a preparation process based on the estimated progress level, The degree of progress of the above preparation process is the degree of progress of the intestinal cleansing state of the endoscopy examinee, The timing when the intestinal cleaning state reaches a certain degree is determined as the end timing of the preparation process; An excretion image obtained by photographing the state in which the endoscopy examinee excretes; and A computer is used to compare the features of the plurality of excretion images with an excretion progress model to estimate a change in cleansing degree over time, and to estimate an end timing of the intestinal cleansing state. A computer-aided inspection guiding method comprising:

3. 3. The method for guiding examination by a computer according to claim 2, further comprising the step of: taking into consideration the time from the start of intestinal cleansing of the patient undergoing the endoscopy, and estimating the change in cleansing degree over time by the computer.

4. 3. The computer-based examination guidance method according to claim 2, further comprising the steps of: estimating the amount of intestinal cleansing agent taken by the patient undergoing the endoscopy examination using a computer; and further taking into account this estimated amount of intestinal cleansing agent, thereby estimating the change in cleansing level over time.

5. The computer-based examination guidance method according to claim 2, characterized in that the end timing of the intestinal cleansing state is estimated by a computer using an AI model trained using excretion images of the patient excreting, and a plurality of excretion images of the patient excreting.

6. The computer-based examination guidance method according to claim 2, characterized in that an AI model is trained using images of the subject excreting, as well as images of the subject's excretion, and at least one of the time until completion of cleaning and the amount of cleaning agent, and an image of the subject excreting is input to the AI ​​model, and the computer estimates the timing of the end of the intestinal cleaning state.

7. The computer-based examination guidance method according to claim 2, characterized in that an AI model is trained using multiple learning data sets in which images of the excretion state of the patient undergoing an endoscopic examination are input into the AI ​​model, and the AI ​​model estimates the cleansing level by computer, thereby estimating the timing at which the intestinal cleansing state will end.

8. The computer-based inspection guidance method according to claim 2, characterized in that a plurality of images of the subject's excretion state are input to an AI model trained using a plurality of learning data sets in which a plurality of images of the excretion state are annotated with the cleaning time or the amount of detergent ingested until the subject reaches a state where the examination can be performed, and the computer estimates the timing for the end of cleaning.

9. A status acquisition unit that acquires status information of an endoscopy patient; a progress estimation unit that estimates a progress status of the examination preparation of the endoscopy examination subject based on a temporal change in the state information; an advice generating unit for displaying advice regarding the preparation for the examination based on the progress status; having The progress estimation unit estimates the progress of the digestive tract cleansing and the time required to reach a certain degree of cleansing or the number and amount of cleaning agent intake, the advice generation unit generates information based on an estimation result of the progress estimation unit; An inspection guide device comprising:

Images