Information processing system, information processing method, and program

Abstract

To provide an excrement analyzer which can deal with variously shaped toilet bowls and seats and which can analyze imaged excrement with high accuracy.SOLUTION: An excrement analyzer 1 includes an input part 1a, a classification part 1b, and an output part 1c. The input part 1a inputs data on an image taken by an imaging unit which is installed in a manner to include an excretion range of excrement in a toilet bowl into an imaging range. The classification part 1b performs classification of imaged substances relative to the image data input by the input part 1a in a pixel unit by using a semantic segmentation. The output part 1c outputs a classification result of the classification part 1b.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an excrement analysis device, an excrement analysis method, and a program. [Background technology]

[0002] Caregivers who provide excretion assistance in the field of care are expected to maintain the dignity of the person requiring care while reducing incontinence and encouraging independence. Because excretion assistance in the field of care can sometimes damage the dignity of the person requiring care, it places a heavy burden on caregivers, and support is needed to reduce the workload.

[0003] To provide such support, a mechanism has been proposed for managing the excretion of toilet users by installing sensors in the toilet and analyzing the data acquired by the sensors. For example, Patent Document 1 describes a determination device that aims to reduce increases in equipment costs in the analysis of excrement using machine learning.

[0004] The determination device described in Patent Document 1 includes an image information acquisition unit, a preprocessing unit, an estimation unit, and a determination unit. The image information acquisition unit acquires image information of a target image that is a target image for determining a determination item related to feces and is an image of an internal space of a toilet bowl after excretion. The preprocessing unit generates an overall image showing the entire target image, and a partial image showing a partial area of ​​the target image. The estimation unit inputs the overall image into a trained model that has learned, by machine learning using a neural network, a correspondence relationship between a learning overall image, which is an image showing the entire internal space of the toilet bowl after excretion, and a determination result of a global first determination item among the determination items. The estimation unit thereby performs a first estimation regarding the first determination item for the overall image. The estimation unit inputs the partial image into a trained model that has learned, by machine learning using a neural network, a correspondence relationship between a learning partial image, which is a partial area of ​​the learning overall image, and a second determination item, which is more detailed than the first determination item among the determination items. The estimation unit thereby performs a second estimation regarding the second determination item for the partial image. The determination unit makes a determination regarding the determination item for the target image based on a result of estimation by the estimation unit.

[0005] Furthermore, in colonoscopy, a pretreatment is carried out to cleanse the intestines using an intestinal cleansing agent (laxative) before the examination. This pretreatment can be carried out at home and then followed by a visit to a hospital for the endoscopy, or it can be carried out while the patient is hospitalized. If the pretreatment is carried out at home, it is carried out by the patient himself, and if the patient is hospitalized, the examiner checks the effectiveness of the cleansing agent. For the examination, it is necessary to ensure that there is absolutely no residue left in the intestines using the cleansing agent, and particularly when the examination is carried out at a hospital, this requires the examiner to check multiple times, which creates an issue of time and mental burdens for both the examinee (patient) and the examiner. Furthermore, there are cases where an accurate judgment cannot be made by the examinee checking for himself.

[0006] Furthermore, in cases where the pre-examination work for colonoscopy involves assistance with excretion, this may violate the privacy of the examinee and impose a mental and time burden on the examinee or the examiner, and support that can reduce the workload is needed. In particular, a system that maintains the privacy of the examinee and encourages work support is needed. In some cases, the examinee performs the pre-examination work by himself, but in cases where the examiner assists with excretion, the examinee enters the toilet with the examiner to visually check the excretion of the examinee. Being observed while excreting is embarrassing for the examinee and also imposes a mental burden on the examiner.

[0007] In order to solve such problems in pre-examination work, a technology is also known in which images of excrement are used to determine whether it is safe for the subject to undergo endoscopic examination. For example, Patent Document 2 describes an endoscopic work support device that aims to improve the efficiency of work by medical professionals regarding pre-treatment for lower gastrointestinal endoscopy.

[0008] The endoscopic operation support device described in Patent Document 2 includes an image acquisition unit that acquires an image of an excretion object of a patient to whom a pretreatment drug for a lower gastrointestinal endoscopy has been administered, and an image analysis unit that analyzes the captured image. The endoscopic operation support device further includes a determination unit that determines whether or not the patient is in a state in which a lower gastrointestinal endoscopy is possible based on a result of the image analysis, and a notification unit that notifies a terminal device of the determination result via a network. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] JP 2020-187693 A [Patent Document 2] JP 2016-066301 A Summary of the Invention [Problem to be solved by the invention]

[0010] In the technology described in Patent Document 1, image information of a target image capturing an image of the internal space of a toilet bowl after excretion is acquired, a segmented image including a partial region of the target image is generated, and the entire image and the partial image are input to a trained model and another trained model, respectively, to perform the first and second estimations. However, since the partial region of the technology described in Patent Document 1 is determined according to the shape of the toilet bowl, and considering that foreign objects other than excrement are also captured, the technology can only handle toilet bowls with a common shape. If the technology described in Patent Document 1 is applied to a toilet bowl with a shape different from the common shape, accurate estimation cannot be performed.

[0011] In other words, the technology described in Patent Document 1 cannot handle toilet bowls of various shapes that are in circulation, and in order to do so, it is necessary to build and implement two trained models for each shape of toilet bowl. Furthermore, this problem becomes even more complicated when it is taken into consideration that when a buttocks washer is attached to the toilet seat of a toilet, the buttocks washer is reflected in the image. In other words, in order to perform accurate estimation in accordance with sets of toilet bowls and toilet seats of various shapes, the technology described in Patent Document 1 requires building and implementing two trained models for each set.

[0012] Therefore, there is a need for the development of an excrement analyzer that can accommodate toilet bowls and toilet seats of various shapes and that can accurately analyze imaged excrement.

[0013] The technology described in Patent Document 2 detects the ratio of black, brown, and intermediate color pixels to all pixels in the analysis area, and if the ratio exceeds a predetermined ratio, it determines that solid matter is mixed in the excrement and that a lower endoscopy examination is not possible. Therefore, the technology described in Patent Document 2 does not assume detailed analysis of excrement in a toilet bowl, nor is it a technology aimed at improving the accuracy of excrement.

[0014] Furthermore, the technology described in Patent Document 2 requires not only that the patient or medical staff manually photograph the excrement in the toilet bowl using a terminal device to obtain an image to be analyzed, but also that a mark indicating the photographing range be formed in the water-filled portion of the toilet bowl. Therefore, the technology described in Patent Document 2 requires not only time and effort for photographing, but also that it can only be used for dedicated toilet bowls with marks formed in advance, and cannot be used for various toilet bowls in circulation. It is conceivable to manually form the mark by sticker application or painting after the toilet bowl is manufactured, but it is difficult to form the mark in a position that allows accurate judgment for each of the various shapes of toilet bowls, and forming the mark also requires time and effort.

[0015] The present disclosure has been made to solve the above-mentioned problems, and aims to provide an excrement analysis device, an excrement analysis method, and a program that can accommodate toilets and toilet seats of various shapes and that can accurately analyze imaged excrement. [Means for solving the problem]

[0016] A first aspect of the present disclosure provides an excrement analysis device that includes an input unit that inputs imaging data captured by an imaging device that is installed so that the imaging range includes the excrement excretion range in a toilet bowl. The excrement analysis device also includes a classification unit that performs semantic segmentation on the imaging data input by the input unit to classify the imaged substance on a pixel-by-pixel basis, and an output unit that outputs the classification result by the classification unit.

[0017] A second aspect of the present disclosure provides an excrement analysis method, which inputs imaging data captured by an imaging device installed so as to include an excrement excretion area in a toilet bowl within an imaging range, performs a classification process on the input imaging data to classify imaged substances on a pixel-by-pixel basis using semantic segmentation, and outputs a classification result from the classification process.

[0018] A program according to a third aspect of the present disclosure is a program for causing a computer to execute an excrement analysis process. The excrement analysis process inputs imaging data captured by an imaging device installed so as to include an excrement excretion area in a toilet bowl within an imaging range. The excrement analysis process executes a classification process for classifying imaged substances on a pixel-by-pixel basis using semantic segmentation for the input imaging data, and outputs a classification result from the classification process. Effect of the Invention

[0019] The present disclosure makes it possible to provide an excrement analysis device, an excrement analysis method, and a program that can accommodate toilet bowls and toilet seats of various shapes and that can accurately analyze imaged excrement. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing an example of the configuration of an excrement analyzer according to a first embodiment. [Diagram 2] FIG. 11 is a diagram showing an example of the configuration of an excrement analyzing system according to a second embodiment. [Diagram 3] 3 is a block diagram showing an example of the configuration of an excrement analyzer in the excrement analysis system of FIG. 2. FIG. [Figure 4] FIG. 3 is a conceptual diagram for explaining an example of processing in the excrement analyzing system of FIG. 2. [Diagram 5] 3 is a diagram for explaining an example of processing in an excrement analyzer in the excrement analyzing system of FIG. 2. FIG. [Figure 6] 3 is a diagram for explaining an example of processing in an excrement analyzer in the excrement analyzing system of FIG. 2. FIG. [Figure 7] FIG. 7 is a diagram showing an example of a stool behavior analysis included in the example of the process of FIG. 6. [Figure 8] 3 is a diagram for explaining an example of processing in an excrement analyzer in the excrement analyzing system of FIG. 2. FIG. [Figure 9] 3 is a diagram for explaining an example of processing in an excrement analyzer in the excrement analyzing system of FIG. 2. FIG. [Figure 10] 3 is a flow diagram for explaining an example of processing in an excrement analyzer in the excrement analyzing system of FIG. 2. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of an excrement analyzer (a device for checking a condition before a colonoscopic examination) according to a third embodiment. [Figure 12] 12 is a flow chart for explaining a processing example in the status checking device of FIG. 11. [Figure 13] 13 is a conceptual diagram for explaining a processing example in a status confirmation device according to a fourth embodiment. FIG. [Figure 14] 14 is a diagram for explaining an example of processing in the status checking device of FIG. 13. [Figure 15] 14 is a diagram for explaining an example of processing in the status checking device of FIG. 13. [Figure 16] 14 is a diagram for explaining an example of processing in the status checking device of FIG. 13. [Figure 17] 14 is a flow chart for explaining an example of processing in the status checking device of FIG. 13. [Figure 18] 14 is a flow chart for explaining an example of processing in the status checking device of FIG. 13. [Figure 19] This is a flow diagram continuing from FIG. 18. [Figure 20] FIG. 19 is a diagram showing an example of a stool color analysis included in a secondary analysis in the example processing of FIG. 18. [Figure 21] FIG. 2 illustrates an example of a hardware configuration of the apparatus. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] Hereinafter, the embodiments will be described with reference to the drawings. In the embodiments, the same or equivalent elements may be given the same reference numerals, and duplicated descriptions will be omitted as appropriate. In addition, the reference numerals and names of elements in the drawings are conveniently given to each element as an example to aid understanding, and they do not limit the contents of the present disclosure in any way. In addition, although some of the drawings described below depict unidirectional and bidirectional arrows, each arrow simply indicates the direction of a certain signal (data) flow, and does not exclude bidirectionality or unidirectionality.

[0022] <Embodiment 1> The excrement analyzer according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the excrement analyzer according to the first embodiment.

[0023] As shown in FIG. 1, an excrement analyzer 1 according to this embodiment can include an input unit 1a, a classification unit 1b, and an output unit 1c.

[0024] The input unit 1a inputs imaging data (image data) captured by an imaging device (hereinafter, exemplified as a camera) installed so as to include within its imaging range the excrement excretion area in the toilet bowl. This imaging data is used by the excrement analyzer 1 to analyze the contents of the excrement and obtain information about it.

[0025] Therefore, the excrement analysis device 1 is connected to or includes a camera installed in this manner. However, it can be said that it is preferable for the excrement analysis device 1 to include a camera in terms of integrating the device and preventing the image data from leaking to other devices. The camera is not limited to a visible light camera, but may be an infrared camera or the like, and may also be a video camera as long as it can extract still images. When the camera is connected to the outside of the excrement analysis device 1, it may be connected to the input unit 1a. This image data may include additional information (attached information) such as the image capture date and time, the image capture conditions, etc. The image capture conditions may include, for example, the resolution if the resolution can be set, and the zoom magnification if the camera has a zoom function.

[0026] The excretion range can be an area including the water-filled part of the toilet bowl, and can also be called the expected excretion range. By installing the camera so that such an excretion range is included in the imaging range, the captured imaging data will include excrement and the like as a subject. Of course, the excretion range is preferably an area that does not include the user (a toilet user), and the camera is preferably installed so that the lens of the camera is not visible to the user. In addition, when the excretion analyzer 1 is used in a hospital or a care facility, for example, the user is mainly a patient or other person who requires care. In addition, examples of the caregiver include a caregiver, and in some cases, a doctor, but it is not limited to a caregiver, and it may also be an assistant, or anyone else.

[0027] The classification unit 1b classifies the imaged material in pixel units (pixel units) using semantic segmentation for the imaging data (analysis target data) input by the input unit 1a. Semantic segmentation refers to a deep learning algorithm that classifies all pixels in an image and associates labels and categories with all pixels. The following explanation is based on the premise that labels are associated with pixels, but it is also possible to associate categories with pixels, or to associate labels and categories to which multiple labels belong with pixels. Examples of semantic segmentation include, but are not limited to, FCN (Fully Convolutional Network), U-net, SegNet, etc.

[0028] Note that the pixel unit basically means, but is not limited to, one pixel unit. For example, data obtained by performing a filter process on the imaging data in the pre-processing may be input, and the classification unit 1b may classify the imaged substances in units of multiple pixels in the original imaging data for the input data to be analyzed.

[0029] The imaged substance is a substance imaged by the camera, and depending on the installation position and purpose of the camera, the imaged substance may include feces (also called stool or feces). Therefore, when a pixel corresponds to feces, the classification unit 1b performs a process of classifying the pixel as feces, that is, a process of associating a label indicating feces. As will be described later in the second embodiment, feces can be classified into a plurality of feces types. In this case, when such classification is performed, the classification unit 1b can perform a process of classifying the pixel as feces and corresponding to a certain feces type, that is, a process of associating a label indicating the feces type. In this case, for example, it is possible to associate the pixel with the category of feces and also with a label indicating the feces type.

[0030] Other substances to be imaged may include urine (urination), urine drips, toilet paper, and buttocks washer. Similarly, when a pixel corresponds to urine, urine drips, toilet paper, or buttocks washer, the classification unit 1b performs a process of classifying the pixel as urine, urine drips, toilet paper, or buttocks washer, respectively. That is, the classification unit 1b performs a process of associating a label or category indicating urine, urine drips, toilet paper, or buttocks washer, respectively. In addition, the color of feces and urine can also be classified, and in that case, a label indicating the corresponding feces color or urine color can be associated with the pixel. Note that a buttocks washer is a device for washing the buttocks, and may be called a buttocks washer, buttocks washer, etc., and will be described below as a buttocks washer. A buttocks washer may be included in a warm water washing toilet seat such as Washlet (registered trademark) that has a function of flushing a toilet.

[0031] As described above, the classification unit 1b classifies the imaged material in units of pixels using semantic segmentation, and such classification can divide the image of the imaging range into classifications (i.e., labels). Therefore, semantic segmentation can also be called an image region division algorithm. Note that the classification unit 1b can also be called an analysis unit because it analyzes the imaging data by performing such classification. The classification unit 1b analyzes the imaging data inputted by the input unit 1a in real time, more specifically, it can classify each region in the image in a single process for one input image data, so the analysis performed here corresponds to real-time analysis (real-time classification).

[0032] Hereinafter, information obtained from the excrement analyzer 1 is also referred to as excretion information. In this embodiment, the excretion information includes the classification results of the above-mentioned labels, etc., as information indicating the contents of the excretion. However, the excretion information also implicitly includes the shapes of the areas classified into each label, which are shown as the entire image data, and can also include information that separately specifies the shapes of such areas (e.g., the shape of stool, etc.). In addition, the excretion information can include or be added with additional information such as date and time information indicating the image capture date and time or acquisition date and time of the image capture data, and the image capture conditions.

[0033] The output unit 1c outputs the classification result from the classification unit 1b or excretion information including the classification result. The excrement analyzer 1 may include a communication unit (not shown) as a part of the output unit 1c, and this communication unit may be configured, for example, by a wired or wireless communication interface.

[0034] The format of the classification result output from the output unit 1c does not matter, and only a part of the classification result can be output. For example, if the classification result indicates that a foreign object has been mixed in, only information indicating the presence of the foreign object can be output as the classification result. In addition, the output destination of the classification result may be determined in advance, and the specific output destination does not matter, and the output destination is not limited to one location.

[0035] The output destination of the classification result may be, for example, a terminal device carried by a monitor who monitors users of the toilet. In this case, the classification result is output to the terminal device used by the monitor as notification information for the monitor. The notification information may include the classification result itself, but may also be only information with content predetermined according to the classification result (for example, excretion notification information indicating that excretion has occurred). Note that the terminal device used by the monitor is not limited to a terminal device used by an individual monitor such as a caregiver, but may be, for example, a terminal device installed in a monitoring station such as a nurse's station, and this terminal device may function as an alarm device. In addition, when the output destination of the classification result is a terminal device used by the monitor, the direct output destination may be a server device capable of receiving notification information and transferring the notification to the terminal device.

[0036] In this way, the classification result can be output as notification information to a monitor or the like, but can also be output to a server device that collects and manages excretion information, for example, as excretion information for a caregiver to create a diary about the care recipient as a toilet user. This server device can be, for example, a cloud server device. In the case of a facility such as a hospital, the server device can be installed within the facility, and in the case of personal use, it can be installed in a personal home or an apartment building.

[0037] The excrement analyzer 1 may include a control unit (not shown) for controlling the entire excrement analyzer 1, and this control unit may include a part of the above-mentioned input unit 1a, classification unit 1b, and output unit 1c. This control unit may be realized, for example, by a CPU (Central Processing Unit), a working memory, and a non-volatile storage device storing a program. This program may be a program for causing the CPU to execute the processing of each unit 1a to 1c. Furthermore, the imaging data inputted by the input unit 1a may be temporarily stored in this storage device and read out at the time of classification by the classification unit 1b, but the imaging data may also be temporarily stored in another storage device. Furthermore, the control unit provided in the excrement analyzer 1 may be realized, for example, by an integrated circuit. As this integrated circuit, an FPGA (Field Programmable Gate Array) may also be adopted.

[0038] The classification in the classification unit 1b can be started by a simple detection process with a smaller load than classification. For example, the image data input by the input unit 1a or the image data output by the input unit 1a to the classification unit 1b at the subsequent stage can be data when an object is detected as a subject in the excretion area or a change such as a change in the color of the standing water is detected. These detections can be performed by the camera or the input unit 1a, for example, by capturing images with the camera all the time or at regular intervals, and using the captured image data obtained there. Alternatively, the image can be captured based on the user detection result from a separately provided user detection sensor (such as a load sensor provided on the toilet seat or other human sensor), and the captured image data at that time can be selected as data to be output by the camera or the input unit 1a to the subsequent stage.

[0039] As described above, the excrement analyzer 1 is a device that analyzes the contents of excrement excreted in the toilet by classification and outputs excretion information including at least the classification results, and can also be called a toilet excretion analyzer or an excretion information acquisition device. The excrement analyzer 1 can be a device that functions as an edge toilet sensor in an excrement analysis system (analysis system) configured on a network including a monitor's terminal device and an external server device.

[0040] In the excrement analyzer 1 configured as described above, as long as the imaging range includes the range in which excrement is excreted, it is possible to accurately classify the imaged substance and output the classification results without precisely determining the installation positions of the camera and the sensor including the camera (toilet sensor). In other words, the excrement analyzer 1 can accurately classify the imaged substance and output the classification results even by attaching the camera and toilet sensor to various types of toilet bowls and toilet seats that are on the market. Therefore, the excrement analyzer 1 according to this embodiment can be adapted to toilet bowls and toilet seats of various shapes, and can accurately analyze the imaged excrement.

[0041] Furthermore, the excrement analyzer 1 does not need to transmit imaging data acquired from the camera or other image data to an external location such as the cloud, and excrement analysis can be performed by the excrement analyzer 1 installed in, for example, a toilet. In other words, all images and videos used in the analysis by the excrement analyzer 1 are processed within the excrement analyzer 1, and the excrement analyzer 1 can be configured so that the images and videos are not transmitted to the outside. Therefore, it can be said that the excrement analyzer 1 can be configured to reduce the mental burden on the user regarding privacy.

[0042] Furthermore, the excrement analyzer 1 can accurately collect information indicating the contents of the excrement excreted into the toilet bowl without having to ask the toilet user while respecting the privacy of the toilet user, and can also respond to situations where an immediate notification to the monitor is required. In other words, the excrement analyzer 1 can realize both consideration of the privacy of the toilet user and notification and recording, while improvements are being made to install sensors in toilets to reduce the burden of excretion management in monitoring such as nursing care. The notification and recording here are notification of an immediate event at the monitoring site such as a nursing care site based on the classification result and recording of accurate information. Therefore, the excrement analyzer 1 can be configured to reduce the physical and mental burden on the monitor and toilet user.

[0043] <Embodiment 2> The second embodiment will be described with reference to Figures 2 to 10, focusing on the differences from the first embodiment, but the various examples described in the first embodiment can also be applied. Figure 2 is a diagram showing an example of the configuration of an excrement analysis system according to the second embodiment, and Figure 3 is a block diagram showing an example of the configuration of an excrement analysis device in the excrement analysis system of Figure 2.

[0044] The excrement analysis system according to this embodiment (hereinafter, this system) can include an excrement analysis device 10 attached to a toilet 20, a terminal device 50 used by a caregiver, and a server device (hereinafter, server) 40. The caregiver can be considered an example of a monitor who monitors toilet users.

[0045] The excrement analyzer 10 is an example of the excrement analyzer 1, and is illustrated as a toilet-mounted device, but it may be any device that is installed inside a toilet. The toilet 20 may be provided with a toilet seat 22 equipped with a warm water flushing function for user washing, and a toilet seat cover 23 for covering the toilet seat 22, on its main body 21. The excrement analyzer 10 and the toilet 20 may constitute a toilet 30 with an analysis function that is equipped with a function for outputting analysis results including at least classification results.

[0046] The shape of the excrement analyzer 10 is not limited to the shape shown in FIG. 2, and it may be configured such that all or part of its functions are embedded in the toilet seat 22 or the like. The excrement analyzer 10 may also be configured such that the second external box 11 described later is separated from the box-to-box connector 12 and placed on the side or back side of the toilet bowl 20. Some of the functions of the excrement analyzer 10 may be provided on the toilet seat 22 side. For example, instead of providing the excrement analyzer 10 with the distance sensor 16a described later, a weight sensor may be provided on the toilet seat 22, and the excrement analyzer 10 may receive information from the weight sensor by wireless or wired communication. This weight sensor may be provided in the box-to-box connector 12 described later, or may simply be a pressure sensor that detects pressure above a certain level. In addition, instead of providing the first camera 16b described below in the excrement analyzer 10, a camera can be provided on the toilet seat 22 side, and the excrement analyzer 10 can receive the image data from the camera via wireless or wired communication.

[0047] The server device (server) 40 and the terminal device 50 can be wirelessly connected to the excrement analyzing device 10, and the terminal device 50 can be wirelessly connected to the server 40. These connections can be made within one wireless LAN (Local Area Network), for example, but other connection forms such as connection via separate networks can also be adopted. Also, some or all of these connections may be made by wire.

[0048] In this system connected in this manner, the excrement analyzer 10 outputs notification information according to the classification result by transmitting it to the terminal device 50, and outputs excretion information including the classification result by transmitting it to the server 40. The terminal device 50 is a terminal device held by a caregiver of the toilet user, and can be a portable terminal device, but may also be a stationary device such as a PC (Personal Computer). In the former case, the terminal device 50 can be a mobile phone (including what is called a smartphone), a tablet, a mobile PC, etc. The server 40 can be a device that collects and manages excretion information, and stores the excretion information received from the excretion analyzer 10 in a state that can be viewed from the terminal device 50.

[0049] The server 40 may also include a control unit 41 for controlling the entire server, a storage unit 42 for storing excretion information in, for example, a database (DB) format, and a communication unit (not shown) for making the above-mentioned connection. The control unit 41 controls the storage of the excretion information transmitted from the excreta analyzer 10 in the storage unit 42, and controls browsing from the terminal device 50. The control unit 41 may be realized, for example, by a CPU, a working memory, and a non-volatile storage device storing a program. This storage device may also serve as the storage unit 42, and the program may be a program for causing the CPU to realize the functions of the server 40. The control unit 41 may also be realized, for example, by an integrated circuit.

[0050] In addition, the terminal device 50 may include a control unit for controlling the entire device, a storage unit, and a communication unit for making the above-mentioned connection, although these are not shown. This control unit, like the control unit 41, may be realized by, for example, a CPU, a working memory, a non-volatile storage device storing a program, or an integrated circuit. In addition, the program stored in the storage device may be a program for causing the CPU to realize the functions of the terminal device 50.

[0051] Moreover, the terminal device 50 preferably includes a diary generation unit that generates an excretion diary based on the notification information received from the excretion analyzer 10 and the excretion information stored in the server 40. This diary generation unit can be installed, for example, by incorporating a diary creation application program into the terminal device 50. The created excretion diary can be stored in an internal memory unit. The diary creation unit can also be installed as a part of a care record unit that creates a care record. The care record creation unit can also be realized by incorporating an application program into the terminal device 50.

[0052] Next, a detailed example of the excrement analysis device 10 will be described. The excrement analysis device 10 can be composed of two devices, for example, as shown in Fig. 2 and Fig. 3. More specifically, the excrement analysis device 10 can have two boxes, for example, a first external box 13 and a second external box 11, as its housing. The excrement analysis device 10 can also have an inter-box connection part (inter-box connection structure) 12 that connects between the first external box 13 and the second external box 11. The first external box 13 and the second external box 11 can be connected by an interface, a specific example of which is shown in Fig. 3.

[0053] The excrement analysis device 10 in this example can be installed in the main body 21 of the toilet 20, for example, as follows. That is, the excrement analysis device 10 can be installed in the toilet 20 by placing the inter-box connector 12 on the edge of the main body 21 so that the first external box 13 is disposed on the inside of the main body 21 (the side where the excrement is excreted) and the second external box 11 is disposed on the outside of the main body 21.

[0054] For example, a distance sensor 16a and a first camera 16b can be stored in the first external box 13. As will be described later, the distance sensor 16a is an example of a seating sensor that detects a person sitting on the toilet seat 22, and the first camera 16b is a camera that captures an image of excrement and obtains the image data inputted by the input unit 1a in FIG.

[0055] The second external box 11 includes a device that performs real-time analysis based on the imaging data (image data) captured by the first camera 16b. The second external box 11 also includes a communication device 14 that, under the control of the device, notifies the caregiver when an event occurs and transmits the analysis result to the server 40.

[0056] For example, the second external box 11 can accommodate a CPU 11a, a connector 11b, USB I / Fs 11c and 11d, a WiFi module 14a, a Bluetooth module 14b, a human sensor 15a, and a second camera 15b. Note that USB is an abbreviation for Universal Serial Bus, and USB, WiFi, and Bluetooth are all registered trademarks (hereinafter the same). The communication device 14 is exemplified by the modules 14a and 14b, and the CPU 11a executes real-time analysis while transmitting and receiving data to and from other parts via the elements 11b, 11c, and 11d as necessary. Note that in this example, the CPU 11a is described as also having a memory for temporarily storing imaging data. Also, the communication device 14 is not limited to the communication module of the exemplified standard, and may be wireless or wired. As the communication module, for example, various types such as a Long Term Evolution (LTE) communication module, a fifth generation mobile communication module, and a Low Power, Wide Area (LPWA) communication module can be mentioned.

[0057] As shown in FIG. 3, the first external box 13 and the second external box 11 are connected by an interface exemplified by a connector 11b and a USB I / F 11c, and the connection wire is provided inside the inter-box connection section 12 to form a single excrement analysis device 10.

[0058] The first external box 13 will now be described. Distance sensor 16a is a sensor that measures the distance to an object (the buttocks of the user of toilet bowl 20) and detects that the user has sat on toilet seat 22, and detects that the object has sat on toilet seat 22 when a certain period of time has elapsed after the threshold value is exceeded. Distance sensor 16a also detects that the user has left the toilet seat 22 when the distance to the object changes after sitting down.

[0059] The distance sensor 16a may be, for example, an infrared sensor, an ultrasonic sensor, an optical sensor, or the like. When the distance sensor 16a is an optical sensor, a transmitting / receiving element may be disposed so as to transmit and receive light (not limited to visible light) through a hole provided in the first external box 13. The transmitting / receiving element may be configured as a transmitting element and a receiving element separately, or may be integrated. The distance sensor 16a is connected to the CPU 11a via the connector 11b, and is capable of transmitting the detection result to the CPU 11a.

[0060] The first camera 16b is an example of a camera that captures the imaging data input to the input unit 1a in Fig. 1, and can be an optical camera with a lens portion disposed in a hole provided in the first external box 13. As described in the first embodiment, the first camera 16b is installed so that the imaging range includes the excrement excretion range in the toilet bowl 20. The first camera 16b is connected to the CPU 11a via the USB I / F 11c, and transmits the imaging data to the CPU 11a.

[0061] The second external box 11 will now be described. The CPU 11a is an example of a main control unit of the excrement analyzer 10, and controls the entire excrement analyzer 10. As described below, real-time analysis is performed by the CPU 11a. The connector 11b connects the human presence sensor 15a and the distance sensor 16a to the CPU 11a. The USB I / F 11c connects the first camera 16b to the CPU 11a, and the USB I / F 11d connects the second camera 15b to the CPU 11a.

[0062] The human presence sensor 15a is a sensor that detects the presence of a person (entry or exit of a person) in a specific area (a measurement area range of the human presence sensor 15a), and this specific area can be set as an area in which entry and exit to the restroom can be determined. The human presence sensor 15a can adopt any detection method, for example, an infrared sensor, an ultrasonic sensor, an optical sensor, etc. The human presence sensor 15a is connected to the CPU 11a via the connector 11b, and when it detects a person in the specific area, it transmits the detection result to the CPU 11a.

[0063] Based on the detection result, the CPU 11a can control the operation of the distance sensor 16a and the first camera 16b. For example, the CPU 11a can perform processing such as activating the distance sensor 16a when the detection result indicates that a person has entered the room, and activating the first camera 16b when the distance sensor 16a detects that a person is seated.

[0064] The second camera 15b may be an optical camera with a lens portion disposed in a hole provided in the second external box 11, and is an example of a camera that captures a facial image of a user to obtain facial image data in order to identify the user of the toilet. The second camera 15b may be installed on the toilet bowl 20 so that the user's face is included in the imaging range, but may also be installed in the toilet room in which the toilet bowl 20 is installed.

[0065] The Bluetooth module 14b is an example of a receiver that receives identification data for identifying a user from a Bluetooth tag held by the user, and can be replaced with a module based on other short-range communication standards. The Bluetooth tag held by the user can have a different ID for each user and can be embedded in a wristband or the like so that the user can keep it.

[0066] The WiFi module 14a is an example of a communication device that transmits various data including notification information to the terminal device 50 and various data including excretion information to the server 40, and can be replaced with a module that adopts other communication standards. The face image data acquired by the second camera 15b and the identification data acquired by the Bluetooth module 14b can be added to or embedded in the notification information and excretion information and then transmitted to the terminal device 50 and the server 40, respectively. The terminal device 50 and the server 40 that receive the face image data can perform face recognition processing based on the face image data and identify the user. However, the excrement analyzer 10 can be configured not to transmit face image data. In that case, if the face recognition processing is performed by the CPU 11a, user identification by face recognition is possible, and identification data indicating the result can be the subject of transmission.

[0067] The USB I / F 11c, or the CPU 11a and the USB I / F 11c, may be an example of the input unit 1a in FIG. 1, which inputs image data captured by the first camera 16b. The CPU 11a and the WiFi module 14a may be an example of the classification unit 1b in FIG. 1. The CPU 11a analyzes the image data in real time, and can transmit notification information to the terminal device 50 and excretion information to the server 40 via the WiFi module 14a. This real-time analysis classifies the imaged substance on a pixel-by-pixel basis using semantic segmentation, as described for the classification unit 1b.

[0068] The notification information and excretion information can also be transmitted via the Bluetooth module 14b. In this way, the notification information and excretion information can be transmitted to the terminal device 50 and the server 40 connected to the excretion analyzer 10 via a network or a short-distance wireless communication network, respectively. Of course, the notification information may be transmitted via the server 40 or another server as long as it is transferred to the terminal device 50. The transmitted notification information and excretion information are information according to the classification result and information including the classification result, respectively, and neither of them includes the imaging data itself. This not only reduces the mental burden on the user regarding privacy, but also reduces the amount of transmitted data. In particular, reducing the amount of data is beneficial in an environment with a poor network bandwidth. Note that additional information of the imaging data (such as the imaging date and time) may be transmitted by being included in the notification information and excretion information.

[0069] In the illustrated example, a smartphone is used as the terminal device 50. However, the notification destination (transmission destination) may be, in addition to or instead of a smartphone, a notification device of a nurse call system, another terminal device owned by a caregiver, an intercom (intercommunication), etc. Examples of the other terminal devices include a PHS (Personal Handy-phone System).

[0070] The real-time analysis will be described with reference to Fig. 4 to Fig. 9. Fig. 4 is a conceptual diagram for explaining a processing example in this system, and Fig. 5 to Fig. 9 are diagrams for explaining a processing example in the excrement analyzer 10. Here, Fig. 6 is a diagram showing an example of a classification image, Fig. 7 is a diagram showing an example of a stool nature analysis (stool nature classification) included in the processing example in Fig. 6, and Fig. 8 and Fig. 9 are diagrams showing other examples of the classification image.

[0071] As shown in Fig. 4, an example will be given in which a user P uses a toilet bowl 30 with an analysis function installed in a toilet, and a caregiver C of the user P monitors the state of the user P. When the user P uses the toilet bowl 30 with an analysis function, the CPU 11a detects that the user has sat on the toilet seat based on the detection result from the distance sensor 16a, which functions as a seating sensor. When the CPU 11a detects that the user has sat on the toilet seat, it instructs the first camera 16b to start capturing images, and performs real-time analysis 31 based on the captured image data.

[0072] As the real-time analysis 31, the CPU 11a classifies the imaged substance pixel by pixel using semantic segmentation, and can obtain a classification result. The number of classifications (number of labels) is not important. For example, the CPU 11a can classify the imaged substance into any of excrement, foreign body, and other substances for each pixel. The CPU 11a can also classify the excrement into any of feces, urine, and urine drips, or into any of feces, urine, feces and urine (feces + urine), and urine drips. In other words, the CPU 11a can classify the imaged substance into any of feces, urine, urine drips, foreign body, and other substances, or into any of feces, urine, feces + urine, urine drips, foreign body, and other substances, for each pixel.

[0073] Here, a foreign object can refer to a substance that is not permitted to be disposed of in the toilet bowl 20. The foreign object can be liquid or solid, and can include, for example, one or more of a urine pad, a diaper, a toilet paper roll, etc. In other words, when a pixel is labeled as a substance that constitutes such an object, it means that a foreign object is present.

[0074] The other substances include at least one of a bidet, toilet paper, and substances left after excrement is flushed (which may be only water). The other substances may be classified as one label, but may also be classified into three labels, for example, a label showing a bidet, a label showing toilet paper, and a label showing substances left after excrement is flushed.

[0075] A foreign object can also be defined as a substance other than feces and urine, excluding the toilet bowl and the liquid used to clean the toilet bowl. When using this definition, a foreign object can be a liquid or solid other than feces and urine, and can include, for example, one or more of a urine pad, a diaper, and a toilet paper roll. In addition, the foreign object or the above other substance can include, for example, one or more of vomit, bloody stool, and vomiting of blood (hematemesis).

[0076] Note that as long as the definitions of foreign bodies and the above-mentioned other substances do not overlap, the method of distinction is not limited to the above-mentioned example, and the method of distinction can be determined, for example, depending on the type of notification to caregiver C. Of course, all of the substances given as examples of foreign bodies and the above-mentioned other substances can be classified as individual substance labels rather than as foreign bodies or the above-mentioned other substances.

[0077] The CPU 11a can also perform at least one of the following: classifying stool into a plurality of predetermined stool textures, classifying stool into a plurality of predetermined stool colors, and classifying urine into a plurality of predetermined urine colors. Here, the stool texture can indicate the shape or form of stool, and can adopt the classification exemplified by the Bristol scale 1 to 7, for example.

[0078] Then, when the result of the real-time analysis 31 indicates that a foreign object is detected and an immediate notification to the caregiver is required, the CPU 11a transmits notification information (real-time notification 32) to the terminal device 50 of the caregiver C located away from the toilet via the WiFi module 14a. In this way, the CPU 11a can transmit foreign object information (foreign object information indicating a foreign object determination result) indicating whether or not a foreign object is included to the terminal device 50. This foreign object information is output as at least a part of the notification information, and the CPU 11a can determine whether or not a foreign object is included (foreign object determination) based on, for example, whether or not there is a pixel labeled as a foreign object (or whether there is a predetermined number of pixels). In what situation the notification information is output, not limited to a foreign object, can be set in advance, and the setting can also be configured to be changeable from the terminal device 50 or the like. For example, when the classification result is classified as excrement, the CPU 11a can output an excretion notification to the monitor to the terminal device 50 or the like.

[0079] Moreover, the other substances may include at least a posterior washer. When the pixel classification result is classified as a posterior washer, or when a predetermined number or more of pixels classified as a posterior washer are present in succession, the CPU 11a can stop the subsequent classification process and output an excretion completion notification to the monitor. The subsequent classification process may be, for example, a classification process for the next pixel, or other notification process other than the excretion completion notification. In this way, the excretion analyzer 10 can be configured to detect the end of excretion by finding the posterior washer. With this configuration, it is possible to eliminate the possibility that subsequent dripping of water will mix with the washing water from the posterior washer, thereby reducing the accuracy of the classification result. Furthermore, by adopting a configuration that can accurately classify the posterior washer in such imaging data, not only can the caregiver be accurately notified of the completion of excretion, but also erroneous detection such as determining dripping of washing liquid as urine during detection of the posterior washer can be eliminated.

[0080] By the above-mentioned notification, the caregiver C is relieved from the situation of having to be constantly present with the user P during excretion, and the real-time notification 32 enables the caregiver C to rush to the scene in an emergency 51. Here, the real-time notification 32 sent does not include imaging data.

[0081] Furthermore, the CPU 11a executes transmission 34 of real-time analysis results regarding the excretion information including the results (classification results) of the real-time analysis 31 to the server 40 via the WiFi module 14a. In this manner, the analysis results of the real-time analysis 31 are transmitted to the server 40 by executing the analysis result transmission 34 using the communication function. The analysis result transmission 34 is transmitted without including the imaging data. The information recorded in the server 40 can be used as a subject for reference 52 by the caregiver C for creating 53 a care record (excretion diary) or for future care support.

[0082] Furthermore, the caregiver C of the user P creates 53 a care record (excretion diary) for the user P on the terminal device 50 while appropriately referring 52 to the excretion information of the user P stored in the server 40 based on the received notification information. The excretion diary can be created as a part of the care record. In this way, the excretion diary for each user can be recorded on the terminal device 50. The format of the excretion diary is not important.

[0083] The CPU 11a can also output the classification result as information including a classification image drawn with a different color for each classification (each label). Such a classification image can be output to the terminal device 50 as notification information or as a part of notification information, or as excretion information or as a part of excretion information for creating a later excretion diary. An example of the classification image will be described later with reference to FIG. 6.

[0084] The CPU 11a can also perform classification in stages. For example, when a substance classified as excrement is present, the CPU 11a outputs an excretion notification to the terminal device 50 or the like. After outputting the excretion notification, the CPU 11a classifies each pixel classified as excrement into feces, urine, or urine dripping, or into feces, urine, feces + urine, or urine dripping, and can also perform detailed classification. The detailed classification here can include at least one of classification of feces into a plurality of predetermined feces types, classification of feces into a plurality of predetermined feces colors, and classification of urine into a plurality of predetermined urine colors.

[0085] Here, with reference to FIG. 5, an example of the input, method, and output of the real-time analysis 31 will be described, including an example of classification. The real-time analysis is an analysis that requires real-time performance, such as notification to the caregiver C. The real-time analysis inputs image data (image data) captured by the first camera 16b, classifies the data into one of the following five types by Deep Learning (DL), and outputs the classification result. Here, semantic segmentation (image region division algorithm) is used as DL. The classification result can be associated with a label corresponding to the type. The five types exemplified here are foreign bodies (diapers, urine leakage pads, etc.), feces (fecal matter), urine, urine dripping, and a posterior washer, and if fecal matter is classified into eight types, there will be a total of 12 types. These classification types are examples of events that trigger real-time notification. For example, if it is classified as a posterior washer, it can be determined that excretion has been completed. In addition, similar to a rear washing machine, other classifications that can be used to determine that excretion has been completed include toilet paper (or a certain amount or more of toilet paper) and materials remaining after excretion has been completed, and these classifications can be included.

[0086] In addition, the DL can perform machine learning by inputting learning data with a correct answer label as correct answer data (teaching data). The resulting learning model (i.e., a trained model) can be stored inside the CPU 11a or in a storage device accessible from the CPU 11a. In real-time analysis to be performed during operation, imaging data is input to such a trained model (specifically, input for each image data, such as for each video frame) to obtain a classification result. In other words, the real-time analysis is a comparison with trained image data. In addition, a plurality of trained models may be used in the real-time analysis, and for example, a trained model different from at least one of the above six types and the other types may be used. Note that the algorithm of the trained model (machine learning algorithm) may be any algorithm belonging to semantic segmentation, and hyperparameters such as the number of layers are not important.

[0087] An example of the above-mentioned classified image will be described with reference to FIG. 6. The image Img-o shown in FIG. 6 is one piece of image data acquired by a camera. The CPU 11a classifies each pixel of the input image Img-o into urine, urine drip, feces (feces type 1), feces (feces type 2), feces (feces type 3), feces (feces type 4), feces (feces type 5), feces (feces type), water, a bidet, and a foreign object, as shown in the legend in FIG. 6. Then, a classified image Img-r can be generated as a result of the classification. The classified image Img-r can be generated by applying a color corresponding to the label classified to each pixel so as to correspond to the image Img-o. It can be seen that the classified image Img-r is an image in which the area is divided for each classification.

[0088] An example of the classification of fecal matter described above will be described with reference to Fig. 7. Fecal matter can be classified, for example, according to the Bristol scale shown in Fig. 7, and as a result of the classification, it can be classified into any of types 1 to 7 as shown in Fig. 7. "Water" in the legend of Fig. 6 can be considered to correspond to type 7.

[0089] In addition, the classified image may be an image like the example shown in FIG. 8 or the example shown in FIG. 9. As shown in FIG. 8, if the input image Img-o1 contains a pixel group Img-w indicating a rear washer, the rear washer area Img-rw in the classification image Img-r1 will be classified as something other than excrement. Also, as shown in FIG. 9, if the input image Img-o2 contains a pixel group Img-p indicating paper (toilet paper), the paper area Img-rp in the classification image Img-r2 will be classified as something other than excrement. Note that in images Img-o1 and Img-o2, the blackened areas are areas that have been subjected to a process of blackening the input image (hereinafter referred to as masking) when a human body part is detected or that is excluded from analysis by default.

[0090] Next, an example of the procedure of the real-time analysis process will be described with reference to FIG. 10. FIG. 10 is a flow diagram for explaining an example of the process in the excrement analyzer 10, and is a flow diagram showing an example of the operation contents of the real-time analysis triggered by a user entering a toilet and sitting on the toilet seat. The operation contents described here can be performed mainly by the CPU 11a while controlling each part. In addition, here, an example of the process using two trained models to which semantic segmentation is applied is given, but only one of them may be a model to which semantic segmentation is applied. In addition, only one trained model or three or more trained models can be used.

[0091] First, the presence or absence of a reaction from the distance sensor 16a functioning as a seating sensor is checked (step S1). If there is no reaction in step S1 (if NO), the system will wait until the seating sensor reacts. If the user sits down, the distance sensor 16a will react, and step S1 will result in YES. If step S1 results in YES, the terminal device 50 is notified of the user's sitting down (step S2), and real-time analysis is started (step S3). Note that if the human sensor 15a detects entry before the user sits down, the terminal device 50 can also be notified of the user's entry, and the same applies to the user's leaving the room.

[0092] In the real-time analysis, an optical camera (illustrated as the first camera 16b) takes an image of the inside of the toilet bowl, and first, it is determined whether the acquired image data (for example, image Img-o in FIG. 6) can be normally identified (step S4). Whether or not the image can be normally identified can be determined as whether or not the image can be normally classified, and the criterion for this determination is not important, but for example, an image that is totally reflected or an image that is out of focus can be determined as not being normally identified. If an abnormality is detected (NO in step S4), an abnormality notification is sent to the terminal device 50 of the caregiver (step S5). In this way, even if the image of the inside of the toilet bowl cannot be normally captured, it is preferable that notification information indicating that is sent to the terminal device 50. On the other hand, if the image can be normally identified (YES in step S4), classification is performed (step S6).

[0093] In step S6, a trained model for classifying each pixel of the image as a foreign object, excrement, a bidet, paper (toilet paper), or a substance left after excrement has been flushed is used to perform this classification. Furthermore, in step S6, it is determined from the classification result whether the detected object corresponds to (a) a foreign object, (b) excrement, or (c) a bidet or paper (or a predetermined amount or more of paper), or a substance left after excrement has been flushed. Here, from the classification result for each pixel, by obtaining, for example, the image Img-r of FIG. 6, it is possible to determine whether the detected object corresponds to (a), (b), or (c). For example, the determination of whether the amount of paper is a predetermined amount or more can be performed as a determination of whether the paper has a predetermined area or more based on the area of ​​the classified region. A trained model can also be constructed to perform such a determination.

[0094] If a foreign object is detected in step S6, a foreign object detection notification is sent to the caregiver's terminal device 50 (step S7). If excrement is detected, an excretion notification (transmission of notification information indicating that excretion has occurred) is sent to the caregiver's terminal device 50 (step S8), and excrement analysis is performed (step S9). This excrement analysis is a classification of excrement on a pixel-by-pixel basis using a trained model for classifying excrement into 10 types shown in FIG. 6. This trained model is also a model using semantic segmentation. This excrement analysis classifies each pixel into 10 types shown in the legend of FIG. 6, and the image Img-r of FIG. 6 can be obtained. After the process of step S9, the process returns to step S4 and the process for the next image is performed.

[0095] If the detection object detected in step S6 corresponds to the above (c), it is determined that excretion has been completed, and an excretion completion notification (notification information indicating that excretion has been completed) is sent to the caregiver's terminal device 50 (step S10). Upon completion of the processing in step S10, the real-time analysis ends (step S11). Alternatively, the excretion completion notification may be sent only when there is no reaction from the seating sensor. This is because the posterior washer may be used more than once. The real-time analysis also ends after steps S5 and S7.

[0096] In this way, in the processing example of FIG. 10, foreign object detection is always performed, and when a foreign object is detected, a notice is given to the caregiver, and then, when the person sits down, image capture is started, and images captured at a fixed interval are judged as feces (feces), urine, or urine dripping, as exemplified in FIG. 6. When feces (feces), urine, or urine dripping is detected, a preset label is associated, and classification is completed. In addition, the above-mentioned (c) of a buttock washer or the like is also detected, and when any of the above (c) is detected, a notice of excretion completion is sent to the terminal device 50, and the judgment of feces (feces), urine, or urine dripping is completed. In this way, by notifying the start and completion of excretion, the inclusion of a foreign object, etc., to the caregiver, etc., the caregiver, etc. can obtain this information in real time, which makes it possible to reduce physical and mental burdens.

[0097] Furthermore, the timing of transmitting the excretion information to the server 40 does not matter; for example, the information can be transmitted after the analysis in step S11 is completed, or after the processing in step S9 and before returning to step S4.

[0098] As described above, the excrement analyzer 10 can obtain real-time analysis results such as the start of excretion, detection of foreign objects, detection of excretion, and completion of excretion, as well as detailed excretion information such as feces pattern. All analysis results can be recorded in the server 40 on the cloud in a state that can be viewed from the terminal device 50, and can also be configured to transmit them to the terminal device 50. The server 40 can also be configured to accumulate the received analysis results, perform further analysis from the accumulated data, and notify the terminal device 50 of the analysis results or make them viewable from the terminal device 50.

[0099] In addition, the excrement analyzer 10 or the system including the same can be used in a private home on the assumption that there is only one user, but it is preferable to provide a function for identifying the user on the assumption that there are multiple users. This allows the system to be used suitably in private homes with multiple users, or in facilities such as hospitals and nursing homes. This function has been described using face image data acquired by the second camera 15b and identification data acquired by the Bluetooth module 14b. This makes it possible to notify the caregiver of entry notifications, exit notifications, sitting notifications, leaving notifications, excretion start notifications, and excretion completion notifications along with the user name, record excretion information for each user, and create an excretion diary and a care record including the same. In addition, the description here is based on the assumption that the user of the toilet is a person, but it is also possible to apply the system to animals kept by people.

[0100] Here, a supplementary explanation will be given regarding the excretion diary and the care record including the same. The information obtained by the real-time analysis can be used when the caregiver creates the user's excretion diary, etc. The program of the terminal device 50 can be executablely incorporated into the terminal device 50 as care software including a presentation function for presenting the notification information received from the excretion analyzer 10. The care software can have a function for automatically inputting the information transferred from the server 40 or the information obtained when accessing the server 40 into the excretion diary or the care record including the excretion diary. Such care software may be provided on the server 40, and in that case, it is sufficient to receive the notification information and excretion information from the excretion analyzer 10 and automatically input the information into the excretion diary or the care record.

[0101] As described above, the present system can achieve the effects described in embodiment 1. In particular or in addition to those effects, the present system achieves the following effects, for example.

[0102] The first effect is that since classification is possible for each region in an image, classification is possible even if multiple objects are captured in the image, unlike image classification (hereinafter, image classification according to a comparative example) which can only classify one image. The first effect is also that since excrement divided into multiple small pieces (when there are multiple small objects) which is difficult to detect by object detection can be classified by region, feces, urine, urine dripping, and foreign objects can be accurately classified. Hereinafter, this object detection is referred to as object detection according to a comparative example. Another effect of the first effect is that even if multiple objects overlap, classification can be performed from the regions where the objects do not overlap, and the multiple objects are not classified as one, so accurate classification is possible.

[0103] In particular, to notify caregivers of the start of excretion, completion of excretion, and abnormalities, and to accurately record excretion management, it is necessary to accurately detect excrement, foreign objects, and rear washing devices from images taken inside the toilet bowl. Advanced analysis is possible if analysis is performed using a cloud server, but since the image data is sent to the cloud server, this leads to a heavy mental burden in terms of privacy for the user. In addition, in that case, depending on the network environment, it may take a long time for the analysis results to be obtained because the image data is sent. Therefore, from the perspective of privacy protection and the network environment, it is desirable to perform excretion analysis on an edge device, which corresponds to the so-called edge of the communication network.

[0104] However, when performing excrement analysis in real time on an edge device, taking into consideration the use of a space-saving and power-saving CPU, the processing power is low, and it is therefore possible to realize this using image classification according to the comparative example. However, in this case, there are problems with accuracy, and the above image classification cannot accurately classify objects when multiple objects are captured in the image due to the classification of the entire image into one label.

[0105] It is also possible to adopt the object detection according to the comparative example, which can perform classification more accurately than the image classification according to the comparative example. In the object detection according to the comparative example, when an object is detected in an image, a rectangle (bounding box) that surrounds the detected object is arranged, and the objects in the bounding box are classified. Therefore, even if multiple objects are captured in an image, each object can be surrounded by a bounding box and classified. However, when there are multiple small objects, when multiple objects overlap, or when multiple objects are surrounded by a single bounding box, accurate classification may not be possible depending on the accuracy of the bounding box that surrounds the target object. Furthermore, in the case of object detection according to the comparative example, accurate object detection may not be performed from the imaging data due to the influence of differences in the structure inside the toilet and the reflected image depending on the manufacturer and type of the toilet bowl and toilet seat. In particular, if a buttock washer can be detected from the imaging data, it is an important determination factor that can notify the caregiver of the completion of excretion, but there is a possibility that accurate object detection may not be possible due to differences depending on the manufacturer and type of the toilet bowl and toilet seat. As described above, even when the object detection method according to the comparative example was adopted, there were problems such as poor classification accuracy and being affected by the structure inside the toilet bowl.

[0106] In contrast, in this embodiment, classification is performed on a pixel-by-pixel basis using semantic segmentation, which solves these problems and provides the first effect. In other words, in this embodiment, while improvements are being made to install sensors in toilets to reduce the burden of excretion management in care, the accuracy of excretion analysis related to notifications to caregivers and excretion records can be improved. And, since the first effect increases the reliability of the analysis results, the burden on caregivers can be reduced, and it can be said that thorough support for users is possible.

[0107] The second effect is that by classifying stool with a label including stool type (for example, Bristol scale 1 to 7), classification including accurate stool type determination can be performed in one processing, improving the accuracy of excrement analysis. And, due to the second effect, it can be said that the burden on caregivers can be reduced and thorough support for users can be provided.

[0108] In particular, by being able to classify with labels that include feces type, accurate classification is possible even when multiple feces types are identified in the excrement in the image data. Furthermore, if there is a difference in feces type between the start and end of excretion, it can be used for assessment to take appropriate measures, making it easier to manage excretion. In addition, feces type determination can be performed in one process together with image region segmentation, i.e., at the time of classification, making real-time analysis possible.

[0109] The third effect is that in this embodiment, classification by pixel unit results in classification by region, so it is not affected by differences in image data inside the toilet due to differences in manufacturers or types of toilets and toilet seats. Furthermore, this effect does not cause accuracy to deteriorate even with machine learning (a factor that inhibits the use of a trained model), and machine learning can be applied, so there is also an effect that high accuracy can be achieved.

[0110] The fourth effect is that by classifying by area, it is possible to accurately distinguish not only excrement but also the rear washer, and since dripping cleaning water while detecting the rear washer can be distinguished from urine, it is possible to accurately determine when excretion is complete and to accurately notify the caregiver.

[0111] <Embodiment 3> In the third embodiment, an excrement analyzer incorporating a function for checking the state before a colonoscopic examination into the excrement analyzer according to the first or second embodiment, and the processing thereof will be described with reference to Figs. 11 and 12. The excrement analyzer according to this embodiment can be called a state checking device before a colonoscopic examination, or a colonoscopic examination timing determination device. The present embodiment will be described focusing on the differences from the second embodiment, but the various examples described in the first and second embodiments can also be applied. Fig. 11 is a block diagram showing an example of the configuration of an excrement analyzer according to this embodiment (a state checking device before a colonoscopic examination).

[0112] As shown in Fig. 11, a pre-colonoscopy condition confirmation device (hereinafter simply referred to as a condition confirmation device) 5 according to this embodiment includes an input unit 5a, a classification unit 5b, and an output unit 5c, which correspond to the input unit 1a, the classification unit 1b, and the output unit 1c in Fig. 1, respectively. The condition confirmation device 5 can be incorporated into the system shown in Fig. 2, as in the second embodiment, and will be described with reference to Figs. 2 and 3. The condition confirmation device 5 can include a control unit (not shown) and a communication unit (not shown) that control the entire device, and this control unit can include parts of the above-mentioned input unit 5a, classification unit 5b, output unit 5c, and judgment unit 5d (and a calculation unit described later).

[0113] However, the contents output by the output unit 5c are different from the contents output by the output unit 1c, as described below. The output destination of the output unit 5c can basically be the terminal device 50 of the staff performing the colonoscopy, the terminal device of the subject, or the server 40. However, the server 40 is capable of transferring information to the terminal device 50 of the staff or the terminal device of the subject, or stores the information so that it can be viewed from the terminal device 50 or the terminal device of the subject.

[0114] Furthermore, the state confirmation device 5 according to this embodiment includes a determination unit 5d. The determination unit 5d determines whether or not the toilet user has completed pre-treatment before colonoscopic examination based on the classification result by the classification unit 5b. There are no restrictions on the criteria for this determination, but the criteria must basically be such that it can be determined that the pre-treatment has been completed. For example, if the stool is watery and the color of the stool is clear or yellowish clear, it is determined that the pre-treatment has been completed.

[0115] In order to make such a judgment possible, the classification unit 5b in this embodiment also executes classification of feces as excrement into a plurality of predetermined feces properties and a plurality of predetermined feces colors. The output unit 5c outputs the judgment result of the judgment unit 5d as the classification result of the classification unit 5b or as a part of the classification result of the classification unit 5b. The output destination can be set in advance, for example, the terminal device 50 of the colonoscopy examination staff or the terminal device of the examinee. The colonoscopy examination staff is the examiner, and corresponds to a doctor or a nurse. The terminal device of the examinee can be a portable terminal device such as a mobile phone (including those called smartphones), a tablet, or a mobile PC, but even if it is a stationary device such as a PC, there is no problem when the judgment result is viewed at home or the like.

[0116] The state confirmation device 5 according to this embodiment can output such a determination result, and therefore it is possible to reduce the burden on the subject (recipient) and the examiner.

[0117] The status checking device 5 may also include a calculation unit (not shown) that calculates the amount of feces based on the classification result by the classification unit 5b. The amount of feces can be calculated, for example, by obtaining the classification image Img-r in FIG. 6, as the total area of ​​the areas classified as feces, or as the total area of ​​the object (area classified as feces) in the classification image Img-r within a certain size. This calculation may be an estimation. In this case, the determination unit 5d determines whether the toilet user has finished pre-treatment based on the classification result by the classification unit 5b and the amount of feces calculated by the calculation unit. In particular, it is preferable to calculate the amount of feces based on the classification result using the last image taken before flushing.

[0118] Moreover, the state confirmation device 5 according to the present embodiment may not include the judgment unit 5d, but may include the judgment unit 5d on the server 40 side and may be configured to output the classification results to the server 40, that is, may be configured as a system in which the functions are distributed among a plurality of devices. The classification results may be output as classification images, but may not be classification results constructed as images. In other words, in this configuration, the server 40 has a function for automatically judging whether or not the pretreatment before the colonoscopic examination has been completed, for example, by using a database for judgment stored in advance. The server 40 can obtain the judgment results by providing the received classification results to the above function. The above function can be incorporated into the server 40 as a program. Even in this configuration, the present embodiment can reduce the burden on the subject and the examiner. In addition, the state confirmation device 5, whether configured as a single device or as a distributed system, can achieve the following effects as long as at least an optical camera for acquiring image data and a communication device are installed in the toilet of the home. That is, the state confirmation device 5 in such a configuration achieves the effect that at least one of the subject and the examiner can know the judgment results while the subject is at home.

[0119] Furthermore, in this embodiment, if an imaging device and a communication device, such as an optical camera and a communication device, are installed on the toilet bowl side, a configuration can be adopted in which other processes are executed on the server 40 side.

[0120] Next, a processing example of the status confirmation device 5 of Fig. 11 will be described with reference to Fig. 12. Fig. 12 is a flow diagram for explaining a processing example in the status confirmation device 5 of Fig. 11. The operation contents described here can be performed mainly by the CPU 11a in Fig. 3 controlling each part. Note that even in a configuration example in which the server 40 side is provided with some functions, the processing is basically the same as the processing example below, except that sending and receiving of information is added and the subject of some operations changes.

[0121] In the following, the process after the real-time analysis is performed and the classification result is obtained by the process illustrated in FIG. 10 will be mainly described. First, it is checked whether the real-time analysis is completed (step S21). If it is not completed in step S21 (NO), the process waits until it is completed. If it is completed (YES in step S21), the state confirmation device 5 judges whether the analysis result (classification result) of the stool nature is watery stool (for example, "stool nature 7" in the legend of FIG. 6, or "water" with a lower ratio of stool) (step S22). This judgment can be, for example, whether there is any stool area other than "water" and "stool nature 7" in the legend of FIG. 6 in the classification image Img-r. This is because if there is even a part of the area classified as stool nature 1 to 6, it means that the pre-treatment is not completed.

[0122] If the answer is YES in step S22, the status confirmation device 5 proceeds to judge the stool color analysis result, and judges whether the stool color analysis result is either "transparent" or "yellowish transparent", or other than that (step S23). If the answer is YES in step S23, the status confirmation device 5 generates a judgment result indicating that the pre-processing judgment is test OK, as it meets the pre-processing judgment condition (step S24). Next, the status confirmation device 5 transmits a notice (pre-processing judgment notice) indicating the pre-processing judgment result (here, test OK) to at least one of the terminal device of the subject who is the toilet user and the terminal device 50 of the staff (step S25), and ends the process. Of course, the order of the judgments in steps S22 and S23 does not matter.

[0123] This allows the subject to know that he / she is ready for testing, and to inform the staff accordingly. Alternatively, the staff can determine that the subject is ready for testing, and when the testing system for the subject is ready, they can speak to the subject. In particular, when it comes to notifying the tester, instead of notifying them as text information, it is possible to notify them by automatic voice via an intercom, etc., thereby saving the tester the trouble of having to look at the text information.

[0124] On the other hand, if the result is NO in step S22 or NO in step S23, the status confirmation device 5 generates a judgment result indicating that the pre-processing judgment is test NG as the pre-processing judgment condition is not met (step S28). Next, the status confirmation device 5 transmits a pre-processing judgment notice indicating that the test is NG to at least one of the subject's terminal device and the staff's terminal device 50 (step S25), and ends the process. The subject can wait for a period of time as necessary to relieve himself / herself, or the staff can urge the subject to relieve himself / herself, until a pre-processing judgment notice indicating that the test is OK is received.

[0125] Although not shown, the status confirmation device 5 can also output the analysis result to the server 40 after the processing of step S24 and after the processing of step S28. This analysis result can include the result of the preprocessing judgment, but can also include the result of the preprocessing judgment only when the inspection is OK, for example. Note that, regarding the imaging data, it is basically not transmitted to the server 40 from the viewpoint of privacy and reduction of the amount of transmitted data, but it may be transmitted to the server 40 on the premise that only persons having the authority to manage the server 40 can access it.

[0126] As described above, in addition to the effects described in the second embodiment, the present embodiment provides the following effects, for example.

[0127] The first effect is that by automatically determining the contents of excrement identified by a combination of optical cameras and machine learning, it is possible to reduce the variability in judgment criteria that have previously been dependent on people (especially the subjects).

[0128] The second effect is that real-time analysis notifies users of events occurring in the toilet (sitting, excretion, foreign object detection, etc.), allowing them to immediately grasp the status of the subjects' pre-examination tasks, freeing examiners from having to constantly watch the subjects excrete, thereby reducing the time burden on them.

[0129] The third effect is that when analyzing images captured by the optical camera, all analytical processing is performed by the toilet sensor, so the image data is not seen by third parties, reducing the mental burden on the subject regarding their privacy.

[0130] The fourth effect is that, in conjunction with the second and third effects, the examiner does not invade the privacy of the subject, thereby reducing the mental burden of being in the opposite position.

[0131] The fifth effect is that by making a judgment using a database that records the analysis results of excrement, it is expected that the standard accuracy of pre-test judgment, which has been performed up until now, will be improved.

[0132] The sixth effect is that pre-test results can be checked remotely, so that even if the person being tested is carrying an infectious disease, the risk of infection to the examiner during pre-test work can be avoided.

[0133] The seventh effect is that the product can be attached to toilets of a general shape (Western-style toilets), can be produced and distributed as a single type of product, can be inexpensive, and is portable.

[0134] <Embodiment 4> In the third embodiment, it is assumed that a device including the excrement analyzer according to the first or second embodiment is used as a state confirmation device before a colonoscopic examination, but such an excrement analyzer may not be used. In the fourth embodiment, an example of state confirmation before a colonoscopic examination is described, regardless of the classification method of the imaged substance. The components of the state confirmation device according to the fourth embodiment are the same as those of the state confirmation device 5 described in FIG. 11, and only the details of the processing of some of the components are different, so this embodiment will also be described with reference to FIG. 11, FIG. 2, FIG. 3, etc. Note that, in this embodiment, various examples applied in the third embodiment, which incorporates the first and second embodiments, can basically be applied, except for the contradictory processing examples.

[0135] As shown in FIG. 11, like the status monitoring device 5 according to the third embodiment, the status monitoring device 5 according to this embodiment also includes an input unit 5a, a classification unit 5b, an output unit 5c, and a determination unit 5d.

[0136] To briefly explain each part, the input unit 5a inputs imaging data captured by an imaging device installed so that the imaging range includes the excrement excretion range in the toilet bowl. The classification unit 5b classifies the imaged substance for the imaging data input by the input unit 5a. The determination unit 5d determines whether the toilet user has completed pre-treatment before the colonoscopy examination based on the classification result by the classification unit 5b. The output unit 5c outputs the determination result by the determination unit 5d as notification information to at least one of the colonoscopy staff who monitor the toilet user as a colonoscopy subject and the subject.

[0137] Furthermore, the status checking device 5 according to this embodiment can also employ a configuration including a calculation unit described in embodiment 3. This calculation unit calculates the stool volume, which is the amount of stool, based on the classification results of the classification unit 5b (particularly the classification results of the second classification unit described below). For example, this calculation unit can calculate the stool volume based on the classification results of the second classification unit described below. If the stool is not classified as stool, the stool volume can be calculated as zero. Then, the determination unit 5d can determine whether the toilet user has completed pre-treatment or not based on the classification results of the classification unit 5b and the stool volume calculated by the calculation unit 5e.

[0138] In addition, the status checking device 5 according to this embodiment may also include a control unit (not shown) and a communication unit (not shown) that control the entire device, and this control unit may include parts of the input unit 5a, classification unit 5b, output unit 5c, and judgment unit 5d (and calculation unit) described above.

[0139] However, in this embodiment, the classification unit 5b classifies the excrement among the imaged substances into either feces, urine, or urine dripping, or into either feces, urine, feces + urine, or urine dripping, and also classifies the excrement into a plurality of predetermined stool types and a plurality of predetermined stool colors.

[0140] In other words, the classification unit 5b in this embodiment is only required to be able to classify the imaged material in this manner, and may not use the semantic segmentation described in embodiments 1 to 3 at all, or may use it only partially. In the following, an example will be described in which the classification unit 5b executes a primary classification (primary analysis) and a secondary classification (secondary analysis) as classification processing, which will be described later, and uses semantic segmentation only in the primary analysis. However, semantic segmentation may be used only in the secondary analysis, for example, or may not be used in either the primary analysis or the secondary analysis.

[0141] Here, the classification unit 5b may include a first classification unit that performs a primary analysis and a second classification unit that performs a secondary analysis, although these are not shown. Since the classification unit 5b also performs a secondary analysis after the primary analysis, it is assumed that the classification unit 5b includes a storage unit that temporarily stores the imaging data to be analyzed until the secondary analysis. This storage unit may be a storage device such as a memory.

[0142] The first classification unit classifies the imaged substance into excrement, foreign matter that is not permitted to be disposed of in the toilet 20, and other substances, and classifies the excrement into feces, urine, and urine drips, or into feces, urine, feces + urine, and urine drips. In this embodiment, the other substances can include at least one of a bidet, toilet paper, and substances that are left over after excrement is flushed. The first classification unit can be executed in real time as the imaging data is acquired.

[0143] In the present embodiment, the determination unit 5d determines that the toilet user has not completed pretreatment when the first classification unit has classified the toilet as something other than feces. Therefore, when the determination unit 5d has determined that the pretreatment has not been completed, the output unit 5c can output, as notification information, information indicating that the colonoscopic examination cannot be performed yet.

[0144] The notification information output by the output unit 5c may include the classification result by the first classification unit. In this case, the notification information may include information indicating the classification result, and may be information that is predetermined according to the classification result. For example, the notification information may be information that notifies the user that a foreign object is present in the captured image data when the foreign object is captured in the captured image data. In particular, the notification information may include a classification image that depicts the classification result by the first classification unit in a different color for each classification. The classification image may be, for example, the classification image Img-r in FIG. 6. The excretion information that is the classification result by the first classification unit may be output to a server 40 that collects and manages excretion information.

[0145] Furthermore, when the first classification unit classifies the imaged substance into a plurality of stool types and a plurality of stool colors for the imaging data, the second classification unit classifies the imaged substance into a plurality of stool types and a plurality of stool colors for the imaging data. The second classification unit can perform classification based on the imaging data stored in the storage unit after classification by the first classification unit, and can be performed in non-real time because it requires higher accuracy than the processing by the first classification unit.

[0146] Furthermore, in the present embodiment, when the classification result of the first classification unit is feces, the judgment unit 5d judges whether or not the toilet user has completed pre-treatment based on the classification result of the second classification unit. Also, when the classification result of the first classification unit is something other than feces, the judgment unit 5d can stop classification by the second classification unit and issue an excretion completion notification indicating that pre-treatment has not been completed.

[0147] The notification information output by the output unit 5c may include the classification result by the second classification unit. In this case, the notification information may include information indicating the classification result, and may be information that is predetermined according to the classification result. For example, the notification information may be information notifying that there has been a change in bowel movements. In particular, the notification information may include a classification image in which the classification result by the second classification unit is depicted in a different color for each classification. This classification image may be, for example, one exemplified by the classification image Img-r in FIG. 6. The excretion information that is the classification result by the second classification unit may also be output to a server 40 that collects and manages excretion information as an output destination.

[0148] As described above, the status confirmation device 5 separates the analysis of the image data acquired from the camera into a primary analysis aimed mainly at notification requiring immediacy and a secondary analysis aimed at notification (and recording) not requiring immediacy. This allows the status confirmation device 5 to have a built-in control unit such as a CPU that is space-saving and power-saving. This means that the status confirmation device 5 efficiently uses limited computational resources by separating the analysis processing into functions requiring immediacy and other functions. Furthermore, the status confirmation device 5 does not need to transmit the image data acquired from the camera and other image data to an external device such as a cloud, and can analyze excrement by itself installed in the toilet. In other words, all images and videos used in the analysis by the status confirmation device 5 are processed within the status confirmation device 5, and the images and videos are not transmitted to the outside. Therefore, it can be said that the status confirmation device 5 is configured to reduce the mental burden on the user regarding privacy.

[0149] As described above, the status confirmation device 5 can determine the completion of the pre-treatment for colonoscopy without having to ask the toilet user while respecting the privacy of the toilet user. In addition, the status confirmation device 5 can accurately collect information indicating the contents of the excrement excreted into the toilet bowl, and can also respond to situations where an immediate notification to the monitor is required. In other words, the status confirmation device 5 can realize both consideration of the privacy of the toilet user and notification and recording, while improvements are being made to install sensors in toilets to reduce the burden of excretion management in monitoring such as nursing care. The notification and recording here are notification of an immediate event at the monitoring site such as a nursing care site and recording of accurate information. Therefore, the status confirmation device 5 can reduce the physical and mental burden on the monitor and toilet user.

[0150] As described above, the status confirmation device 5 can obtain the start of excretion, detection of foreign body, detection of excrement, and completion of excretion as primary analysis results, and can obtain stool texture, stool color, and stool volume as secondary analysis results. All analysis results can be recorded in the server 40 on the cloud in a state that can be viewed from the terminal device 50, and can also be configured to transmit them to the terminal device 50. Also, the server 40 can be configured to accumulate the received analysis results, perform further analysis from the accumulated data, and notify the terminal device 50 of the analysis results or make them viewable from the terminal device 50.

[0151] In addition, the state confirmation device 5 or the present system including the same can be used in a private home on the assumption that there is only one user, but it is preferable to provide a function for identifying users on the assumption that there are multiple users. This function has been described using face image data acquired by the second camera 15b and identification data obtained by the Bluetooth module 14b. This makes it possible to notify the examiner or subject of entry notifications, exit notifications, seating notifications, leaving notifications, excretion start notifications, excretion completion notifications, pre-treatment judgment notifications, etc. along with the user name, and to record detailed excretion information for each user in the chart.

[0152] Next, real-time analysis as a primary analysis (preliminary analysis) and non-real-time analysis as a secondary analysis (main analysis) will be outlined with reference to Fig. 13, Fig. 3, and Fig. 2. Fig. 13 is a conceptual diagram for explaining an example of processing in the status checking device 5.

[0153] In this embodiment, the second external box 11 includes the following devices. This device performs real-time analysis as a primary analysis based on the imaging data (image data) captured by the first camera 16b, and non-real-time analysis as a secondary analysis based on the image data and the real-time analysis result. The second external box 11 also includes a communication device 14 that notifies the examiner or the subject when an event occurs and transmits the analysis result to the server 40 under the control of the device. The CPU 11a performs real-time analysis and non-real-time analysis while transmitting and receiving data to other parts via each of the elements 11b, 11c, and 11d as necessary. In this example, the CPU 11a may also include a memory as an example of a storage unit.

[0154] As shown in FIG. 13, an example will be given in which a user P uses a toilet bowl 30 with an analysis function installed in a toilet, and an examiner C of the user P monitors the state of the toilet bowl. However, the device 30 with an analysis function in this embodiment is also equipped with the judgment function of a judgment unit 5d. When the user P uses the toilet bowl 30 with an analysis function, the CPU 11a detects that the user has sat on the toilet seat based on the detection result from the distance sensor 16a that functions as a seating sensor. When the CPU 11a detects that the user has sat on the toilet seat, it instructs the first camera 16b to start capturing images, and performs a primary analysis 31a based on the captured image data. The CPU 11a can perform foreign object judgment and the like as the primary analysis 31a.

[0155] When the result of the primary analysis 31a requires immediate notification to the examiner, such as detection of a foreign object, the CPU 11a transmits notification information (primary analysis notification 32a) to the terminal device 50 of the examiner C who is located away from the toilet via the WiFi module 14a. In this way, the CPU 11a can transmit foreign object information (foreign object information indicating the foreign object determination result) indicating whether or not a foreign object is contained to the terminal device 50. This foreign object information is output as at least a part of the notification information. This relieves the examiner C from the situation of accompanying (staying with) the user P who is the subject during excretion, and the primary analysis notification 32a enables the examiner C to respond in an emergency 51 or to log in the chart that the subject has started pre-examination work. Here, the primary analysis notification 32a transmitted does not include imaging data.

[0156] After the primary analysis 31a is completed, the CPU 11a performs a secondary analysis 33a, which is a more detailed excrement analysis, based on the imaging data and the primary analysis results. For this reason, the storage unit in the CPU 11a temporarily stores the primary analysis results as part of the second analysis target data. The CPU 11a transmits 34a the secondary analysis results to the server 40 via the WiFi module 14a. In addition, the examiner C of the user P performs recording 54 of the user P's chart on the terminal device 50 while appropriately referring 52 to the detailed excretion information of the user P stored in the server 40 based on the received notification information.

[0157] In this way, the analysis results of the primary analysis 31a and the secondary analysis 33a are transmitted to the server 40 by executing the analysis result transmission 34a by the communication function. The analysis result transmission 34a is transmitted without including the imaging data, but may be stored on the cloud as learning data for future pretreatment judgment and accessible only to those authorized to manage the system. In parallel with the analysis result transmission 34a, the pretreatment judgment result is transmitted to the terminal device 50 as a secondary analysis notification 32b and is recorded (logged) in the medical chart. The information recorded in the server 40 can also be used by the examiner to create a medical chart 54 or to check the log after the fact.

[0158] The contents of primary analysis 31a and secondary analysis 33a will be described with reference to Figures 14 to 16. Figures 14 to 16 are diagrams for explaining an example of processing in status monitoring device 5.

[0159] First, an example of the input, method, and output of the primary analysis and secondary analysis will be described with reference to FIG. 14. The primary analysis is an analysis that requires real-time performance, such as notification to the examiner C. The primary analysis takes as input image data (imaging data) captured by the first camera 16b, classifies the data into one of the following six types using, for example, semantic segmentation, and outputs the classification result. The six types are foreign objects (diapers, incontinence pads, etc.), feces, feces + urine, urine, urine drips, and bidet.

[0160] Semantic segmentation can also be used to compare an image (background image) before excretion or the like occurs with an image (image during excretion or after excretion is completed) afterwards. For example, the background image and the image afterwards can be input as input to the learning model, and it can be output as to which of the six types it corresponds to. Alternatively, a difference image of the image afterwards from the background image can be obtained as preprocessing, and this difference image can be input to the learning model, and it can be output as to which of the six types it corresponds to. If it is classified as a bottom washer, it can be determined that excretion has been completed. These classification types are examples of events that trigger real-time notifications.

[0161] In this way, in the primary analysis, the notification information can be obtained from the imaging data using a trained model that inputs the imaging data and outputs the notification information. The notification information can be, for example, information that is predetermined in response to the classification result. As a result, the state confirmation device 5 can notify the inspector, etc. of information such as the start and end of excretion, and the presence of foreign objects in the excrement, as the notification information, and the inspector, etc. can obtain this information in real time. Note that the algorithm (machine learning algorithm) of the trained model and hyperparameters such as the number of layers do not matter, and they may be generated by machine learning. In addition, the machine learning here does not matter whether or not there is teacher data. However, in this example, a model that performs semantic segmentation is used as the trained model, and it is assumed that there is teacher data. In addition, a plurality of trained models may be used in the primary analysis, and for example, a trained model that is different from at least one of the above six types and the other types may be used.

[0162] The secondary analysis can be performed, for example, by inputting the image data from the first camera 16b and the primary analysis result, using two techniques, DL and Image Processing (IP). For example, analysis using DL can output stool characteristics, and analysis using IP can output stool color, stool volume, and urine color. Semantic segmentation can also be used for the analysis of stool characteristics. Note that the primary analysis is treated as a pre-processing for the secondary analysis. In the secondary analysis, DL and IP are used to compare the analysis results (which may be images) after this pre-processing with the learned data, and output stool characteristics, stool color, etc.

[0163] Here too, DL technology can be used to compare an image (background image) before excretion and the like is performed with an image (image during excretion or after excretion is completed) after excretion. For example, the classification result of the first analysis, the background image, and the image after excretion can be input as input to the learning model, and fecal matter can be output. Alternatively, a difference image of the image after excretion from the background image can be obtained as preprocessing, and the classification result of the first analysis and the difference image can be input to the learning model to output fecal matter. Also, only if the classification result of the first analysis includes feces, analysis by DL in the second analysis may be performed, in which case the above classification result is not necessary for input to the trained model. Also, the processing method in IP does not matter, as long as the desired detailed excretion information can be obtained. For example, a matching process can be performed with a comparison target image stored in advance by extracting the features of the image, and the fecal color, etc. shown by the comparison target image with a high matching rate can be output. In addition, in the second analysis, all outputs may be obtained by either IP or DL.

[0164] In this way, in the secondary analysis, at least a portion of the detailed excretion information can be obtained from the second analysis target data by using a trained model that inputs the second analysis target data (which may include the results of the primary analysis) and outputs excretion information. The trained model's algorithm (machine learning algorithm) and hyperparameters such as the number of layers are not important, as long as they are generated by machine learning. The machine learning here does not require the presence or absence of teacher data. Furthermore, multiple trained models may be used in the primary analysis. Furthermore, as described above, in the secondary analysis, at least a portion of the detailed excretion information can be obtained by image processing of the second analysis target data. As described above, the method of this image processing is not important, as long as the desired detailed excretion information can be obtained.

[0165] A detailed example of the primary analysis will be described with reference to FIG. 15. In the primary analysis, foreign objects, types of excretion, and bidets can be judged. First, foreign object detection is performed based on an image (image data) captured by the first camera 16b, which is an optical camera. Foreign object detection can be performed at any time, and when a foreign object is detected, a notice is given to the examiner. After that, an image captured at the timing of sitting down is used as a background image, and then, based on a preprocessed image (and / or additional information) obtained by preprocessing images captured at regular intervals, a DL is used at regular intervals to judge whether the excreta is feces, feces + urine, urine, or urine dripping. This judgment is performed until the timing of leaving the seat. Here, in order to prevent the human body or internal equipment, which are not the subject of analysis, from being reflected in the background image, it is preferable to perform processing (masking processing) to blacken the detected human body parts as not being the subject of analysis. It is also preferable to perform the same mask processing as the background image on the images captured at regular intervals after the background image is obtained. The additional information can include information such as the date and time of photography, and can also be, for example, information showing a statistical value taking into account the above-mentioned fixed period, information showing an area such as width, etc. In addition, using the same method and timing, detection of a posterior washer is also performed, and the determination of feces, feces + urine, urine, and urine dripping is completed when a posterior washer is detected.

[0166] However, when semantic segmentation is used in the primary analysis, for example, the masked imaging data can be input and these classifications can be performed as a single process. On the other hand, as illustrated in Fig. 15, the primary analysis can classify different objects depending on the timing. In that case, a corresponding trained model can be switched depending on the timing, and classification can be performed using the trained model according to the timing (i.e., according to the classification object).

[0167] In this way, the CPU 11a may transmit at least one of information indicating the usage status of the posterior washer installed on the toilet and information indicating that a person has sat on the toilet as at least a part of the notification information to the terminal device 50 as a result of the primary analysis. As described above, information indicating the usage status of the posterior washer can be obtained as a result of the primary analysis of the imaging data. This is because the nozzle that discharges the cleaning liquid or the cleaning liquid itself is included as a subject of the imaging data during use. Also, information indicating that a person has sat on the toilet can be obtained by a seating sensor exemplified by the distance sensor 16a. In this way, the primary analysis can also be performed using information other than the imaging data. The usage status of the posterior washer can be known by the CPU 11a not only by analyzing the imaging data, but also by obtaining information from the posterior washer if it is connected to the imaging data, for example.

[0168] A detailed example of the secondary analysis will be described with reference to FIG. 16. In the secondary analysis, the background image and the input image are selected, and the analysis can be performed on all images that have been pre-processed in the primary analysis. The background image and the input image are selected in a combination suitable for each judgment target, and a detailed analysis is performed. To explain an example of the combination, first, for feces quality, an image after sitting is selected as the background image, and the last feces image is selected as the input image. In the case of feces only or urine only, similarly, target images are selected for feces color, feces volume, and urine color. However, for urine color, a urine image is used instead of feces. In the case of feces + urine, the last urine image before urination is used as the background image, and the last urine stool image is used as the input image. Note that analysis of urine volume can also be performed, and in that case, all images judged as urine dripping are used as input images without using the background image.

[0169] In particular, in this embodiment, the stool texture, color, and amount are used as criteria for pretreatment judgment, and in order to obtain information that there is no residual stool in the intestines, the stool is judged to be watery and the color is "yellowish transparent" or "transparent." Then, in this embodiment, such judgments are used to obtain information on whether or not to carry out the test as a pretreatment judgment result.

[0170] In this way, in the secondary analysis, the stool texture and color can be identified and the stool volume can be calculated from the acquired imaging data, and detailed excretion information can be output. In addition, in the secondary analysis, threshold processing can be performed on the stool volume and urine volume, and information indicating whether or not a predetermined threshold has been exceeded can be used as detailed excretion information, or can be added to the detailed excretion information. It is desirable that the detailed excretion information output as a result of these threshold processing processes is also sent (notified) to the terminal device 50 directly or via the server 40. Such notifications (which may include warnings) allow the examiner to understand events that require action.

[0171] Next, an example of the procedure of the primary analysis process will be described with reference to Fig. 17. Fig. 17 is a flow diagram for explaining an example of the process in the status confirmation device 5, and is a flow diagram showing an example of the operation content of the primary analysis triggered by the user entering the toilet and sitting on the toilet seat. The operation content described here can be performed mainly by the CPU 11a controlling each part.

[0172] First, the presence or absence of a reaction from the distance sensor 16a functioning as a seating sensor is checked (step S51). If there is no reaction in step S51 (if NO), the system will wait until the seating sensor reacts. If the subject as a user sits down, the distance sensor 16a will react, and step S51 will result in YES. If step S51 results in YES, the terminal device 50 is notified of the seating (step S52) and the primary analysis is started (step S53). Note that if the human sensor 15a detects entry before the person sits down, the terminal device 50 can also be notified of the entry, and the same applies to leaving the room.

[0173] In the primary analysis, the first camera 16b takes an image of the inside of the toilet bowl, and first, it is determined whether the image can be normally identified (step S54). If an abnormality is detected (NO in step S54), an abnormality notification is sent to at least one of the terminal device 50 of the examiner and the terminal device of the subject (step S55). Here, the case where the notification is sent to the terminal device 50 corresponds to the case where the examiner checks the pre-treatment judgment on behalf of the subject, and the terminal device of the subject corresponds to the case where the subject makes the pre-treatment judgment by himself / herself, and this relationship is the same in the subsequent processes. In this way, even if the image of the inside of the toilet bowl cannot be normally captured, it is preferable that notification information indicating that is sent to at least one of the terminal device 50 of the examiner and the terminal device of the subject. On the other hand, if the image can be normally identified (YES in step S54), the process proceeds to a detailed analysis, and first, pre-processing of the captured image is performed (step S56).

[0174] After pre-processing of the captured image in step S56, the detected object is classified as a foreign object, excrement, or bidet (step S57). If a foreign object is detected, a foreign object detection notification is sent to the examiner's terminal device 50 (step S8). If excrement is detected, an excretion notification (notification information indicating that excretion has occurred) is sent to at least one of the examiner's terminal device 50 and the subject's terminal device (step S59), and an excrement analysis is performed (step S60). This excrement analysis classifies the object as stool, stool + urine, urine, or urine drips. After processing in step S60, the process returns to step S54.

[0175] If the object detected in step S57 is a posterior washer, it is determined that excretion has been completed, and an excretion completion notification (notification information indicating that excretion has been completed) is sent to at least one of the examiner's terminal device 50 and the subject's terminal device (step S61). Following the excretion completion notification in step S61, the primary analysis ends (step S62). Alternatively, the excretion completion notification may be sent only when there is no reaction from the seating sensor. This is because the posterior washer may be used more than once. The primary analysis also ends after steps S55 and S58.

[0176] An example of the procedure of the secondary analysis process will be described with reference to Figs. 18 to 20. Figs. 18 and 19 are flow diagrams for explaining an example of the process in the status checking device 5, and are flow diagrams showing an example of the operation contents of the secondary analysis. The operation contents described here can be performed mainly by the CPU 11a controlling each part. Also, Fig. 20 is an example of stool color analysis included in the secondary analysis in the process example of Fig. 18. An example of stool nature analysis will be described with reference to Fig. 7 again.

[0177] The primary analysis shown in Figure 17 is performed using a space-saving, power-saving CPU, and performs the minimum analysis necessary to realize immediate notification to the subject or examiner. In contrast, the secondary analysis performs a more detailed analysis of the excrement.

[0178] First, it is determined whether the primary analysis is complete (step S71), and if it is complete (YES), the secondary analysis is started (step S72). Alternatively, if a user identification function is provided, it may be determined whether a predetermined number of excretion counts has been exceeded (or a predetermined period of time has elapsed) for each user, and if so, the secondary analysis may be started.

[0179] The inputs for the secondary analysis and the respective analysis methods can be as described with reference to FIG. 16, but first, the results of the primary analysis are judged (step S73), and different processes are carried out depending on the results.

[0180] If the result of the primary analysis judgment in step S73 is urine, urine dripping, or feces + urine, the pretreatment judgment is determined to be test NG (test should not be performed) because it cannot be judged properly because it contains something other than feces, which is the target of the pretreatment judgment. Specifically, the status confirmation device 5 generates a judgment result that the pretreatment judgment is test NG (step S83) since the condition of the pretreatment judgment is not met in this case. Next, the status confirmation device 5 transmits a pretreatment judgment notice indicating test NG to at least one of the subject's terminal device and the staff's terminal device 50 (step S80), and proceeds to step S81. The subject can wait a while as necessary to excrete, or the staff can encourage the subject to excrete, until a pretreatment judgment notice indicating the test is OK is obtained.

[0181] If the result of the primary analysis in step S73 is stool, then a stool nature analysis (step S74), a stool color analysis (step S75), and a stool volume analysis (step S76) are performed. Of course, these steps can be performed in any order. If the result of the primary analysis in step S73 is urine or urine dripping, then a urine color analysis can be performed, and a urine volume analysis can also be performed. Each analysis in steps S74 to S76 can be performed using, for example, an individual learning model, but multiple analyses or all analyses can also be performed using a single learning model.

[0182] Here, in the fecal analysis in step S74, the most reliable image is used to perform an analysis by comparing it with the image already learned by DL. The most reliable image can be the image itself indicated by the imaging data, or an image obtained by preprocessing the imaging data using a preprocessing method suitable for fecal analysis. In addition, in this fecal analysis, the analysis can be performed in a form conforming to the Bristol scale shown in FIG. 7, for example. As a result of the analysis, the fecal matter can be classified into any one of types 1 to 7 as shown in FIG. 7.

[0183] In addition, in the stool color analysis in step S75, for example, pre-processing can be performed as shown in the processing procedure of transitioning between images 61, 62, and 63 in FIG. 20. In the pre-processing exemplified here, a light-colored portion occupying a wide area is removed from the original image 61 to obtain image 62, and then a narrow area of ​​the same color is removed to obtain image 63. In the stool color analysis in step S75, an image such as image 63 from which necessary information has been extracted (and / or added) by pre-processing is used, and a distance between the extracted stool color and the stool reference color is calculated, and the color occupying the largest area in the extracted stool image can be determined as the stool color. For example, in image 63, there is a stool form consisting of two colors, and the color of the larger area of ​​the two colors can be determined as the stool color. The information added here can be, for example, information indicating the area.

[0184] When performing urine color analysis, the same method as for stool color analysis in step S75 can be adopted, but the target image is a urine image instead of a stool image, and the distance from the reference color is calculated, and the color that occupies the largest area is determined to be the urine color.

[0185] In the stool volume analysis in step S76, the stool volume can be calculated (estimated) as an area ratio within a certain size using a stool image (e.g., image 63, or the primary analysis result) extracted in preprocessing for the image at the time of completion of excretion. However, since the stool volume differs depending on the stool type even for the same area, it is advisable to calculate it using the area ratio corresponding to the stool type and a standard value for the stool volume.

[0186] When each analysis is completed, the status confirmation device 5 judges whether the analysis result (classification result) of the stool type is watery stool (for example, "stool type 7" in the legend of FIG. 6, or "water" which is a lower proportion of stool) (step S77). This judgment can be made, for example, by judging whether there are any areas of stool other than "water" and "stool type 7" in the legend of FIG. 6 in the classification image Img-r of FIG. 6. This is because if there is any area classified as stool type 1 to 6, even if only a part of it, it means that the pretreatment is not completed, that is, the conditions for pretreatment judgment are not met.

[0187] If the answer is YES in step S77, the status confirmation device 5 proceeds to judge the stool color analysis result, and judges whether the stool color analysis result is either "transparent" or "yellowish transparent", or other than that (step S78). If the answer is YES in step S78, the status confirmation device 5 generates a judgment result indicating that the pre-processing judgment is test OK, since the condition for the pre-processing judgment is met (step S79). Next, the status confirmation device 5 transmits a notice (pre-processing judgment notice) indicating the pre-processing judgment result (here, test OK) to at least one of the terminal device of the subject who is the toilet user and the terminal device 50 of the staff (step S80). Of course, the order of the judgments in steps S77 and S78 does not matter.

[0188] After the process of step S80, the status confirmation device 5 transmits the analysis result to the server 40 (step S81), and ends the process. This analysis result may include the result of the pre-processing judgment, but may include the result of the pre-processing judgment only when the inspection is OK, for example. Note that, regarding the imaging data, it is basically not transmitted to the server 40 from the viewpoint of privacy and reduction of the amount of transmitted data, but it may be transmitted to the server 40 on the premise that only persons having the authority to manage the server 40 can access it.

[0189] This allows the subject to know that he / she is ready for testing, and to inform the staff accordingly. Alternatively, the staff can determine that the subject is ready for testing, and when the testing system for the subject is ready, they can speak to the subject. In particular, when it comes to notifying the tester, instead of notifying them as text information, it is possible to notify them by automatic voice via an intercom, etc., thereby saving the tester the trouble of having to look at the text information.

[0190] On the other hand, if the result is NO in step S77 or NO in step S78, the status confirmation device 5 generates a judgment result indicating that the pre-processing judgment is test NG as the pre-processing judgment condition is not met (step S83). Next, the status confirmation device 5 transmits a pre-processing judgment notice indicating that the test is NG to at least one of the subject's terminal device and the staff's terminal device 50 (step S80). After the processing of step S80, the processing of step S81 is performed, and the processing ends. The subject can wait for a period of time as necessary to relieve himself / herself, or the staff can urge the subject to relieve himself / herself, until a pre-processing judgment notice indicating that the test is OK is obtained.

[0191] Also, in this embodiment, the toilet sensor can be configured as a system that only includes an optical camera, a communication device, and a first classification unit, and the server 40 performs other processes. In this configuration example, the server 40 can include a receiving unit, a second classification unit, a determination unit, and an output unit as described below. These components will be briefly described below, but the second classification unit, the determination unit, and the output unit are basically the same as the parts with the same names that were explained with reference to Figures 13 to 20.

[0192] The receiving unit receives the classification result of the first classification process executed by the first classification unit, and receives the imaging data when the classification result of the first classification process indicates classification as stool. The second classification unit in this configuration example classifies the imaged substance into a plurality of predetermined stool types and a plurality of predetermined stool colors for the imaging data received by the receiving unit. The determination unit in this configuration example determines whether the toilet user has completed pre-treatment before the colonoscopy examination based on the classification result by the second classification unit. The output unit in this configuration example outputs the determination result by the determination unit as notification information to at least one of the colonoscopy staff who monitor the toilet user as a subject for the colonoscopy examination and the subject.

[0193] In this configuration example as well, the determination unit can determine that the toilet user has not completed pre-treatment if the classification result received by the receiving unit is other than feces. Furthermore, in this configuration example, when the receiving unit receives image data, the determination unit can determine whether the toilet user has completed pre-treatment based on the classification result by the second classification unit.

[0194] Also, in this embodiment, the toilet sensor may be configured as a system that only includes an optical camera and a communication device, and the server 40 performs other processes. The server 40 in this configuration example only needs to include a receiving unit that can receive image data, and corresponds to an example in which the status monitoring device 5 is implemented in the server 40, and the only difference is the transmission and reception of information, so a detailed description thereof will be omitted.

[0195] As described above, this embodiment provides the first effect and the third to seventh effects described in the third embodiment. Furthermore, this embodiment can provide the following effect in relation to the second effect described in the third embodiment. That is, in this embodiment, the first primary analysis notifies the examiner of events occurring in the toilet (sitting, excretion, foreign body detection, pre-treatment NG, etc.), so that the examiner's pre-examination work status can be grasped in real time. Therefore, this embodiment also provides the effect that the examiner is freed from constantly attending to the excretion of the examinee, and the time burden on the examiner is reduced.

[0196] <Other embodiments> [a] In each embodiment, the functions of each device, such as the excrement analyzer, the server device, the device for checking the state before colonoscopy, and the terminal device that configures the system together with each device, have been described. None of these devices are limited to the configuration examples shown in the figures, and it is sufficient if these functions can be realized as these devices.

[0197] [b] Each of the devices described in the first to fourth embodiments may have the following hardware configuration. Fig. 21 is a diagram showing an example of the hardware configuration of the device. The same applies to the other embodiment [a] described above.

[0198] The device 100 shown in FIG. 21 may include a processor 101, a memory 102, and a communication interface (I / F) 103. The processor 101 may be, for example, a microprocessor, a microprocessor unit (MPU), or a CPU. The processor 101 may include a plurality of processors. The memory 102 is, for example, a combination of a volatile memory and a non-volatile memory. The functions of each device described in the first to fourth embodiments are realized by the processor 101 reading and executing a program stored in the memory 102. At this time, information can be exchanged with other devices via the communication interface 103 or an input / output interface (not shown). In particular, when the device 100 is an excrement analyzer or a status checker, information (including image data) of an imaging device built into or externally attached to the device 100 can also be exchanged via the communication interface 103 or an input / output interface (not shown).

[0199] In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, the computer-readable medium or tangible storage medium includes random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disk or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium includes electrical, optical, acoustic, or other forms of propagated signals.

[0200] The present disclosure is not limited to the above-described embodiment, and may be modified as appropriate without departing from the spirit and scope of the present disclosure. In addition, the present disclosure may be implemented by appropriately combining the respective embodiments.

[0201] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0202] (Appendix 1) an input unit for inputting image data captured by an imaging device installed so as to include an imaging range of the excrement area in the toilet bowl; a classification unit that performs classification of imaged materials on a pixel-by-pixel basis using semantic segmentation on the imaging data input by the input unit; an output unit that outputs a classification result from the classification unit; An excrement analysis device comprising: (Appendix 2) the classification unit classifies the imaged substance for each pixel into one of the excrement, a foreign object that is not permitted to be disposed of in the toilet bowl, and another substance; 2. An excrement analysis device as described in appendix 1. (Appendix 3) The classification unit classifies the excrement into any one of feces, urine, and urine dripping, or into any one of feces, urine, feces and urine, and urine dripping. 3. An excrement analyzer as described in appendix 2. (Appendix 4) The classification unit also performs at least one of classifying the stool into a plurality of predetermined stool types, classifying the stool into a plurality of predetermined stool colors, and classifying the urine into a plurality of predetermined urine colors. 4. The excrement analyzer of claim 3. (Appendix 5) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; 5. The excrement analyzer according to any one of claims 2 to 4. (Appendix 6) The other substances include at least the buttocks washer, When the classification result of the classification unit is classified as the buttocks washer, the classification unit stops the subsequent classification process, When the classification result by the classification unit is classified as the buttocks washer, the output unit outputs an excretion completion notification to a supervisor who supervises the user of the toilet. 6. The excrement analyzer of claim 5. (Appendix 7) the output unit outputs an excretion notification to a monitor who monitors a user of the toilet when the classification result by the classification unit is classified as excrement, the classification unit, after the output unit outputs the excretion notification, classifies each pixel classified as the excretion into one of feces, urine, or urine dripping, or one of feces, urine, feces and urine, or urine dripping, and also performs at least one of classifying the feces into a plurality of predetermined feces types, classifying the feces into a plurality of predetermined feces colors, and classifying the urine into a plurality of predetermined urine colors; The output unit outputs a classification result of the stool, the urine, and the urine dripping, and at least one classification result of the stool texture, the stool color, and the urine color. 3. An excrement analyzer as described in appendix 2. (Appendix 8) The output unit outputs the classification result by the classification unit as information including a classified image drawn in a different color for each classification. 8. An excrement analyzer according to any one of claims 1 to 7. (Appendix 9) The output unit notifies a monitor who monitors users of the toilet of the classification result by the classification unit. An excrement analyzer according to any one of claims 1 to 8. (Appendix 10) a determination unit that determines whether or not a user of the toilet has completed pre-treatment before a colonoscopy based on a classification result by the classification unit, The classification unit also classifies the feces as the excrement into a plurality of predetermined feces types and a plurality of predetermined feces colors, The output unit outputs the determination result of the determination unit as a classification result of the classification unit or as a part of the classification result of the classification unit. 10. An excrement analyzer according to any one of claims 1 to 9. (Appendix 11) A calculation unit is provided for calculating a stool volume, which is the volume of the stool, based on the classification result by the classification unit, The determination unit determines whether or not the user of the toilet has completed the pre-treatment based on the classification result by the classification unit and the amount of stool calculated by the calculation unit. 11. The excrement analysis device of claim 10. (Appendix 12) an input unit for inputting image data captured by an imaging device installed so as to include an imaging range of the excrement area in the toilet bowl; a classification unit that classifies imaged substances based on the imaging data input by the input unit; a determination unit that determines whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on the classification result by the classification unit; an output unit that outputs the determination result by the determination unit as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy and the subject; Equipped with the classification unit classifies the excrement of the imaged substance into one of feces, urine, and urine dripping, or into one of feces, urine, feces and urine, and urine dripping, and also classifies the feces into a plurality of predetermined fecal types and a plurality of predetermined fecal colors. A device for checking the condition before colonoscopy. (Appendix 13) The classification unit includes: a first classification unit that classifies the imaged substance into one of the excrement, foreign objects that are not permitted to be disposed of in the toilet, and other substances, and classifies the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips; a second classifying unit that classifies the imaged substance into the plurality of fecal textures and the plurality of fecal colors for the imaging data when the imaged substance is classified into the feces by the first classifying unit; Equipped with the determination unit determines that the user of the toilet has not completed pre-treatment when the classification result of the first classification unit is other than the feces, and determines whether the user of the toilet has completed the pre-treatment based on the classification result of the second classification unit when the classification result of the first classification unit is the feces. 13. A pre-colonoscopy condition checking device as described in appendix 12. (Appendix 14) a receiving unit which receives a classification result obtained by performing a first classification process for image data captured by an imaging device installed so that the imaging range of the excrement in the toilet bowl is included in the imaging range, classifying the imaged substance into the excrement, foreign objects that are not permitted to be disposed of in the toilet bowl, and other substances, and classifying the excrement into feces, urine, and urine drips, or feces, urine, feces and urine, and urine drips, and receives the image data when the classification result in the first classification process indicates that the excrement has been classified into feces; a second classification unit that classifies the imaged substance into a plurality of predetermined fecal properties and a plurality of predetermined fecal colors for the imaging data received by the receiving unit; A determination unit that determines whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on the classification result by the second classification unit; an output unit that outputs the determination result by the determination unit as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy and the subject; A condition checking device before a colonoscopy examination. (Appendix 15) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; 15. A device for checking a condition before a colonoscopy examination as described in appendix 13 or 14. (Appendix 16) The notification information includes a classification result by the second classification unit. 16. A pre-colonoscopy condition checking device according to any one of appendices 13 to 15. (Appendix 17) The notification information includes a classification image in which the classification results of the second classification unit are color-coded for each classification. 17. A pre-colonoscopy condition checking device as described in appendix 16. (Appendix 18) A calculation unit is provided for calculating a stool volume, which is the volume of the stool, based on the classification result by the second classification unit, The determination unit determines whether or not the user of the toilet has completed the pre-treatment based on the classification result by the second classification unit and the amount of stool calculated by the calculation unit. 18. A pre-colonoscopy condition checking device according to any one of claims 13 to 17. (Appendix 19) The present invention comprises an excrement analysis device and a server device connected to the excrement analysis device, The excrement analysis device comprises: an input unit for inputting image data captured by an imaging device installed so as to include an imaging range of the excrement area in the toilet bowl; a first classification unit that classifies the imaged substance into one of the excrement, foreign matter that is not permitted to be disposed of in the toilet, and other substances, and classifies the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips, for the image data input by the input unit; a transmission unit that transmits a classification result by the first classification unit to the server device, and transmits the imaging data to the server device when the classification result by the first classification unit indicates that the flight has been classified; The server device includes: a receiving unit that receives the classification result by the first sorting unit transmitted by the transmitting unit, and receives the imaging data transmitted by the transmitting unit when the classification result by the first sorting unit indicates that the flight has been classified; a second classification unit that classifies the imaged substance into a plurality of predetermined fecal properties and a plurality of predetermined fecal colors for the imaging data received by the receiving unit; A determination unit that determines whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on the classification result by the second classification unit; an output unit that outputs the determination result by the determination unit as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy and the subject; Equipped with A system for checking the condition before colonoscopy. (Appendix 20) the determination unit determines that the user of the toilet has not completed pre-treatment when the classification result received by the receiving unit is other than the stool, and determines whether the user of the toilet has completed the pre-treatment based on the classification result by the second classification unit when the receiving unit receives the imaging data. 20. A colonoscopy pre-conditioning system as described in appended claim 19. (Appendix 21) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; 21. A system for checking a condition before a colonoscopy examination according to claim 19 or 20. (Appendix 22) Inputting image data captured by an imaging device installed so as to include the excrement excretion area in the toilet bowl within an imaging range; A classification process is performed on the input imaging data to classify the imaged material on a pixel-by-pixel basis using semantic segmentation; outputting a classification result in the classification process; Excrement analysis method. (Appendix 23) the classification process classifies the imaged substance for each pixel into one of the excrement, a foreign object that is not permitted to be disposed of in the toilet, and other substances; 23. A method for analyzing excrement as described in appendix 22. (Appendix 24) The classification process classifies the excrement into any one of feces, urine, and urine dripping, or into any one of feces, urine, feces and urine, and urine dripping. 24. A method for analyzing excrement as described in appendix 23. (Appendix 25) The classification process includes at least one process of classifying the stool into a plurality of predetermined stool types, classifying the stool into a plurality of predetermined stool colors, and classifying the urine into a plurality of predetermined urine colors. 25. A method for analyzing excrement as described in appendix 24. (Appendix 26) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; 26. The method for analyzing excrement according to any one of claims 23 to 25. (Appendix 27) a determination process for determining whether or not a user of the toilet has completed a pre-treatment before a colonoscopy based on a classification result in the classification process; The classification process includes a process of classifying the stool as the excrement into a plurality of predetermined stool types and a plurality of predetermined stool colors, Outputting the classification result in the classification process means outputting the judgment result in the judgment process as a classification result in the classification process or as a part of the classification result in the classification process. 27. The method for analyzing excrement according to any one of claims 22 to 26. (Appendix 28) A calculation process for calculating a stool volume, which is the volume of the stool, based on a classification result in the classification process, The determination process determines whether or not the user of the toilet has completed the pre-treatment based on a classification result in the classification process and the amount of stool calculated in the calculation process. 28. A method for analyzing excrement described in appendix 27. (Appendix 29) Inputting image data captured by an imaging device installed so as to include the excrement excretion area in the toilet bowl within an imaging range; A classification process is carried out on the input image data to classify the imaged substances; execute a determination process for determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on a classification result of the classification process; outputting a result of the determination process as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy examination and the subject; the classification process is a process of classifying the excrement of the imaged substance into one of feces, urine, and urine dripping, or into one of feces, urine, feces and urine, and urine dripping, and includes a process of classifying the feces into a plurality of predetermined feces types and a plurality of predetermined feces colors. How to check your condition before a colonoscopy. (Appendix 30) The classification process includes: a first classification process for classifying the imaged substance into one of the excrement, foreign objects that are not permitted to be disposed of in the toilet, and other substances, and classifying the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips; A second classification process for classifying the imaging data into the plurality of stool types and the plurality of stool colors when the stool is classified into the plurality of stool types in the first classification process; Including, The determination process determines that the toilet user has not completed pre-treatment when the classification result of the first classification process is other than the feces, and determines whether the toilet user has completed the pre-treatment based on the classification result of the second classification process when the classification result of the first classification process is the feces. A method for checking the condition before a colonoscopy examination as described in Appendix 29. (Appendix 31) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; A method for checking the condition before a colonoscopy examination as described in Appendix 30. (Appendix 32) The excrement analysis device inputs imaging data captured by an imaging device installed so as to include the excrement excretion area in the toilet bowl as an imaging range, the excrement analyzer executes a first classification process for classifying the imaged substance into one of the excrement, foreign matter that is not permitted to be disposed of in the toilet, and other substances, and classifying the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips, for the input image data; the excrement analysis device transmits a classification result in the first classification process to a server device connected to the excrement analysis device, and transmits the imaging data to the server device when the classification result in the first classification process indicates that the stool has been classified; the server device receives a classification result in the first classification process transmitted from the excrement analyzer, and receives the imaging data transmitted from the excrement analyzer when the classification result in the first classification process indicates that the feces has been classified; the server device executes a second classification process for classifying the imaged substance into a plurality of predetermined fecal properties and a plurality of predetermined fecal colors for the received imaging data; the server device executes a determination process for determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on a classification result in the second classification process; the server device outputs a determination result in the determination process as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy examination and the subject. How to check your condition before a colonoscopy. (Appendix 33) receiving a classification result obtained by performing a first classification process for image data captured by an imaging device installed so as to include within an imaging range the excrement excretion area in the toilet bowl, classifying the imaged substance into one of the excrement, foreign objects that are not permitted to be disposed of in the toilet bowl, and other substances, and classifying the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips; receiving the imaging data when the classification result in the first classification process indicates that the flight has been classified; A second classification process is executed for the received imaging data to classify the imaged substance into a plurality of predetermined fecal textures and a plurality of predetermined fecal colors; execute a determination process for determining whether or not the user of the toilet has completed a pre-treatment before a colonoscopy based on a classification result of the second classification process; a determination result in the determination process is output as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy examination and the subject; How to check your condition before a colonoscopy. (Appendix 34) The determination process determines that the user of the toilet has not completed pre-treatment when the received classification result is other than the stool, and determines whether the user of the toilet has completed the pre-treatment based on the classification result in the second classification process when the image data is received. A method for checking a condition before a colonoscopy examination according to appendix 32 or 33. (Appendix 35) On the computer, Inputting image data captured by an imaging device installed so as to include the excrement excretion area in the toilet bowl within an imaging range; A classification process is performed on the input imaging data to classify the imaged material on a pixel-by-pixel basis using semantic segmentation; outputting a classification result in the classification process; A program for executing excrement analysis processing. (Appendix 36) the classification process classifies the imaged substance for each pixel into one of the excrement, a foreign object that is not permitted to be disposed of in the toilet, and other substances; 36. The program according to claim 35. (Appendix 37) The classification process classifies the excrement into any one of feces, urine, and urine dripping, or into any one of feces, urine, feces and urine, and urine dripping. 37. The program according to claim 36. (Appendix 38) The classification process includes at least one process of classifying the stool into a plurality of predetermined stool types, classifying the stool into a plurality of predetermined stool colors, and classifying the urine into a plurality of predetermined urine colors. 38. The program according to claim 37. (Appendix 39) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; The program according to any one of appendices 35 to 38. (Appendix 40) the excrement analysis process includes a determination process for determining whether or not the user of the toilet has completed pre-treatment prior to a colonoscopy based on a classification result in the classification process; The classification process includes a process of classifying the stool as the excrement into a plurality of predetermined stool types and a plurality of predetermined stool colors, Outputting the classification result in the classification process means outputting the judgment result in the judgment process as a classification result in the classification process or as a part of the classification result in the classification process. 40. The program according to any one of appendices 35 to 39. (Appendix 41) The excrement analysis process includes a calculation process of calculating a stool volume, which is the volume of the stool, based on a classification result in the classification process, The determination process determines whether or not the user of the toilet has completed the pre-treatment based on a classification result in the classification process and the amount of stool calculated in the calculation process. 41. The program according to claim 40. (Appendix 42) On the computer, Inputting image data captured by an imaging device installed so as to include the excrement excretion area in the toilet bowl within an imaging range; A classification process is carried out on the input image data to classify the imaged substances; execute a determination process for determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy based on a classification result of the classification process; outputting a result of the determination process as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy examination and the subject; the classification process is a process of classifying the excrement of the imaged substance into one of feces, urine, and urine dripping, or into one of feces, urine, feces and urine, and urine dripping, and includes a process of classifying the feces into a plurality of predetermined feces types and a plurality of predetermined feces colors. A program for performing a condition check process before a colonoscopy. (Appendix 43) The classification process includes: a first classification process for classifying the imaged substance into one of the excrement, foreign objects that are not permitted to be disposed of in the toilet, and other substances, and classifying the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips; A second classification process for classifying the imaging data into the plurality of stool types and the plurality of stool colors when the stool is classified into the plurality of stool types in the first classification process; Including, The determination process determines that the toilet user has not completed pre-treatment when the classification result of the first classification process is other than the feces, and determines whether the toilet user has completed the pre-treatment based on the classification result of the second classification process when the classification result of the first classification process is the feces. 43. The program according to claim 42. (Appendix 44) The other material includes at least one of a buttock washer, toilet paper, and material remaining after the waste is flushed; 44. The program according to claim 43. (Appendix 45) On the computer, receiving a classification result obtained by performing a first classification process for image data captured by an imaging device installed so as to include within an imaging range the excrement excretion area in the toilet bowl, classifying the imaged substance into one of the excrement, foreign objects that are not permitted to be disposed of in the toilet bowl, and other substances, and classifying the excrement into one of feces, urine, and urine drips, or into one of feces, urine, feces and urine, and urine drips; receiving the imaging data when the classification result in the first classification process indicates that the flight has been classified; A second classification process is executed for the received imaging data to classify the imaged substance into a plurality of predetermined fecal textures and a plurality of predetermined fecal colors; execute a determination process for determining whether or not the user of the toilet has completed a pre-treatment before a colonoscopy based on a classification result of the second classification process; a determination result in the determination process is output as notification information to at least one of a colonoscopy staff member who monitors the user of the toilet as a subject of a colonoscopy examination and the subject; A program for performing a condition check process before a colonoscopy. (Appendix 46) The determination process determines that the user of the toilet has not completed pre-treatment when the received classification result is other than the stool, and determines whether the user of the toilet has completed the pre-treatment based on the classification result in the second classification process when the image data is received. 46. ​​The program according to claim 45. [Explanation of symbols]

[0203] 1, 10 Excrement analyzer 1a, 5a Input section 1b, 5b classification section 1c, 5c Output section 5. Status check device (status check device before colonoscopy) 5d Judgment section 11 Second external box 11a CPU 11b Connector 11c,11d USB I / F 12 Box-to-box connections 13 First external box 14a WiFi module 14b Bluetooth module 15a Human Sensor 15b Second Camera 16a Distance sensor 16b First Camera 20 Toilet 21 Main unit 22 Toilet seat 23 Toilet seat cover 30 Toilet with excrement analyzer 40 Servers 41 Control section 42 Storage section 50 Terminal Equipment 100 devices 101 Processor 102 Memory 103 Communication Interface

Claims

1. an excretion information acquisition means for acquiring excretion information indicating the content of the excretion based on a classification result obtained by classifying the imaged substance on a pixel-by-pixel basis using semantic segmentation on image data captured by an imaging device installed so as to include the excretion range in the toilet bowl; A toilet user information acquisition means for acquiring toilet user information indicating a user of the toilet; Equipped with The classification includes classification of the stool as the excrement into a plurality of predetermined stool textures and a plurality of predetermined stool colors; The classification result includes a result of determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy, Recording the excretion information and the toilet user information in a server; Information processing system.

2. A human presence sensor and other imaging devices are installed in the toilet, the toilet user information acquisition means, when the human presence sensor detects a user of the toilet, captures a facial image of the user of the toilet with the other imaging device, and acquires the toilet user information based on the facial image. The information processing system according to claim 1 .

3. a wireless communication unit that communicates with a wireless communication tag carried by a user of the toilet; the toilet user information acquisition means acquires the toilet user information from the wireless communication tag via the wireless communication unit. The information processing system according to claim 1 .

4. acquiring excretion information indicating the content of the excretion based on a classification result obtained by performing semantic segmentation on image data captured by an imaging device installed so as to include within an imaging range the excretion range of the toilet bowl, and classifying the imaged substance on a pixel-by-pixel basis; Acquire toilet user information indicating a user of the toilet; Recording the excretion information and the toilet user information in a server; The classification includes classification of the stool as the excrement into a plurality of predetermined stool textures and a plurality of predetermined stool colors; The classification result includes a result of determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy. Information processing methods.

5. On the computer, acquiring excretion information indicating the content of the excretion based on a classification result obtained by performing semantic segmentation on image data captured by an imaging device installed so as to include within an imaging range the excretion range of the toilet bowl, and classifying the imaged substance on a pixel-by-pixel basis; Acquire toilet user information indicating a user of the toilet; Recording the excretion information and the toilet user information in a server; A process comprising: The classification includes classification of the stool as the excrement into a plurality of predetermined stool textures and a plurality of predetermined stool colors; The classification result includes a result of determining whether or not the user of the toilet has completed pre-treatment before a colonoscopy. A program that executes a process.

Images