Image acquisition device and toilet system

Abstract

To provide a technology that eliminates the need for a user to remove dirt from an imaging device by himself / herself.SOLUTION: An image acquisition device may have an imaging device that captures images of the inside of a toilet bowl, a cleaning mechanism that performs a cleaning process to remove dirt from the imaging device, and a control unit that causes the cleaning mechanism to perform the cleaning process.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The technology disclosed in this specification relates to an image acquisition device for imaging the inside of a toilet bowl.

Background Art

[0002] Patent Document 1 discloses an image sensor for imaging the inside of a toilet bowl. The image sensor is installed on the back surface of the toilet seat.

Prior Art Documents

Patent Documents

[0003]

Patent Document 【1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When dirt adheres to an imaging device such as an image sensor, it becomes difficult to image the intended image of the user inside the toilet bowl. The user needs to appropriately remove the dirt on the imaging device by themselves. This specification provides a technology that eliminates the need for the user to remove the dirt on the imaging device themselves.

Means for Solving the Problems

[0005] The technology disclosed in this specification relates to an image acquisition device. The image acquisition device may include an imaging device for imaging the inside of the toilet bowl of a toilet, a cleaning mechanism for performing a cleaning process to remove dirt on the imaging device, and a control device for causing the cleaning mechanism to perform the cleaning process.

[0006] The technology disclosed in this specification relates to a toilet system. The toilet system may include a toilet and the above-described image acquisition device.

Brief Description of the Drawings

[0007] [Figure 1] An outline perspective view of a flush toilet is shown. [Figure 2] This shows the configuration of the image acquisition device, the operating device, and the sensor unit. [Figure 3] The cross-sectional view of section III-III in Figure 1 is shown. [Figure 4] An enlarged diagram showing the positional relationship between the lens and the cleaning device is provided. [Figure 5] A flowchart of the process performed by the image acquisition device of the first embodiment is shown. [Figure 6] A flowchart of the process performed by the image acquisition device of the second embodiment is shown. [Modes for carrying out the invention]

[0008] (First Embodiment) As shown in Figure 1, the toilet system 2 comprises a toilet unit 10 and an operating device 80. As shown in Figure 2, the toilet system 2 further comprises a sensor unit 70. As shown in Figure 1, the toilet unit 10 comprises a toilet bowl 12 and a toilet seat 14. The toilet unit 10 further comprises a toilet lid and a tank, which are not shown. The toilet unit 10 may have a functional part that performs functions such as a washing function for washing the buttocks of the human body, a warm air drying function, and a deodorizing function. The toilet bowl 12 comprises a toilet bowl 12a. The toilet bowl 12a receives the excrement. Above the toilet bowl 12, the toilet seat 14 is rotatably positioned relative to the toilet bowl 12.

[0009] (Configuration of the image acquisition device) The toilet unit 10 is further equipped with an image acquisition device 20, and the image acquisition device 20 is attached to the toilet seat 14. As shown in Figure 2, the image acquisition device 20 comprises an imaging device 30, a cleaning device 40, a control device 50, and a communication module 60.

[0010] The imaging device 30 images the inside of the toilet bowl 12a and generates image data. The imaging device 30 outputs the processing result by processing the generated image data. The imaging device 30 comprises a lens 32, an image sensor 36, and an illumination unit 38. In the imaging device 30, the image sensor 36 generates image data using light incident from the subject through the lens 32. The image sensor 36 includes at least one of an area sensor and a line sensor. The lens 32 and the image sensor 36 are housed in the toilet seat 14. As shown in Figure 3, the lens 32 is directed from the back surface 14a of the toilet seat 14 into the toilet bowl 12a. The outer surface of the lens 32 is exposed to the toilet bowl 12a. The toilet seat 14 further houses an illumination unit 38. The illumination unit 38 has LEDs. The illumination unit 38 illuminates the inside of the toilet bowl 12a. When the LEDs of the illumination unit 38 are lit, the inside of the toilet bowl 12a is illuminated. The imaging device 30 captures images of objects within the imaging range AR1.

[0011] As shown in Figure 4, a cleaning device 40 is located adjacent to the imaging device 30 on the underside of the toilet seat 14. As shown in Figure 2, the cleaning device 40 includes a discharge pipe 42, a switching valve 44, and a control unit 46. The discharge pipe 42 receives water from the water supply pipe. One end of the discharge pipe 42 opens toward the lens 32 on the underside 14a. A switching valve 44 is located on the discharge pipe 42. The switching valve 44 switches the discharge pipe 42 between a connected state and a disconnected state. When the switching valve 44 is open, the discharge pipe 42 is maintained in a connected state. As a result, water is discharged from the water supply pipe through the discharge pipe 42 and from one end of the discharge pipe 42. The water discharged from the discharge pipe 42 collides with the lens 32. The lens 32 is cleaned by the water discharged from the discharge pipe 42. The water that collides with the lens 32 flows into the toilet bowl 12a. When the switching valve 44 is closed, the discharge pipe 42 is kept in a non-communicated state. As a result, the discharge of water from the opening of the discharge pipe 42 is stopped. The control unit 46 includes a CPU and memory for controlling the switching valve 44. The memory has at least one type of memory, such as a hard disk, ROM, and RAM. The memory stores computer programs and data for the control unit 46 to execute processing. The control unit controls the switching valve 44 by having the CPU execute processing according to the computer programs stored in the memory.

[0012] The image sensor 36, the illumination unit 38, and the cleaning device 40 are each communicateably connected to the control device 50. The control device 50 is housed in the toilet seat 14. The control device 50 includes a CPU 52 and a memory 54. The memory 54 has at least one type of memory, such as a hard disk, ROM, and RAM. The memory 54 stores computer programs and data for the control device 50 to perform processing. The control device 50 controls the imaging device 30, i.e., the image sensor 36, the illumination unit 38, and the cleaning device 40, by having the CPU 52 perform processing according to the computer programs stored in the memory 54.

[0013] The control device 50 includes an interface for communicating with the image acquisition device 20 and other devices via a communication module 60. The other devices include a sensor unit 70 and an operating device 80. For example, the communication module 60 may include an interface for performing at least one of wired communication and wireless communication in accordance with a well-known communication standard.

[0014] The sensor unit 70 includes one or more sensors for detecting a user using the toilet unit 10. The sensor unit 70 includes a seating sensor for detecting when a user is seated on the toilet seat 14. The seating sensor is located on the toilet unit 10. The seating sensor includes a switch that switches from off to on when a user sits on the toilet seat 14 and from on to off when a user leaves the toilet seat 14. The seating sensor may include at least one of a capacitive sensor whose capacitance changes depending on whether a user is seated on the toilet seat 14 or not, and an infrared distance measuring sensor.

[0015] Each of the one or more sensors included in the sensor unit 70 is connected to the control device 50 via the communication module 60 so as to be able to communicate. The one or more sensors transmit detection results to the control device 50. When the seat sensor switches between on and off, it transmits a signal to the control device 50 indicating whether the switch is on or off.

[0016] The operating device 80 is mounted on the wall of the toilet room. The operating device 80 is used to operate the toilet unit 10. The operating device 80 comprises an operating button 82, an output unit 84, a control unit 86, and a communication module 88. The operating button 82 includes a plurality of buttons for operating the toilet unit 10. The operating button 82 includes a toilet flushing button 82a for flushing the toilet unit 10, and a function operating button 82b for operating a function unit, such as a washing function for washing the buttocks of a person.

[0017] The communication module 88 includes an interface for communicably connecting the operation device 80 and other devices. The other devices include the toilet unit 10 and the image acquisition device 20. For example, the communication module 88 has the same configuration as the communication module 60.

[0018] The output unit 84 includes a display unit, a lighting unit, and an audio output unit. The display unit includes a liquid crystal display. The display unit displays various information on the liquid crystal display. The lighting unit includes an LED. The lighting unit notifies the user by lighting and extinguishing the LED. The audio output unit includes a speaker. The audio output unit outputs audio represented by audio data. The output unit 84 may include at least one of the display unit, the lighting unit, and the audio output unit.

[0019] The control unit 86 is connected to the operation button 82, the output unit 84, and the communication module 88 by wiring (not shown). The control unit 86 includes a CPU 86a and a memory 86b. The memory 86b has at least one type of memory such as a hard disk, a ROM, and a RAM. The memory 86b stores a computer program and data for the control unit to execute processing. The control unit 86 controls the operation button 82, the output unit 84, and the communication module 88 by the CPU 86a executing processing according to the computer program stored in the memory.

[0020] The control unit 86 transmits a signal corresponding to the operation of the operation button 82 by the user to the toilet unit 10 via the communication module 88. The control unit 86 causes the output unit 84 to output information represented by data received from the toilet unit 10 via the communication module 88.

[0021] (Processing executed by the image acquisition device) The control device 50 causes the imaging device 30 to image the user's excrement in the toilet bowl 12a, and acquires image data representing the image of the excrement from the imaging device 30. The control device 50 determines the characteristics of the excrement (e.g., the consistency of the stool) represented by the acquired image data. The control device 50 transmits, for example, the characteristic information representing the determined consistency of the stool to the operating device 80 via the communication module 60. When the control unit 86 of the operating device 80 receives the characteristic information, it causes the output unit 84 to output the consistency of the stool represented by the characteristic information. Specifically, the control unit 86 performs at least one of the following processes: displaying the consistency of the stool on the display unit, lighting up an LED of a color corresponding to the consistency of the stool, and outputting an audio statement representing the consistency of the stool. In addition to the consistency of the stool, the control device 50 may also determine the characteristics of the urine from the color of the urine, for example.

[0022] Referring to Figure 5, the cleaning and condition determination processes performed by the control device 50 will be explained. The control device 50 continuously performs the cleaning and condition determination processes while the toilet unit 10 is powered on. In this process, the control device 50 performs a process to determine the condition of the stool. The control device 50 may also determine other characteristics of the excrement besides the condition of the stool.

[0023] In the cleaning and condition determination process, in S12, the control device 50 waits to receive toilet usage completion information from the sensor unit 70. Specifically, the control device 50 waits to receive an off-switching signal from the seat sensor of the sensor unit 70, indicating that the switch has switched from on to off. When the control device 50 receives the off-switching signal from the seat sensor, it determines that it has acquired toilet usage completion information (YES in S12).

[0024] When toilet use completion information is received, in S14, the control device 50 acquires image data from the imaging device 30 representing the image of the toilet bowl 12a captured by the imaging device 30. In S16, the control device 50 determines whether or not dirt is present in the image represented by the acquired image data. The lens 32 is exposed to the toilet bowl 12a. Dirt such as excrement, flushing water from the toilet bowl 12a, and splashes from washing the buttocks of the human body may adhere to the lens 32. If dirt is present on the lens 32, the image of the user's excrement captured by the imaging device 30 may contain dirt. In this case, it may not be possible to accurately determine the characteristics of the stool.

[0025] In S16, the control device 50 uses the image represented by the image data acquired in S14 (hereinafter referred to as the "judgment image") to determine whether or not dirt is present in the image. Specifically, the memory 54 of the control device 50 has a plurality of indicators, including a first indicator 54a and a second indicator 54b, stored in advance. The plurality of indicators are stored in the memory 54 in advance by the manufacturer of the toilet unit 10. The first indicator 54a defines a predetermined abnormal area in the image represented by the image data. The abnormal area includes an area where user excrement that should be judged in the condition judgment process cannot exist, that is, an area on the toilet bowl 12a where excrement cannot adhere. The abnormal area is the peripheral area of ​​the judgment image, excluding the central part. If the abnormal area of ​​the judgment image contains an image of a different color from the color of the toilet bowl 12, the control device 50 determines that dirt is present in the image (YES in S16).

[0026] The second indicator 54b defines a threshold value indicating the proportion of the judgment image occupied by areas of colors other than the color of the toilet bowl 12. At the time when the image data representing the judgment image is acquired in S14, the user is not using the toilet unit 10. If there is no dirt attached to the lens 32, the judgment image will include the toilet bowl 12, but will not include any colors other than the color of the toilet bowl 12. For example, the threshold value of 5.0% is defined for the second indicator 54b. The control device 50 determines that there is dirt in the image if the proportion of the judgment image occupied by areas of colors other than the color of the toilet bowl 12 is greater than or equal to the threshold value (YES in S16). The control device 50 determines that there is no dirt in the image if the abnormal area of ​​the judgment image does not contain an image of a color different from the color of the toilet bowl 12, and the proportion of the judgment image occupied by areas of colors other than the color of the toilet bowl 12 is less than the threshold value (NO in S16).

[0027] If the answer to S16 is YES, in S18 the control device 50 instructs the cleaning device 40 to start the cleaning process and proceeds to S20. Specifically, the control device 50 sends an instruction signal to the control unit 46 to execute the cleaning process. When the control unit 46 receives the instruction signal, it switches the switching valve 44 from the closed state to the open state. While the switching valve 44 is maintained in the open state, water is discharged from the discharge pipe 42 toward the lens 32. This removes dirt from the lens 32. If the answer to S16 is NO, S18 is skipped and the process proceeds to S20. The control unit 46 of the cleaning device 40 stops the cleaning process after a predetermined cleaning period has elapsed since the start of the cleaning process and sends a completion signal to the control device 50 indicating that the cleaning process has been completed. The cleaning period is the period during which water is discharged from the discharge pipe 42 toward the lens 32 for cleaning. The cleaning period is stored in the control unit 46 in advance.

[0028] In S20, the control device 50 determines whether or not it is time to acquire image data during the cleaning period. The timing for acquiring image data is when the user uses the toilet 12. During the cleaning period, the control device 50 waits to receive a usage start signal from the sensor unit 70 indicating that the user will use the toilet 12. The usage start signal is an ON switching signal acquired from the seat sensor, indicating that the switch has been switched from OFF to ON.

[0029] If an ON switching signal is received during the cleaning period, the control device 50 determines that it is time to acquire image data (YES in S20). If YES in S20, in S22 the control device 50 determines whether or not the cleaning process is in progress. If it is determined that dirt is present in the judgment image (YES in S16), the cleaning process is started in S18. In this case, the control device 50 can determine that the cleaning process is in progress until a termination signal is received from the cleaning device 40. If it is determined that dirt is not present in the judgment image (NO in S16), the process in S18 is skipped and the cleaning process has not started. In this case, the cleaning process is not being performed at the time the process in S22 is executed. If it is determined that the cleaning process is in progress (YES in S22), the process proceeds to S24; if it is determined that the cleaning process is not in progress (NO in S22), the process proceeds to S26. In S24, the control device 50 instructs the cleaning device 40 to stop the cleaning process and proceeds to S26. Specifically, the control device 50 transmits a cancellation signal to the control unit 46 indicating that the cleaning process should be stopped. Upon receiving the cancellation signal, the control unit 46 stops the cleaning process by closing the switching valve.

[0030] In S26, the control device 50 acquires image data representing the images captured by the imaging device 30 while the user is using the toilet 12. Specifically, the control device 50 continuously acquires image data representing the images captured by the imaging device 30 until it receives an off-switching signal from the seat sensor indicating that the switch has switched from on to off. The imaging device 30 supplies the image data captured a predetermined number of times per second (e.g., 100 times) to the control device 50.

[0031] When the control device 50 receives the OFF switching signal, it terminates the acquisition of image data and proceeds to S28. In S28, the control device 50 uses the image data acquired in S26 to determine whether or not dirt is present in the image. Specifically, the control device 50 uses the first index 54a to perform the same processing as in S16 on multiple images represented by multiple image data acquired immediately before the completion of processing in S26, thereby determining whether or not dirt is present. In S28, unlike S16, the second index 54b is not used to determine whether or not dirt is present. This prevents situations where user excrement contained in the image is mistakenly identified as dirt.

[0032] If the image is determined to contain dirt (YES in S28), in S30, the control device 50 performs a first fecal type determination process using the image determined to contain dirt. Specifically, the control device 50 performs deep learning, for example, using a CNN (Convolutional Neural Network). The deep learning uses a large number of image data in which dirt and feces have been previously identified. The control device 50 uses the CNN to identify data representing dirt and data representing feces for each of the multiple image data acquired in S26. The control device 50 uses the data representing feces to determine the characteristics of the feces.

[0033] In S32, the control device 50 transmits information indicating that dirt is present on the lens 32 to the operating device 80, and proceeds to S36. When the control unit 86 of the operating device 80 receives the information indicating that dirt is present from the image acquisition device 20, it causes the output unit 84 to output information indicating that dirt is present on the lens 32. Specifically, the control unit 86 performs at least one of the following processes: displaying an image on the display unit indicating that dirt is present on the lens 32; turning on or blinking an LED in a manner corresponding to the presence of dirt on the lens 32; and outputting an audio signal indicating that dirt is present on the lens 32.

[0034] In S28, if it is determined that the image does not contain any dirt (NO in S28), in S34, the control device 50 performs a second stool type determination process to determine the properties of the stool using the image that was determined to be dirt-free. Specifically, the control device 50 identifies data representing excrement for each of the multiple image data acquired in S26. The control device 50 determines the properties of the stool using the data representing excrement and proceeds to S36. In S34, no data representing dirt is identified. In S36, the control device 50 transmits the data representing the properties of the stool, which has been determined in either S30 or S34, to the operating device 80 and returns to S12.

[0035] When the control unit 86 of the operating device 80 receives data representing the characteristics of the stool from the image acquisition device 20, it causes the output unit 84 to output the characteristics of the stool.

[0036] In S20, if no ON switching signal is received during the cleaning period (NO in S20), in S40, the control device 50 determines whether or not it has received an end signal from the cleaning device 40. If it is determined that an end signal has been received (YES in S40), the process proceeds to S42. If it is determined that an end signal has not been received (NO in S40), in S41, the control device 50 determines, similar to S22, whether or not the cleaning process is in progress. If it is determined that the cleaning process is in progress (YES in S41), the process returns to S20. If it is determined that the cleaning process is not in progress (NO in S41), the process returns to S12. In S41, if no end signal has been received (NO in S40) and the cleaning process is not in progress (NO in S41), then in S16, the judgment image does not contain any dirt (NO in S16), and there is no need to perform the cleaning process.

[0037] Once the cleaning process is complete, the control device 50 acquires image data in S42 and S44, similar to the processes in S14 and S16, and determines whether or not dirt is present. If it is determined that dirt is present (YES in S44), in S48, the control device 50 transmits information indicating the presence of dirt to the operating device 80, similar to S32. When the control unit 86 receives the information indicating the presence of dirt from the image acquisition device 20, it causes the output unit 84 to output information indicating that dirt is attached to the lens 32. If it is determined that no dirt is present (NO in S44), in S46, the control device 50 transmits cleaning completion information indicating that the cleaning process is complete to the operating device 80. When the control unit 86 receives the cleaning completion information from the image acquisition device 20, it causes the output unit 84 to output information indicating that the cleaning of the lens 32 is complete.

[0038] (effect) The cleaning device 40 can remove dirt adhering to the lens 32 by performing a cleaning process on the lens 32. This eliminates the need for the user to clean the lens 32 themselves.

[0039] The cleaning device 40 performs the cleaning process when the user is not using the toilet 12. By performing the cleaning process before the user uses the toilet 12, dirt can be removed from the lens 32. This prevents a situation where the excrement cannot be properly imaged when the user uses the toilet 12 due to the cleaning process.

[0040] The cleaning device 40 stops the cleaning process if the timing to acquire image data representing the excrement arrives during the cleaning process (YES in S20). This prevents a situation where the excrement cannot be properly imaged due to the cleaning process when the timing to acquire image data arrives.

[0041] When determining the properties of stool, if it is determined that there is dirt attached to the lens 32, that is, that the image data representing the image captured by the imaging device 30 contains dirt, the control device 50 determines the properties of stool while taking the dirt into consideration (S30). This prevents a situation where the properties of stool are not properly determined due to dirt attached to the lens 32. When determining the properties of stool, if it is determined that there is no dirt attached to the lens 32, the control device 50 determines the properties of stool without considering the dirt, that is, without distinguishing data representing dirt from the image data (S34). When it is determined that there is no dirt attached to the lens 32, the processing load on the control device 50 during the determination process can be reduced compared to when it is determined that there is dirt attached to the lens 32.

[0042] If the control device 50 determines that dirt is attached to the lens 32 after the cleaning process is completed (YES in S40) and after the cleaning process is stopped (S24) (YES in S44 and S28), it notifies the user of the presence of dirt on the lens 32 using the operating device 80. The user can then recognize that dirt is attached to the lens 32. The user can then clean the lens 32 themselves before the image is acquired by the image acquisition device 20. If the user is repeatedly notified that dirt is attached to the lens 32, that is, if the user determines that dirt is attached to the lens 32 after the cleaning process is completed (S40) and after the cleaning process is stopped (S24) (YES in S44 and S28), and this situation occurs repeatedly, the user can determine that the dirt attached to the lens 32 is difficult to remove. The control device 50 may change the information it transmits to the operating device 80 depending on whether it determines that dirt is attached to the lens 32 after the cleaning process is completed (YES in S40) or after the cleaning process is stopped (YES in S24) (YES in S44 and S28), or whether it determines that dirt is attached without the cleaning process being performed (YES in S44 and S28). Specifically, in addition to the information about the presence of dirt, it may transmit to the operating device 80 information indicating whether the cleaning process has been performed or not. The operating device 80 may output to the output unit 84 according to the information received from the control device 50.

[0043] (Correspondence) An example of the "first period" is the period from when the ON switching signal is received in the control device 50 until the OFF switching signal is received. An example of the "end information" is the OFF switching signal. An example of the "notification device" is the output unit 84, and an example of "dirt-related information" is information represented by an image indicating that dirt is attached to the lens 32, the lighting and flashing of an LED in a manner corresponding to the presence of dirt on the lens 32, and an audio message indicating that dirt is attached to the lens 32.

[0044] (Second Embodiment) The toilet unit 10 of this embodiment will be described with reference to Figure 6. In this embodiment, the cleaning and condition determination processes performed by the control device 50 differ from those of the first embodiment. The reference numerals for each component of the toilet unit 10 in this embodiment are the same as those used in the first embodiment. Unless otherwise specifically described below, each component of the toilet unit 10 in this embodiment is the same as each component of the toilet unit 10 in the first embodiment.

[0045] The control device 50 continuously performs cleaning and condition determination processing while the toilet unit 10 is powered on. In the cleaning and condition determination processing, in S52, the control device 50 determines whether or not a user has entered the toilet room where the toilet unit 10 is located. Specifically, the sensor unit 70 includes an open / close sensor that detects when the door of the toilet room where the toilet unit 10 is located is opened or closed. The control device 50 waits to receive a signal from the open / close sensor of the sensor unit 70 indicating that the door has been opened. When the control device 50 receives a signal from the open / close sensor indicating that the door has been opened, it determines that a user has entered the room (YES in S52).

[0046] In S54, the control device 50 acquires image data from the imaging device 30 that represents the image of the inside of the toilet bowl 12a captured by the imaging device 30, similar to S14. In S56, the control device 50 determines whether or not dirt is present in the image represented by the acquired image data. Specifically, the control device 50 determines whether or not the judgment image contains out-of-focus images. In the imaging device 30, the focus of the lens 32 is set near the bottom surface of the toilet bowl 12a in order to image the excrement. Dirt adhering to the lens 32 is out of focus and is captured as a blurred image. The control device 50 determines that dirt is present if the judgment image contains images with blurred outlines. In a modified example, the control device 50 may perform the same processing as in S16. If it is determined that dirt is present in the judgment image (YES in S56), in S58, the control device 50 causes the cleaning device 40 to start the cleaning process, similar to S18, and proceeds to S60. When the cleaning period has elapsed, the control unit 46 of the cleaning device 40 stops the cleaning process and sends a termination signal to the control device 50 indicating that the cleaning process has been completed.

[0047] In S60, the control device 50 determines whether or not the user has taken a seat. Specifically, similar to S20, the control device 50 determines whether or not it receives an ON switching signal from the seat sensor indicating that the switch has switched from OFF to ON. If an ON switching signal is received, it determines that the user has taken a seat (YES in S60) and proceeds to S61. The control device 50 executes the processes in S61, S62, S64, S65, S66, S67, S68, and S69, similar to the processes in S22, S24, S26, S28, S30, S32, S34, and S36, and then proceeds to S88.

[0048] If no ON switching signal is received from the sensor unit 70 in S60 (NO in S60), the process proceeds to S70. The control device 50 executes the processes of S70, S71, S72, S74, S76, and S78 in the same way as the processes of S40, S41, S42, S44, S46, and S48. After the process of S76 is executed, the control device 50 returns to S52. After the process of S78 is executed, the control device 50 proceeds to S90.

[0049] In S88, the control device 50, as in S12, waits to obtain toilet usage completion information from the sensor unit 70. If toilet usage completion information is obtained (YES in S88), in S90, the control device 50, as in S58, causes the cleaning device 40 to perform the cleaning process and returns to S52.

[0050] Specific embodiments have been described in detail. These are merely illustrative examples and do not limit the scope of the claims. The technology described in the claims includes various modifications and changes to the specific examples illustrated above. Modifications of the above embodiments are listed below.

[0051] (1) The sensor unit 70 may be equipped with another sensor in addition to the seating sensor, or it may be equipped with another sensor instead of the seating sensor. For example, the sensor unit 70 may be equipped with at least one of the following sensors: a human presence sensor for detecting a user in the toilet room, a motion sensor, a sensor for detecting the opening and closing of the toilet room door, a sensor for detecting knocking on the toilet room door, a sensor for detecting sounds outside and inside the toilet room, a load sensor for detecting the weight of the user in the toilet room, a distance sensor for detecting a user approaching the toilet room, a flow velocity sensor for detecting the flow velocity of the drainage from the toilet bowl 12, an odor sensor, and an illuminance sensor for detecting the brightness of the toilet room. The load sensor may be placed on the floor of the toilet room, inside the slippers in the toilet room, or on the mat laid in the toilet room.

[0052] (2) The control device 50 may communicate with external devices other than the sensor unit 70, such as a camera placed outside the toilet room, location information from the user's mobile device, a GPS terminal, and a wireless terminal capable of location identification. The control device 50 may use at least one of the image data and location information acquired from the external devices to determine whether the user enters the toilet room, whether the user uses the toilet 12, and whether the user defecates in the toilet 12.

[0053] (3) The control device 50 may be communicatively connected to at least some of the parts of the toilet unit 10 that perform washing functions for washing the buttocks of the human body, deodorizing functions, toilet flushing, etc. The control device 50 may acquire signals to indicate the start and end of operation of at least one of the communicable parts.

[0054] Using at least one of the above embodiments and modified versions (1) to (3), the control device 50 may determine at least one of the following situations in at least one of the processes S12, S20, S52, S60, and S88: the user entering the room, starting to use the toilet, ending use, sitting on the toilet seat 14, getting off the toilet seat 14, starting to defecate, and flushing the toilet.

[0055] In the first embodiment, in S12, the control device 50 may determine that it has acquired toilet usage completion information if a predetermined period of time has elapsed since the user left the toilet and the user has not been detected sitting down again. The predetermined period may be set to avoid situations in which the control device detects that the user has finished using the toilet 12 when the user has not finished using the toilet 12, for example, by standing up and sitting down again. The user leaving the toilet may be detected by an increase in the illuminance inside the toilet bowl 12a. In S12, in the first embodiment, the control device 50 may determine that it has acquired toilet usage completion information if at least one of the following events is detected: toilet flushing has been performed, washing of the buttocks has been performed, deodorization has been performed, the toilet lid has been closed, toilet paper has been used, or a predetermined period of time has elapsed since a user was no longer detected in the toilet room.

[0056] In the first embodiment, in S20, the control device 50 may determine that it is time to acquire image data when at least one of the following events is detected: the user enters the toilet room, the user starts using the toilet bowl 12, the user sits on the toilet seat 14, and the user defecates. Alternatively, the control device 50 may determine that it is time to acquire image data for the user to enter the toilet room when at least one of the following events is detected: a user is detected inside the toilet room, the toilet room door is opened, the user enters the toilet room, the user moves toward the toilet room, the toilet room door is knocked, a sound is detected inside the toilet room, a user's weight is detected inside the toilet room, or the illumination of the toilet room increases. The user entering the toilet room and moving toward the toilet room may be detected by at least one of the following: the detection result of the motion sensor, the user's movements included in the image captured by the camera, the position of an external device, and the distance between the user and the toilet room.

[0057] The control device 50 may determine that it is time to acquire image data in order to start using the toilet bowl 12 when at least one of the following events is detected: the toilet lid has been opened and the user has sat down. In addition to the seating sensor, the control device 50 may also determine that the user has sat down when at least one of the following events is detected: the user's weight is detected around the toilet bowl 12, the illumination inside the toilet bowl 12a decreases, and the user's movement (i.e., the position of at least one of the human presence sensor, motion sensor, external device, GPS terminal, and communication terminal). If the control device 50 does not detect that the user has sat down within a predetermined period of time after the toilet lid has been opened, or after a predetermined period of time has elapsed, it may determine that the user has entered the toilet room but has not sat down.

[0058] The system may determine that a user has defecated if at least one of the following is detected: the image represented by the image data acquired from the imaging device 30 contains excrement; the flow rate of the drainage from the toilet bowl 12 has increased; a predetermined period of time has elapsed since the user sat down; the sound of excrement falling into the toilet bowl 12 is detected; there is an odor; and at least one of the fluctuations in the water surface and waveform of the water in the toilet bowl 12 in the image represented by the image data acquired from the imaging device 30.

[0059] In S60 of the second embodiment, at least one of the above-described configurations may be used to determine when the user has started to excrete waste.

[0060] (4) The cleaning device 40 may perform a cleaning process to remove dirt by spraying a gas such as air onto the lens 32 from the discharge pipe 42. The cleaning device 40 may perform a cleaning process by spraying a chemical such as detergent onto the lens 32 from the discharge pipe 42. The cleaning device 40 may be equipped with at least one of a wiper and a brush that moves back and forth on the surface of the lens 32. The cleaning device 40 may perform a cleaning process by moving at least one of the wiper and the brush back and forth on the surface of the lens 32. The cleaning device 40 may discharge liquids such as water and detergent from the discharge pipe 42 when moving at least one of the wiper and the brush back and forth. The cleaning device 40 may perform a cleaning process by vibrating the lens 32.

[0061] (5) Lens 32 may be superhydrophobic. Lens 32 may be superhydrophilic. Lens 32 may be replaceable.

[0062] (6) At least one of the processes S16, S28, and S44 may be the same as the processes S66 and S74. The control device 50 may be trained by deep learning using a large number of image data in which dirt has been identified in advance in at least one of the processes S16, S28, S44, S66, and S74, and determine from the judgment image whether or not dirt is present. The control device 50 may also determine whether or not dirt is attached to the lens 32 without using the judgment image. For example, if the cleaning device 40 is equipped with at least one of a wiper and a brush, the control device 50 may determine that dirt is attached to the lens 32 if the resistance force of the movement of at least one of the wiper and brush is higher than a predetermined threshold representing the resistance force when the lens 32 is not dirty. For example, the control device 50 may determine whether or not dirt is present on the lens 32 from the change in surface free energy at the solid-liquid interface.

[0063] (7) In the contamination detection process of S30, the control device 50 may ignore the data of pixels in the image data that are included in the abnormal region defined by the first index 54a.

[0064] (8) The toilet bowl 12 may be equipped with an output unit similar to the output unit 84. The control device 50 may cause the output unit of the toilet bowl 12 to perform an output similar to that of the output unit 84 in at least one of the processes S32, S46, S48, S67, S76, and S78. In this modified example, the output unit of the toilet bowl 12 is an example of a "notification device," and the information output by the output unit of the toilet bowl 12 is an example of "dirt-related information." At least a part of the toilet bowl 12, such as the toilet seat 14, may vibrate. In this modified example, the toilet bowl 12 is an example of a "notification device," and the information represented by the vibration of the toilet bowl 12 is an example of "dirt-related information." The control device 50 may transmit information to an external device outside the tray room in at least one of the processes S32, S46, S48, S67, S76, and S78. External devices may include, for example, home appliances, PCs, smartphones, wearable devices, and other communication terminals located within the building where the toilet room is situated, as well as at least one output unit of an output device located on at least one of the door and outer wall outside the toilet room. In this modified example, the external device is an example of a "notification device," and the information output from the output unit of the external device is an example of "dirt-related information."

[0065] (9) The control device 50 may output information to a person other than the user in at least one of the processes S32, S46, S48, S67, S76, and S78. For example, the control device 50 may output to a management terminal used by the administrator of the toilet unit 10, a communication terminal of the cleaning company responsible for cleaning the toilet unit 10, and to the output unit of a predetermined terminal set in advance by the user, administrator, etc.

[0066] (10) The control device 50 may output the judgment image to the output unit.

[0067] (11) The control device 50 may output in different ways depending on whether it is outputting information about dirt in S32, S48, S67, and S78 or information about cleaning completion in S46 and S76. For example, the output unit 84 may light up an LED red when it should output information about dirt, and light up an LED blue when it should output information about cleaning completion. The output unit 84 may display a string of characters that suggests dirt is present, such as "dirty," when it should output information about dirt, and a string of characters that suggests cleaning is complete, such as "clean," when it should output information about cleaning completion.

[0068] (12) The control device 50 may execute the processes in S32, S48, S67 and S78, but may not execute the processes in S46 and S76. If cleaning is complete, output may not be sent to the output unit 84.

[0069] (13) The control device 50 and the control unit 46 may be composed of separate CPUs or may be composed of a single CPU.

[0070] (14) The control device 50 may decide whether or not to output in at least one of the processes S32, S46, S48, S67, S76, and S78, depending on the time of day, day of the week, etc. For example, the control device 50 may output during the time period when at least one of the users, administrators, and cleaning staff can visit the toilet 12, such as between 9:00 and 18:00, and not output at other times.

[0071] (15) The control device 50 does not need to determine whether or not dirt is present in the cleaning and condition determination process. For example, in the first embodiment, the process in S28 does not need to be performed. In this case, the control device 50 may perform the dirt determination process in S30 regardless of whether or not dirt is present.

[0072] (16) The toilet unit 10 does not have to be equipped with a cleaning device 40. In this case, the control device 50 does not have to perform the processes S12 to S18, S24, and S40 to S48 in the cleaning and condition determination process of the first embodiment.

[0073] (17) In S84, the control device 50 may perform the first stool type determination process instead of the second stool type determination process.

[0074] (18) The control device 50 may transmit at least one of the information prompting cleaning and the information indicating the degree of dirt along with the information indicating dirt in at least one of the processes S32, S48, S67 and S78. The control device 50 may transmit at least one of the information prompting cleaning and the information indicating the degree of dirt in place of the information indicating dirt in at least one of the processes S32, S48, S67 and S78. The output unit 84 may output the received information in a manner corresponding to the received information. In this modified example, the information notified in a manner corresponding to the information prompting cleaning and the information indicating the degree of dirt output by the output unit 84 is an example of "dirt-related information".

[0075] (19) The control device 50 may use the image acquired by the image acquisition device 20 to detect a blockage in the toilet bowl 12. The control device 50 may perform a cleaning process when it detects a blockage in the toilet bowl 12. The control device 50 may be switchable between a mode for determining the properties of the stool and a mode for detecting a blockage in the toilet bowl 12. With this configuration, when a blockage in the toilet bowl 12 is detected, dirt on the lens 32 can be removed.

[0076] (20) The imaging device 30 does not have to be a toilet seat-integrated type housed in the toilet seat 14. For example, the imaging device 30 may be retrofitted to the toilet seat 14.

[0077] The technical elements described in at least one of this specification and the drawings exhibit technical utility individually or in various combinations, and are not limited to the combinations described in the claims at the time of filing. The technologies illustrated in at least one of this specification and the drawings can achieve multiple objectives simultaneously, and achieving even one of these objectives itself constitutes technical utility. [Explanation of Symbols]

[0078] 2: Toilet system, 10: Toilet unit, 12: Toilet bowl, 12a: Toilet bowl, 14: Toilet seat, 20: Image acquisition device, 30: Imaging device, 32: Lens, 40: Cleaning device, 42: Discharge pipe, 44: Switching valve, 50: Control device, 70: Sensor unit

Claims

1. An imaging device that images the inside of the toilet bowl, A cleaning mechanism that performs a cleaning process to remove dirt from the imaging device, The cleaning mechanism is equipped with a control device that causes it to perform the cleaning process, The imaging device images the object to be imaged during a first period in which the object may be present in the toilet bowl. The control device is an image acquisition device that causes the cleaning mechanism to perform the cleaning process during periods other than the first period.

2. The image acquisition apparatus according to claim 1, wherein the control device causes the cleaning mechanism to terminate the cleaning process when the first period arrives during the cleaning process.

3. The control device is Obtain termination information indicating that the first period has ended, The image acquisition device according to any one of claims 1 to 2, wherein when the aforementioned termination information is obtained, the cleaning mechanism is instructed to perform the cleaning process.

4. The control device is The imaging device detects dirt adhering to it, When dirt is detected adhering to the imaging device, the cleaning mechanism is instructed to perform the cleaning process. The image acquisition apparatus according to any one of claims 1 to 3, wherein, if dirt is detected adhering to the imaging device after the cleaning process is performed, the device notifies the user via a notification device of dirt-related information relating to the presence of dirt on the imaging device after the completion of the cleaning process.

5. The object in question is the inside of a toilet bowl containing the user's excrement. The control device is When a user enters the toilet room where the toilet is located, the cleaning mechanism is instructed to perform the cleaning process. The image acquisition device according to any one of claims 1 to 4, wherein the cleaning mechanism completes the cleaning process before the user begins to excrete waste.

6. The image acquisition apparatus according to claim 5, wherein the control device causes the user to be notified via a notification device of dirt-related information relating to the presence of dirt on the imaging device after the completion of the cleaning process.

7. Toilet bowl and The system includes an image acquisition device, The image acquisition device is An imaging device for imaging the inside of the toilet bowl, A cleaning mechanism that performs a cleaning process to remove dirt from the imaging device, The cleaning mechanism is equipped with a control device that causes it to perform the cleaning process, The imaging device images the object to be imaged during a first period in which the object may be present in the toilet bowl. The control device causes the cleaning mechanism to perform the cleaning process during periods other than the first period in a toilet system.

Images