Monitoring system, monitoring device, and control method

Abstract

To provide a control device, a control method, a control program, and an information processing system with which it is possible to notify caretakers, etc. of information on image-capturing failures.SOLUTION: Provided is a monitoring system comprising: a first imaging unit and a second imaging unit having mutually different imaging fields of view and provided to the bed of a subject; an estimation unit for estimating an action of the subject on the basis of the image captured by at least the first imaging unit out of the first imaging unit and the second imaging unit; a detection unit for detecting a shield state in which at least one of the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit is shielded; and a notification unit for notifying detection information based on the detection result of the detection unit.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a monitoring system, a monitoring device, and a control method.

Background Art

[0002] In Japan, due to the improvement of living standards, the improvement of sanitary environment, and the improvement of medical standards accompanying the high economic growth after the war, the aging population has become remarkable. Therefore, combined with the decline in the birth rate, it has become an aging society with a high aging rate. In such an aging society, an increase in care recipients who require care such as nursing care is expected due to illness, injury, and aging.

[0003] Care recipients may be injured due to falling while walking or falling from a bed in facilities such as hospitals and elderly welfare facilities, or at home. If caregivers such as visiting nurses, nurses, facility staff, and family members can rush to the care recipient immediately according to the condition of the care recipient, it is possible to prevent the occurrence of injury. For this reason, the development of a system for detecting the condition of a care recipient from an image captured by an imaging device has been underway (for example, see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In such a system, if an appropriate image is not captured, there is a risk that the condition of the care recipient cannot be accurately detected. And if caregivers etc. do not notice the imaging defect, there is a risk that the condition of the care recipient cannot be accurately detected for a long time. Therefore, it is desirable for caregivers etc. to notice the imaging defect.

[0006] The present invention has been made to solve such problems. That is, an object of the present invention is to provide a monitoring system, a monitoring device, and a control method capable of notifying a caregiver or the like of an imaging failure.

Means for Solving the Problems

[0007] The above problems of the present invention are solved by the following means.

[0008] (1) A monitoring system including a first imaging unit and a second imaging unit provided on a bed of a subject, having different imaging fields of view from each other, and an estimation unit that estimates the behavior of the subject based on an image captured by at least the first imaging unit among the first imaging unit and the second imaging unit, a detection unit that detects a shielding state in which at least one of the imaging fields of view of the first imaging unit and the second imaging unit is shielded, and a notification unit that notifies detection information based on the detection result of the detection unit.

[0009] (2) The monitoring system according to (1) above, wherein the first imaging unit and the second imaging unit are provided at different positions on the bed.

[0010] (3) The monitoring system according to (1) above, wherein one of the first imaging unit and the second imaging unit is arranged to be able to image the subject on the bed, and the other is arranged to be able to image the feet of the subject outside the bed.

[0011] (4) The monitoring system according to (1) above, wherein the first imaging unit and the second imaging unit are provided on one side of the bed.

[0012] (5) The monitoring system according to (4) above, wherein one side of the bed is a short side of the bed.

[0013] (6) The monitoring system according to (1) above, wherein the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit do not overlap.

[0014] (7) The notification unit in the above-mentioned (1) monitoring system notifies the detection information when the shielding state is detected by the detection unit.

[0015] (8) When the shielding state in which the imaging field of view of the first imaging unit is shielded is detected, the detection information includes the first identification information of the first imaging unit, and when the shielding state in which the imaging field of view of the second imaging unit is shielded is detected, the detection information includes the second identification information of the second imaging unit. The monitoring system according to (7) above.

[0016] (9) The monitoring system according to (1) above further includes a determination unit that determines whether to generate the detection information, and when it is determined by the determination unit that the detection information is to be generated, the notification unit notifies the generated detection information.

[0017] (10) When the shielding state in which the imaging field of view of the first imaging unit is shielded is detected, the determination unit determines whether to generate the detection information based on the image captured by the second imaging unit. When the shielding state in which the imaging field of view of the second imaging unit is shielded is detected, the determination unit determines whether to generate the detection information based on the image captured by the first imaging unit. The monitoring system according to (9) above.

[0018] (11) The monitoring system according to (1) above further includes an adjustment unit that adjusts the timing for the notification unit to notify the detection information, and at the timing adjusted by the adjustment unit, the notification unit notifies the detection information.

[0019] (12) In the monitoring system according to (1) above, one of the imaging fields of view of the first imaging unit and the second imaging unit includes the area on the bed, and the other imaging field of view includes the area outside the bed.

[0020] (13) The detection unit in the above-mentioned (1) monitoring system detects the shielding state based on an image captured by at least one of the first imaging unit and the second imaging unit.

[0021] (14) A monitoring device comprising: a first imaging unit and a second imaging unit that have different imaging fields of view and are provided on a bed of a subject; an estimation unit that estimates the behavior of the subject based on an image captured by at least the first imaging unit among the first imaging unit and the second imaging unit; a detection unit that detects a shielding state in which at least one of the imaging fields of view of the first imaging unit and the second imaging unit is shielded; and an output unit that outputs detection information based on the detection result of the detection unit.

[0022] (15) A control method for a monitoring device having a first imaging unit and a second imaging unit that have different imaging fields of view and are provided on a bed of a subject, the method comprising: a step (a) of estimating the behavior of the subject based on an image captured by at least the first imaging unit among the first imaging unit and the second imaging unit; a step (b) of detecting a shielding state in which at least one of the imaging fields of view of the first imaging unit and the second imaging unit is shielded; and a step (c) of outputting detection information based on the detection result of the step (b).

Advantages of the Invention

[0023] In the monitoring system, monitoring device, and control method according to the present invention, a shielding state in which at least one of the imaging fields of view of the first imaging unit and the second imaging unit is shielded is detected, and detection information based on this detection result is notified. Therefore, it is possible to notify caregivers and the like of imaging failures.

Brief Description of the Drawings

[0024]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Embodiments for Carrying Out the Invention

[0025] Hereinafter, with reference to the drawings, a control device, a control method, a control program, and an information processing system according to an embodiment of the present invention will be described. In the drawings, the same elements are denoted by the same reference numerals, and redundant descriptions are omitted. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation and may be different from the actual ratios.

[0026] (Embodiment) FIG. 1 is a diagram showing the overall configuration of the monitoring system 1 according to an embodiment. The monitoring system 1 is used, for example, in a nursing care facility or the like. The monitoring system 1 includes a monitoring device 10, a server 20, a plurality of mobile terminals 30, and a wireless AP (access point) 31. The monitoring device 10, the server 20, and the mobile terminals 30 are communicably connected to each other via a LAN (Local Area Network) and the wireless AP 31. The monitoring device 10 is installed, for example, in the living room of the target person 80. The server 20 is managed, for example, by the administrator of the nursing care facility or the like. The mobile terminal 30 is carried by the caregiver 70 who takes care of or provides care for the target person 80. The target person 80 is a person in need of nursing care or care such as nursing or care. Here, the mobile terminal 30 corresponds to a specific example of the notification unit.

[0027] (Monitoring device 10) FIG. 2 is a block diagram illustrating the hardware configuration of the monitoring device 10. The monitoring device 10 includes an arithmetic control unit 101, a communication unit 102, a first camera 1031, a second camera 1032, and a third camera 1033. The arithmetic control unit 101, the communication unit 102, the first camera 1031, the second camera 1032, and the third camera 1033 are interconnected by a bus. Here, any one of the first camera 1031, the second camera 1032, and the third camera 1033 corresponds to a specific example of the first imaging unit of the present invention, and the other corresponds to a specific example of the second imaging unit.

[0028] The arithmetic control unit 101 includes a CPU (Central Processing Unit), a RAM (Random-access memory), a ROM (Read Only Memory), etc. The arithmetic control unit 101 may have an HDD (hard disk drive) or an SSD (Solid State Drive). The arithmetic control unit 101 performs control and arithmetic processing of each part of the monitoring device 10 according to a control program. The main functions of the arithmetic control unit 101 will be described later.

[0029] The communication unit 102 is an interface circuit for communicating with, for example, a server 20 via a LAN. The interface circuit is, for example, a LAN card and a wireless communication circuit, etc.

[0030] The first camera 1031, the second camera 1032, and the third camera 1033 image the monitoring area and output images. The first camera 1031, the second camera 1032, and the third camera 1033 may output image data. The images captured by the first camera 1031, the second camera 1032, and the third camera 1033 include still images and moving images. The moving images may include a plurality of still images intermittently captured in time series. The first camera 1031, the second camera 1032, and the third camera 1033 are near-infrared cameras or visible light cameras. Some of the first camera 1031, the second camera 1032, and the third camera 1033 may be near-infrared cameras and the rest may be visible light cameras. The first camera 1031, the second camera 1032, and the third camera 1033 are provided on the bed of the subject 80 (for example, the bed 90 in FIG. 3 described later).

[0031] FIG. 3 shows an example of the bed 90 of the subject 80. This bed 90 is provided in the living room of the subject 80. The bed 90 includes, for example, a frame portion 91, a headboard 92, a footboard 93, and a mattress 94. The bed 90 is, for example, an electric bed. For example, by tilting the mattress 94 at a portion corresponding to the back of the subject 80, a tilted (reclining) state can be created. The bed 90 has, for example, a rectangular planar shape.

[0032] The first camera 1031, the second camera 1032, and the third camera 1033 are attached to the frame portion 91, for example. The first camera 1031, the second camera 1032, and the third camera 1033 are provided at different positions on the bed 90. The first camera 1031 has an imaging field of view F1, the second camera 1032 has an imaging field of view F2, and the third camera 1033 has an imaging field of view F3. It is preferable that the imaging field of view F1, the imaging field of view F2, and the imaging field of view F3 do not overlap with each other. In other words, it is preferable that the first camera 1031, the second camera 1032, and the third camera 1033 image different regions from each other. Thereby, it becomes possible to image the subject 80 from multiple angles.

[0033] For example, the imaging field of view F2 of the second camera 1032 includes a region on the bed 90. For example, the imaging field of view F1 of the first camera 1031 includes one region outside the bed 90. For example, the imaging field of view F3 of the third camera 1033 includes the other region outside the bed 90.

[0034] The first camera 1031, the second camera 1032, and the third camera 1033 are provided along one side of the bed 90, for example. Thereby, it becomes possible to bring the settings of the first camera 1031, the second camera 1032, and the third camera 1033 closer to each other. Therefore, image processing of the images captured by each of the first camera 1031, the second camera 1032, and the third camera 1033 becomes easy. The first camera 1031, the second camera 1032, and the third camera 1033 are provided along the short side of the bed 90, for example. The first camera 1031, the second camera 1032, and the third camera 1033 are provided along the short side on the footboard 93 side, for example.

[0035] Figures 4 and 5 show other examples of the installation positions of the first camera 1031, the second camera 1032, and the third camera 1033. The first camera 1031, the second camera 1032, and the third camera 1033 may be provided along the short side on the head board 92 side (Figure 4). Alternatively, the first camera 1031, the second camera 1032, and the third camera 1033 may be provided across a plurality of sides of the bed 90 (Figure 5). For example, the first camera 1031 and the third camera 1033 may be provided on each of the opposing long sides of the bed 90, and the second camera 1032 may be provided on the short side of the bed 90. For example, a part of each of the imaging fields of view F1, F2, and F3 may overlap. For example, a part of the imaging field of view F1 and a part of the imaging field of view F3 overlap, and a part of the imaging field of view F2 and a part of the imaging field of view F3 overlap (Figures 4 and 5).

[0036] Figure 6 shows an example of an image Im1 captured by the first camera 1031, an image Im2 captured by the second camera 1032, and an image Im3 captured by the third camera 1033. In the images Im1 and Im3, a subject 80 outside the bed 90 is shown. It is preferable that at least the feet of the subject 80 are shown in the images Im1 and Im3. In other words, the first camera 1031 and the third camera 1033 are preferably arranged so as to be able to image the feet of the subject 80 outside the bed 90. Thereby, it becomes easier for the monitoring device 10 to estimate the actions of the subject 80 outside the bed 90, such as the subject 80 falling. In the image Im2, the subject 80 on the bed 90 is shown. In other words, the second camera 1032 is preferably arranged so as to be able to image the subject 80 on the bed 90. Thereby, it becomes easier for the monitoring device 10 to estimate the actions of the subject 80 on the bed 90, such as the subject 80 getting up.

[0037] FIG. 7 shows other examples of the image Im1 and the image Im3. Only the feet of the subject 80 outside the bed 90 may be shown in the images Im1 and Im3. In other words, the first camera 1031 and the third camera 1033 may be arranged to be able to image only the feet of the subject 80 outside the bed 90.

[0038] (Server 20) FIG. 8 is a block diagram showing the hardware configuration of the server 20. The server 20 includes an arithmetic control unit 201, a communication unit 202, a storage unit 203, and a display unit 204. The arithmetic control unit 201, the communication unit 202, the storage unit 203, and the display unit 204 are interconnected by a bus.

[0039] The arithmetic control unit 201 includes a CPU, a RAM, a ROM, etc. The arithmetic control unit 201 controls and performs arithmetic processing on each part of the server 20 according to a program.

[0040] The communication unit 202 is composed of an interface circuit for communicating with, for example, the mobile terminal 30 via a LAN.

[0041] The storage unit 203 is composed of an HDD or an SSD and stores various programs and various data.

[0042] The display unit 204 displays various information.

[0043] (Mobile Terminal 30) FIG. 9 is a block diagram showing the hardware configuration of the mobile terminal 30. The mobile terminal 30 includes a control unit 301, a wireless communication unit 302, a display unit 303, an input unit 304, and an audio input / output unit 305. The control unit 301, the wireless communication unit 302, the display unit 303, the input unit 304, and the audio input / output unit 305 are interconnected by a bus. The mobile terminal 30 may be composed of a portable communication terminal device such as, for example, a tablet computer, a smartphone, or a mobile phone.

[0044] The control unit 301 includes a CPU, a RAM, a ROM, and the like. The control unit 301 controls each part of the mobile terminal 30 and performs arithmetic processing according to a program.

[0045] The wireless communication unit 302 performs wireless communication with each device via the wireless AP 31 or directly by using wireless communication standards such as Wi-Fi and Bluetooth (registered trademark). In the mobile terminal 30, in addition to setting up the wireless LAN, access from the mobile terminal 30 to the server 20 may be enabled by inputting connection information with the server 20 (the IP address of the server 20 and the port number set in the server 20).

[0046] The display unit 303 and the input unit 304 are configured by, for example, a touch panel in which a touch sensor is superimposed on a display surface such as a liquid crystal. The display unit 303 and the input unit 304 display various information and accept various inputs.

[0047] The voice input / output unit 305 is, for example, a speaker and a microphone, and enables a voice call by the caregiver 70 between the mobile terminal 30 and other mobile terminals 30 via the wireless communication unit 302.

[0048] (Functions of the monitoring device 10) FIG. 10 is a block diagram illustrating the main functions of the arithmetic control unit 101 of the monitoring device 10. The arithmetic control unit 101 functions as, for example, an image acquisition unit 111, an estimation unit 112, a detection unit 113, a determination unit 114, and an output unit 115.

[0049] The image acquisition unit 111 acquires image information regarding images captured by the first camera 1031, the second camera 1032, and the third camera 1033 (for example, the images Im1, Im2, and Im3 in FIG. 6).

[0050] The estimation unit 112 estimates the actions of the target person 80 based on the image information acquired by the image acquisition unit 111. For example, the estimation unit 112 estimates the actions of the target person 80 by performing image analysis on the images captured by each of the first camera 1031, the second camera 1032, and the third camera 1033. The actions of the target person 80 are, for example, getting up, getting out of bed, lying position, falling, transferring, and falling down of the target person 80. The estimation unit 112 may estimate the actions of the target person 80 using a learned model. The estimation unit 112 can estimate the actions of the target person 80 using the images captured by at least one of the first camera 1031, the second camera 1032, and the third camera 1033.

[0051] The detection unit 113 detects a blocked state. The blocked state is a state in which at least one of the imaging fields of view F1 of the first camera 1031, F2 of the second camera 1032, and F3 of the third camera 1033 is blocked. The blocked state is a state in which the imaging fields of view F1, F2, and F3 are covered by an object or the like so that the actions of the target person 80 cannot be estimated from the captured images. The blocked state includes a state in which a part of each of the imaging fields of view F1, F2, and F3 is blocked. The blocked state is, for example, a state in which at least one lens portion of the first camera 1031, the second camera 1032, and the third camera 1033 is covered by an object in the living room of the target person 80. The objects in the living room of the target person 80 are, for example, futons, clothes, movable tables, desks, wheelchairs, portable toilets, and furniture. In this embodiment, since the monitoring device 10 includes this detection unit 113, it is possible to notify a caregiver 70 or the like of a problem with the imaging of the first camera 1031, the second camera 1032, or the third camera 1033.

[0052] The detection unit 113 detects the blocked state based on, for example, the images captured by the first camera 1031, the second camera 1032, and the third camera 1033. The detection unit 113 can detect the blocked state using, for example, known image processing techniques or image recognition techniques.

[0053] The detection unit 113 detects the shielding state based on, for example, the number of pixels of the segmentation data of a person, futon, bed, etc. in the image. For example, when 60% or more of the number of pixels in the image are occupied by the segmentation data of a futon, bed, etc., the detection unit 113 detects the shielding state.

[0054] The detection unit 113 may detect the shielding state by comparing images over time. For example, when object recognition such as a person, futon, or furniture is performed in the image at the first time, and object recognition is not performed in the image at the second time after the first time, the detection unit 113 detects the shielding state.

[0055] The detection unit 113 may detect the shielding state based on the pixel values of each pixel constituting the image. The detection unit 113 detects the shielding state based on, for example, the histogram or standard deviation of the pixel values of each pixel. For example, when the luminance of a predetermined ratio or more of the pixels constituting the image is below the threshold value, the detection unit 113 detects the shielding state. The detection unit 113 may detect the shielding state using the number of pixels extracted as edges from the image. For example, when the number of pixels extracted as edges is below the threshold value, the detection unit 113 detects the shielding state.

[0056] The detection unit 113 may detect the shielding state using the background difference method. The detection unit 113 detects the shielding state, for example, when there is an abnormality in the difference between the background image and the image at a predetermined time.

[0057] The detection unit 113 may detect the shielding state using the luminance change detection method. The detection unit 113 detects the shielding state, for example, when there is a region with a rapid luminance change in a plurality of images captured over time.

[0058] The detection unit 113 may detect the shielding state using the motion detection method. The detection unit 113 detects the shielding state, for example, when the motion pattern of the object detected from the image deviates from the normal motion pattern.

[0059] The detection unit 113 may detect the shielding state by using, for example, deep learning-based segmentation. The detection unit 113 can segment the shielded area in the image by using a deep learning model such as Mask R-CNN. At this time, the detection unit 113 detects the shielding state.

[0060] The detection unit 113 may detect the shielding state by using texture analysis, transparency analysis, etc. The detection unit 113 may detect the shielding state by using a machine learning model. For example, a normal image and an image in a shielding state are input into the machine learning model.

[0061] The determination unit 114 determines whether to generate detection information based on the detection result of the detection unit 113. By including this determination unit 114 in the monitoring device 10, it becomes possible to reduce unnecessary notifications to the caregiver 70.

[0062] The detection information includes, for example, information regarding the detection time by the detection unit 113, information regarding the imaging time of the image used by the detection unit 113, and information regarding whether the shielding state has been detected by the detection unit 113, etc. When the shielding state is detected by the detection unit 113, the detection information includes the identification information of the camera in which the shielding state has been detected. For example, when it is detected that the imaging field of view F1 of the first camera 1031 is shielded, the detection information includes the first identification information of the first camera 1031. When it is detected that the imaging field of view F2 of the second camera 1032 is shielded, the detection information includes the second identification information of the second camera 1032. When it is detected that the imaging field of view F3 of the third camera 1033 is shielded, the detection information includes the third identification information of the third camera 1033. When it is detected that the imaging fields of view of a plurality of cameras are shielded, the detection information includes the identification information of each of the plurality of cameras.

[0063] For example, when the detection unit 113 detects a shielded state, the determination unit 114 determines that it generates detection information. For example, when the shielded state is continuously detected for a predetermined time, the determination unit 114 determines that it generates detection information. For example, when the shielded state is continuously detected for 20 seconds or more, the determination unit 114 determines that it generates detection information.

[0064] The determination unit 114 may determine whether to generate detection information based on the action of the subject 80 estimated by the estimation unit 112. For example, when the shielded state is detected after the subject 80 is estimated to have woken up, the determination unit 114 determines that it generates detection information. Thereby, the occurrence of imaging defects after the subject 80 wakes up is suppressed, and it becomes easier for the caregiver 70 to notice when the subject 80 falls or the like. When the shielded state is detected while the subject 80 is estimated to be in the bed 90, the determination unit 114 may generate detection information.

[0065] The determination unit 114 may determine whether to generate detection information based on the setting state of the notification switch. For example, when the notification switch is in the on state, the determination unit 114 determines that it generates detection information. The notification switch is provided in, for example, any one of the monitoring device 10, the server 20, and the mobile terminal 30.

[0066] The determination unit 114 may determine whether to generate detection information based on the number of people present in the room of the subject 80. For example, when the number of people present in the room of the subject 80 is one, the determination unit 114 determines that it generates detection information. The determination unit 114 may determine whether to generate detection information based on the attributes of the people present in the room of the subject 80. For example, when the person present in the room of the subject 80 is the caregiver 70, the determination unit 114 determines that it does not generate detection information. This is because when there are a plurality of people in the room of the subject 80, the possibility of overlooking the fall or the like of the subject 80 is low.

[0067] The determination unit 114 may determine whether to generate detection information based on an image captured by a camera other than the camera in which the shielded state has been detected. For example, when a shielded state in which the imaging field of view F1 of the first camera 1031 is shielded is detected, the determination unit 114 determines whether to generate detection information based on an image captured by at least one of the second camera 1032 and the third camera 1033. For example, when a plurality of people are shown in an image captured by the second camera 1032 or the third camera 1033, the determination unit 114 determines not to generate detection information.

[0068] The output unit 115 outputs estimation information regarding the action of the subject 80 estimated by the estimation unit 112. The output unit 115 may output the estimation information when a predetermined action of the subject 80 is estimated. When it is determined by the determination unit 114 that detection information is to be generated, the output unit 115 generates and outputs the detection information. The output unit 115 outputs the estimation information and the detection information, for example, by transmitting the estimation information and the detection information to the mobile terminal 30 using the communication unit 102. The mobile terminal 30 that has received the estimation information causes, for example, the display unit 303 to display information indicating the action of the subject 80.

[0069] The mobile terminal 30 that has received the detection information causes, for example, the display unit 303 to display a predetermined icon and causes the audio input / output unit 305 to output a predetermined sound. Thereby, the caregiver 70 can notice a problem with imaging. Further, information indicating the first identification information, the second identification information, or the third identification information included in the detection information is displayed on the display unit 303. Thereby, the caregiver 70 can immediately grasp which of the first camera 1031, the second camera 1032, and the third camera 1033 has a problem.

[0070] The output unit 115 may output the detection information by transmitting the detection information to the server 20 using the communication unit 102. The server 20 that has received the detection information may, for example, cause a predetermined icon to be displayed on the display unit 204. The output unit 115 may output the detection information by transmitting the detection information to an external device using the communication unit 102. For example, the output unit 115 may transmit the detection information to a lamp or a speaker installed outside the living room of the subject 80.

[0071] (Control method of the monitoring device 10) FIG. 11 is a flowchart illustrating the processing procedure of the control method of the monitoring device 10. The processing of the flowchart in the figure is realized by the CPU of the arithmetic control unit 101 executing a control program.

[0072] First, the monitoring device 10 acquires image information regarding the images captured by each of the first camera 1031, the second camera 1032, and the third camera 1033 (step S101).

[0073] Next, the monitoring device 10 determines whether it is possible to estimate the behavior of the subject 80 based on the image information acquired by the processing of step S101 (step S102).

[0074] When the behavior of the subject 80 cannot be estimated (step S102: NO), the monitoring device 10 determines whether a shielding state has been detected (step S105). When it is determined that the shielding state has not been detected (step S105: NO), the monitoring device 10 returns to the processing of step S101. When it is determined that the shielding state has been detected (step S105: YES), the monitoring device 10 determines whether to generate detection information (step S106). When it is determined not to generate detection information (step S106: NO), the monitoring device 10 returns to the processing of step S101. When it is determined to generate detection information (step S106: YES), the monitoring device 10 generates and outputs the detection information (step S107) and ends the processing.

[0075] In the process of step S102, when it is determined that the behavior of the target person 80 can be estimated (step S102: YES), the monitoring device 10 estimates the behavior of the target person 80 (step S103) and outputs the estimated information (step S104). After that, the monitoring device 10 proceeds to the process of step S105.

[0076] (Functions and effects of the monitoring device 10 and the monitoring system 1) In the monitoring device 10 and the monitoring system 1 of the present embodiment, a shielding state in which at least one of the imaging fields of view F1, F2, and F3 is shielded is detected, and detection information based on this detection result is notified. Therefore, it is possible to notify a caregiver 70 or the like of a malfunction of imaging. Hereinafter, this function and effect will be described.

[0077] Compared with the case where one camera is installed in the living room of the target person, by installing a plurality of cameras, it becomes possible to estimate the behavior of the target person with higher accuracy. In particular, by providing a plurality of cameras around the bed, it becomes easier for the caregiver to notice events such as the target person getting up and getting out of bed, which require the assistance of the caregiver.

[0078] However, there are various items such as futons, clothes, and movable tables on the bed. Also, items such as wheelchairs and furniture may be placed around the bed. If these items cover the imaging field of view of the camera, the camera cannot capture an appropriate image of the target person, and it becomes difficult to estimate the behavior of the target person from the image. For this reason, there is a possibility that the caregiver may not be able to notice the target person's fall or the like.

[0079] On the other hand, in the monitoring device 10, it is detected that the imaging fields of view F1 of the first camera 1031, F2 of the second camera 1032, and F3 of the third camera 1033 are blocked by an object such as a futon. Therefore, a caregiver 70 or the like can notice that there is a problem with imaging and can move the blocking object. That is, it is possible to shorten the time during which the actions of the target person 80 cannot be estimated from the images. Thus, it is possible to reduce the risk that the caregiver 70 misses an important action such as the target person 80 falling.

[0080] Also, in the monitoring device 10, it is determined whether or not to generate detection information. For example, if the detection information is notified to the caregiver 70 or the like even though the blocked state is immediately resolved, there is a risk of burdening the caregiver 70 or the like with unnecessary work. By the monitoring device 10 determining whether or not to generate the detection information, it is possible to reduce unnecessary notifications to the caregiver 70 or the like.

[0081] As described above, in the monitoring device 10 and the monitoring system 1 of the present embodiment, a blocked state in which at least one of the imaging fields of view F1, F2, and F3 is blocked is detected, and detection information based on this detection result is notified. Therefore, it is possible to notify the caregiver 70 or the like of a problem with imaging.

[0082] Hereinafter, a modified example of the monitoring device 10 described in the above embodiment will be described. To avoid redundant explanation, detailed explanation of the same configuration as in the above embodiment will be omitted.

[0083] (Modified Example) FIG. 12 is a block diagram illustrating the main functions of the arithmetic control unit 101 in the monitoring device 10 according to the modified example. This arithmetic control unit 101 functions as an adjustment unit 116, for example, in addition to an image acquisition unit 111, an estimation unit 112, a detection unit 113, a determination unit 114, and an output unit 115.

[0084] The adjustment unit 116 adjusts the timing for notifying the detection information to the mobile terminal 30 or the like. For example, the adjustment unit 116 adjusts the timing for notifying the detection information by adjusting the timing for the output unit 115 to output the detection information. For example, the adjustment unit 116 adjusts the timing for the output unit 115 to output the detection information so that the detection information is output simultaneously with the estimated information.

[0085] In the monitoring device 10 according to this modification, similar to that described in the above embodiment, at least one of the imaging fields of view F1, F2, and F3 is detected in a shielded state where it is shielded, and the detection information based on this detection result is notified. Therefore, it is possible to notify the caregiver 70 or the like of the imaging defect.

[0086] In addition, since the timing for notifying the detection information is adjusted, it is possible to reduce the number of notifications to the caregiver 70 or the like. Thereby, it is possible to prevent important notifications from being buried among a large number of notifications.

[0087] The configurations of the monitoring device 10 and the monitoring system 1 described above have described the main configurations in explaining the features of the above embodiment and modification, and are not limited to the above configurations, and can be variously modified within the scope of the claims.

[0088] For example, the monitoring device 10 may include at least two cameras, or may include four or more cameras.

[0089] Also, the installation positions of the first camera 1031, the second camera 1032, and the third camera 1033 may be other positions. For example, the first camera 1031 and the second camera 1032 may be provided on the bed 90, and the third camera 1033 may be provided on the ceiling or wall of the room of the subject 80.

[0090] In the above-described embodiment, an example of determining whether the monitoring device 10 generates detection information has been described. However, the monitoring device 10 may always generate detection information. For example, the monitoring device 10 may generate detection information even when the shielding state has not been detected.

[0091] Also, the bed 90 may be configured to incline the mattress 94 and the frame portion 91 not only electrically but also by hydraulic pressure or the like. The bed 90 may be configured such that the mattress 94 and the frame portion 91 are fixed.

[0092] Also, the monitoring device 10 may be installed in a place other than a nursing care facility. For example, it may be installed in the home of the subject 80 or the like.

[0093] Also, in the above-described flowchart, steps other than the steps shown in the flowchart may be included, or some steps may not be included. Also, the order of the steps is not limited to the above-described embodiment. Further, each step may be executed as one step in combination with other steps, may be executed included in other steps, or may be divided into a plurality of steps and executed.

[0094] Also, the means and methods for performing the various processes in the above-described embodiment can be realized by either a dedicated hardware circuit or a programmed computer. The above program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is usually transferred and stored in a storage unit such as a hard disk. Also, the above program may be provided as a single application software, or may be incorporated as a single function into the software of a device such as its detection unit.

Explanation of Reference Numerals

[0095] 1 Monitoring system, 10 Monitoring device, 101 Arithmetic control unit, 102 Communication unit, 1031 First camera, 1032 Second camera, 1033 Third camera, 111 Image acquisition unit, 112 Estimation unit, 113 Detection unit, 114 Judgment unit, 115 Output unit, 116 Adjustment unit, 20 Server, 201 Arithmetic control unit, 202 Communication unit, 203 Memory unit, 204 Display unit, 30 Mobile terminal, 301 Control unit, 302 Wireless communication unit, 303 Display unit, 304 Input unit, 305 Audio input / output unit, 31 Access point, 90 Bed, 91 Frame part, 92 Headboard, 93 Footboard, 94 Mattress.

Claims

1. A monitoring system comprising: a first imaging unit and a second imaging unit that have different imaging fields of view and are provided on a bed of a subject; an estimation unit that estimates the behavior of the subject based on an image captured at least by the first imaging unit among the first imaging unit and the second imaging unit; a detection unit that detects a shielding state in which at least one of the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit is shielded; and a notification unit that notifies detection information based on a detection result of the detection unit.

2. The monitoring system according to claim 1, wherein the first imaging unit and the second imaging unit are provided at different positions on the bed.

3. The monitoring system according to claim 1, wherein one of the first imaging unit and the second imaging unit is arranged to be able to image the subject on the bed, and the other is arranged to be able to image the feet of the subject outside the bed.

4. The monitoring system according to claim 1, wherein the first imaging unit and the second imaging unit are provided on one side of the bed.

5. The monitoring system according to claim 4, wherein one side of the bed is a short side of the bed.

6. The monitoring system according to claim 1, wherein the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit do not overlap.

7. The monitoring system according to claim 1, wherein the notification unit notifies the detection information when the detection unit detects the shielding state.

8. When the shielding state in which the imaging field of view of the first imaging unit is shielded is detected, the detection information includes first identification information of the first imaging unit, When the shielding state in which the imaging field of view of the second imaging unit is shielded is detected, the detection information includes second identification information of the second imaging unit. The monitoring system according to claim 7.

9. further comprising a determination unit that determines whether to generate the detection information, The monitoring system according to claim 1, wherein when the determination unit determines that the detection information is to be generated, the notification unit notifies the generated detection information.

10. When the shielding state in which the imaging field of view of the first imaging unit is shielded is detected, the determination unit determines whether to generate the detection information based on an image captured by the second imaging unit. The monitoring system according to claim 9, wherein when the shielding state in which the imaging field of view of the second imaging unit is shielded is detected, the determination unit determines whether to generate the detection information based on the image captured by the first imaging unit.

11. The monitoring system further includes an adjustment unit that adjusts the timing for causing the notification unit to notify the detection information, and at the timing adjusted by the adjustment unit, the notification unit notifies the detection information. The monitoring system according to claim 1.

12. The monitoring system according to claim 1, wherein an imaging field of view of one of the first imaging unit and the second imaging unit includes an area on the bed, and an imaging field of view of the other includes an area outside the bed.

13. The monitoring system according to claim 1, wherein the detection unit detects the shielding state based on an image captured by at least one of the first imaging unit and the second imaging unit.

14. A monitoring device including a first imaging unit and a second imaging unit that have different imaging fields of view and are provided on a bed of a subject, an estimation unit that estimates the behavior of the subject based on at least an image captured by the first imaging unit among the first imaging unit and the second imaging unit, a detection unit that detects a shielding state in which at least one of the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit is shielded, and an output unit that outputs detection information based on a detection result of the detection unit is provided.

15. A control method for a monitoring device including a first imaging unit and a second imaging unit that have different imaging fields of view and are provided on a bed of a subject, the method comprising: step (a) of estimating the behavior of the subject based on at least an image captured by the first imaging unit among the first imaging unit and the second imaging unit; step (b) of detecting a shielding state in which at least one of the imaging field of view of the first imaging unit and the imaging field of view of the second imaging unit is shielded; and step (c) of outputting detection information based on a detection result of step (b) is included.

Images