Person-in-need-of-care watch system, person-in-need-of-care watch method, and person-in-need-of-care watch program

Abstract

To provide a person-in-need-of-care watch system making it possible to grasp a defect occurring in a drive unit that changes detecting directions, and upgrade the reliability in determining an action of a person in need of care.SOLUTION: A person-in-need-of-care watch system includes an imaging unit that obtains an image of a person in need of care, a determination unit that determines an action of the person in need of care on the basis of at least the image of the imaging unit, a human sensor that detects the position of the person in need of care, a drive unit that changes imaging directions of the imaging unit on the basis of detection information of the human sensor, a detection unit that detects a defect of the drive unit or human sensor, a first notification unit that notifies plural terminals of a result of the determination by the determination unit, and a second notification unit that notifies the plural terminals of a result of the detection by the detection unit.SELECTED DRAWING: Figure 15

Description

[Technical field]

[0001] The present invention relates to a system for monitoring a person requiring care, a method for monitoring a person requiring care, and a program for monitoring a person requiring care. [Background technology]

[0002] In Japan, the standard of living has improved, the sanitary environment has improved, and the level of medical care has improved, all of which have come with the rapid economic growth since the war. This, combined with the declining birth rate, has resulted in an aging society with a high aging rate. In such an aging society, it is expected that the number of people who require nursing care due to illness, injury, aging, etc. will increase.

[0003] In facilities such as hospitals and elderly care facilities, people who need care may fall while walking or fall off their beds and get injured. For this reason, systems are being developed that enable caregivers, nurses, and other staff to rush to the scene immediately in response to the actions of people who need care.

[0004] The following prior art is disclosed in the following Patent Document 1. The monitoring system includes a sensor device including an infrared array sensor unit that captures images to detect the presence of a person, and a drive mechanism that changes the detection direction of the infrared array sensor unit up, down, left, and right. The drive mechanism changes the detection direction of the infrared array sensor unit at regular time intervals to detect persons requiring care at multiple predetermined positions. Then, monitoring is performed by fixing the detection direction of the infrared array sensor unit to a predetermined position that has the longest presence capture time among the presence capture times of persons requiring care at the multiple predetermined positions. This makes it possible to monitor persons requiring care over a wide area with one sensor device. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2022-110634 A Summary of the Invention [Problem to be solved by the invention]

[0006] However, while there is no problem with the function of determining the behavior of the person requiring care based on the image, a malfunction may occur in the drive mechanism of the sensor that changes the detection direction. This malfunction may occur due to dust or the like accumulating in the drive mechanism, or wear of the parts of the drive mechanism due to repeated driving. However, even if such a malfunction occurs, the function of capturing images and determining behavior is normal, and monitoring continues within the imaging range. For this reason, it is difficult to notice that a problem has occurred in the monitoring system, and serious problems may occur, such as failure to detect important behavior such as falls. The above prior art cannot address such problems.

[0007] The present invention has been made to solve the above-mentioned problems, and aims to provide a system, a method, and a program for monitoring a person requiring care that can improve the reliability of determining the behavior of a person requiring care by making it possible to grasp a malfunction that occurs in a drive unit that changes the detection direction. [Means for solving the problem]

[0008] The above-mentioned object of the present invention is achieved by the following means.

[0009] (1) A care-requiring monitoring system that is installed in a nursing care facility and used to monitor care-requiring persons, the system comprising: an imaging unit that captures an image of the care-requiring person; a judgment unit that judges the behavior of the care-requiring person based on at least an image from the imaging unit; a human presence sensor that detects the position of the care-requiring person; a drive unit that changes the imaging direction of the imaging unit based on detection information from the human presence sensor; a detection unit that detects a malfunction of the drive unit or the human presence sensor; a first notification unit that notifies a plurality of terminals of the judgment result of the judgment unit; and a second notification unit that notifies a plurality of terminals of the detection result of the detection unit.

[0010] (2) The care-requiring person monitoring system described in (1) above, wherein the second notification unit notifies in a manner that distinguishes between the malfunction of the drive unit and the malfunction of the human presence sensor.

[0011] (3) A system for monitoring a person in need of care as described in (1) above, which has a memory unit that stores a correspondence between the malfunction and the cause of the malfunction and a method of dealing with the malfunction, and an identification unit that uses the stored correspondence to identify the cause of the malfunction and the method of dealing with the malfunction based on the detection result by the detection unit, and the second notification unit notifies the identified cause of the malfunction and the method of dealing with the malfunction together with the detection result.

[0012] (4) The system for monitoring a person in need of care described in (1) above, wherein the drive unit limits drive control to the imaging unit when the malfunction is detected by the detection unit.

[0013] (5) The system for monitoring a person requiring care described in (1) above, wherein the driving unit fixes the imaging direction to a direction in which the bed of the person requiring care is included in the image based on the detection result of the detection unit.

[0014] (6) A care-requiring person monitoring system as described in (4) above, further comprising a memory unit which stores a fixed direction in which the imaging direction is fixed when the malfunction is detected by the detection unit, and the drive unit fixes the imaging direction to the fixed direction when the malfunction is detected by the detection unit.

[0015] (7) A system for monitoring a person in need of care described in (4) above, wherein the drive unit limits the speed of movement of the imaging unit when changing the imaging direction when the malfunction is detected by the detection unit.

[0016] (8) A system for monitoring a person in need of care as described in (1) above, which has a control unit that, when the malfunction is detected by the detection unit, expands the imaging range of another imaging unit other than the imaging unit to overlap it with the imaging range of the imaging unit.

[0017] (9) A care-requiring person monitoring system as described in (1) above, having a control unit that starts recording or audio recording of the image captured by the imaging unit when the malfunction is detected by the detection unit, or that continues the recording or audio recording at all times.

[0018] (10) A system for monitoring a person in need of care as described in (1) above, having a cover that covers the drive mechanism of the drive unit.

[0019] (11) The care-requiring person monitoring system described in (1) above, wherein the second notification unit notifies the detection result to the terminal of at least one of a care facility, a seller of the care-requiring person monitoring system, and a company that performs maintenance on the care-requiring person monitoring system.

[0020] (12) The system for monitoring a person in need of care described in (1) above, wherein the second notification unit notifies the occurrence of the malfunction by generating an alarm signal from an alarm device that notifies the occurrence of the malfunction.

[0021] (13) The system for monitoring a person in need of care described in (4) above, wherein the drive unit restarts a program that drives the drive unit when the malfunction is detected by the detection unit.

[0022] (14) The system for monitoring a person in need of care described in (4) above, wherein the driving unit limits the range in which the imaging direction of the driving unit can be changed when the malfunction is detected by the detection unit.

[0023] (15) In the care-requiring person monitoring system described in (1) above, when the detection unit detects a malfunction of the human presence sensor, the drive unit changes the imaging direction based on the movement of the person in need of care in the image captured by the imaging unit, instead of the detection information of the human presence sensor.

[0024] (16) The system for monitoring a person in need of care described in (1) above, wherein the judgment unit changes the sensitivity of behavior judgment when the malfunction is detected by the detection unit.

[0025] (17) The care-requiring person monitoring system described in (1) above, wherein the first notification unit changes the notification criteria for the judgment result when the malfunction is detected by the detection unit.

[0026] (18) A method for monitoring a person requiring care, which is executed by a system for monitoring a person requiring care that is installed in a nursing care facility and used to monitor a person requiring care, comprising the steps of: (a) acquiring an image of the person requiring care captured by an imaging unit; (b) determining the behavior of the person requiring care based on at least the image captured by the imaging unit; (c) changing the imaging direction of the imaging unit by a driving unit based on detection information of a human presence sensor that detects the position of the person requiring care; (d) detecting a malfunction of the driving unit or the human presence sensor; (e) notifying a plurality of terminals of the determination result in step (b); and (f) notifying a plurality of terminals of the detection result in step (d).

[0027] (19) The method for monitoring a person requiring care according to (18) above, wherein in the step (f), notification is given in a manner that distinguishes between the malfunction of the drive unit and the malfunction of the human presence sensor.

[0028] (20) A method for monitoring a person in need of care as described in (18) above, comprising a step (g) of storing a correspondence between the malfunction and the cause of the malfunction and a method for dealing with the malfunction, and using the stored correspondence to identify the cause of the malfunction and the method for dealing with the malfunction based on the detection result in step (d), and in step (f), notifying the identified cause of the malfunction and the method for dealing with the malfunction together with the detection result.

[0029] (21) The method for monitoring a person in need of care described in (18) above, further comprising a step (h) of limiting drive control to the imaging unit when the malfunction is detected in the step (d).

[0030] (22) The method for monitoring a person requiring care described in (18) above, wherein the driving unit fixes the imaging direction to a direction in which the bed of the person requiring care is included in the image based on the detection result in step (d).

[0031] (23) A method for monitoring a person in need of care described in (21) above, which includes storing a fixed direction in which the imaging direction is fixed when the malfunction is detected in step (d), and causing the drive unit to fix the imaging direction to the fixed direction when the malfunction is detected in step (d).

[0032] (24) A method for monitoring a person in need of care described in (21) above, in which the drive unit is caused to limit the speed of movement of the imaging unit when changing the imaging direction when the malfunction is detected in step (d).

[0033] (25) A method for monitoring a person in need of care described in (18) above, in which, when the malfunction is detected in step (d), the imaging range of an imaging unit other than the imaging unit is expanded to overlap with the imaging range of the imaging unit.

[0034] (26) A method for monitoring a person in need of care described in (18) above, in which, when the malfunction is detected in step (d), recording or audio recording of the image captured by the imaging unit is started, or the recording or audio recording is continued continuously.

[0035] (27) The method for monitoring a person in need of care described in (18) above, wherein the monitoring system has a cover that covers a drive mechanism of the drive unit.

[0036] (28) In the step (d), the detection result is notified to a terminal of at least one of a care facility, a seller of the care monitoring system, and a company that performs maintenance on the care monitoring system, in the method for monitoring a person in need of care described in (18) above.

[0037] (29) The method for monitoring a person in need of care described in (18) above, wherein in the step (d), the occurrence of the malfunction is notified by generating an alarm signal from an alarm device that notifies the occurrence of the malfunction.

[0038] (30) The method for monitoring a person in need of care described in (21) above, further comprising a step (i) of restarting a program that drives the drive unit when the malfunction is detected in the step (d).

[0039] (31) The method for monitoring a person in need of care described in (21) above, wherein when the malfunction is detected in step (d), the driving unit is caused to limit a range within which the imaging direction of the driving unit can be changed.

[0040] (32) A method for monitoring a person requiring care described in (18) above, in which, when a malfunction of the human presence sensor is detected in step (d), the drive unit changes the imaging direction based on the movement of the person requiring care in the image captured by the imaging unit instead of the detection information of the human presence sensor.

[0041] (33) The method for monitoring a person in need of care described in (18) above, further comprising a step (j) of changing the sensitivity of the behavior determination in step (b) when the malfunction is detected in step (d).

[0042] (34) The method for monitoring a person requiring care described in (18) above, wherein when the malfunction is detected in step (d), the notification criteria for the judgment result in step (b) are changed.

[0043] (35) A program for monitoring a person in need of care, for causing a computer to execute the method for monitoring a person in need of care according to any one of (18) to (34) above. Effect of the Invention

[0044] A driving unit is provided that changes the imaging direction of the imaging unit that captures an image of the care recipient based on detection information from a human presence sensor that detects the position of the care recipient, and notifies the result of the determination of the care recipient's behavior based on the image and the detection result of a malfunction of the driving unit or the human presence sensor. This makes it possible to grasp malfunctions that have occurred in the driving unit that changes the detection direction, thereby improving the reliability of the determination of the care recipient's behavior. [Brief description of the drawings]

[0045] [Figure 1] 1 is a diagram showing an overall configuration of a system according to an embodiment; [Diagram 2] FIG. 1 is a diagram showing the interior of a room of a care recipient in which a monitoring device is installed. [Diagram 3] FIG. 2 is a block diagram showing a hardware configuration of the monitoring device. [Figure 4] FIG. 1 is an explanatory diagram showing a schematic configuration of a monitoring device. [Diagram 5] FIG. 4 is a block diagram showing the functions of a control unit. [Figure 6] FIG. [Figure 7] FIG. 13 is a diagram showing a human rectangle detected in a captured image. [Figure 8] FIG. 2 is a schematic diagram showing joint points. [Figure 9] FIG. 13 is a schematic diagram showing joint points detected based on a human rectangle; [Figure 10] 13 is a diagram showing a correspondence table in which correspondence between causes of malfunctions and countermeasures therefor related to detection results by a malfunction detection unit is registered. FIG. [Figure 11] FIG. 2 is a block diagram showing a hardware configuration of a server. [Figure 12] FIG. 2 is a block diagram showing the hardware configuration of a fixed terminal. [Figure 13] FIG. 2 is a block diagram showing a hardware configuration of a mobile terminal. [Figure 14] FIG. 13 is a diagram showing a defect detection result displayed on a mobile terminal. [Figure 15] 4 is a flowchart showing the operation of the system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0046] Hereinafter, a system for monitoring a person requiring care, a method for monitoring a person requiring care, and a program for monitoring a person requiring care according to an embodiment of the present invention will be described with reference to the drawings. Note that in the drawings, the same elements are given the same reference numerals, and duplicated explanations will be omitted. Also, the dimensional ratios in the drawings are exaggerated for the convenience of explanation, and may differ from the actual ratios.

[0047] FIG. 1 is a diagram showing the overall configuration of a system 1 according to an embodiment.

[0048] As shown in FIG. 1, the system 1 has a plurality of monitoring devices 10, a server 20, a fixed terminal 30, and one or more mobile terminals 40. These are connected to each other so as to be able to communicate with each other via a network 50 such as a LAN (Local Area Network), a telephone network, or a data communication network, by wire or wirelessly. The network 50 may include a relay device that relays communication signals. In the example shown in FIG. 1, the monitoring device 10, the server 20, the fixed terminal 30, and the mobile terminal 40 are connected to each other so as to be able to communicate with each other via the network 50 including an access point 51. The network 50 may be configured by a wireless LAN or the like (for example, a LAN according to the IEEE802.11 standard). The monitoring device 10 constitutes a monitoring system for a person requiring care. The monitoring system for a person requiring care may be configured by the monitoring device 10 and the server 20. The monitoring system for a person requiring care may be configured by the whole of the system 1, or a part including the monitoring device 10.

[0049] The system 1 is preferably placed in a nursing care facility such as a hospital or an elderly care facility. In the example shown in Fig. 1, the system 1 is placed in a building of a facility that has a plurality of rooms including a plurality of rooms and a care station in which a plurality of care recipients 80 reside. Hereinafter, the rooms in which the care recipients 80 reside will also be simply referred to as "rooms". The care recipients 80 are people who require care or nursing, for example, people requiring nursing care.

[0050] The monitoring devices 10 are arranged in the rooms of each floor (Floors A to C) which are the observation areas of the care recipients 80. In the example shown in FIG. 1, on floor A, four monitoring devices 10 are arranged in the rooms of the care recipients 80, namely, Mr. a, Mr. b, Mr. c, and Mr. d. Although not shown, a plurality of monitoring devices 10 are arranged in the rooms of the care recipients 80 on the other floors as well. Each of the care staff 70 carries a mobile terminal 40. The server 20 may be an external server connected to the network 50. Furthermore, the fixed terminal 30 may be omitted, and the server 20 or the mobile terminal 40 may have the function thereof.

[0051] (Monitoring device 10) Fig. 2 is a diagram showing the interior of a room of a care recipient 80 in which the watching device 10 is installed. Fig. 3 is a block diagram showing the hardware configuration of the watching device 10.

[0052] 2, the monitoring device 10 is placed, for example, on the ceiling of a room. The monitoring device 10 may also be placed on the upper part of a wall, or may be attached to a bed 90.

[0053] As shown in FIG. 3, the watching device 10 includes a control unit 11, a communication unit 12, a storage unit 13, and a camera unit 14, which are interconnected by a bus.

[0054] The control unit 11 is configured with a CPU (Central Processing Unit) and memories such as a RAM (Random Access Memory) and a ROM (Read Only Memory), and controls each part of the watching device 10 and performs calculation processing according to a program. The functions of the control unit 11 will be described in detail later.

[0055] The communication unit 12 is an interface circuit (eg, a LAN card) for communicating with the mobile terminal 40 and the like via the network 50.

[0056] The storage unit 13 stores various programs and various data and is configured, for example, by an SSD or HDD.

[0057] The camera unit 14 includes a camera 141, a driving mechanism 142, and a human sensor 143. The camera unit 140 is integrated with the monitoring device 10 and is disposed, for example, at a high position on the ceiling or wall of the room of the care recipient 80. The driving mechanism 142 constitutes a driving section. The camera 141 constitutes an imaging section.

[0058] The camera 141 captures a two-dimensional captured image 700 (see FIG. 7; hereinafter, also simply referred to as "captured image 700"). The captured image 700 includes moving images and still images. The captured image 700 may be a moving image made up of a plurality of frames (still images). The camera 141 is an infrared camera, but a visible light camera may be used instead, or both may be used in combination. For ease of explanation, the following description will be given taking as an example a case where the captured image 700 is a moving image made up of a plurality of frames captured by an infrared camera. The captured image 700 constitutes an image.

[0059] 4 is an explanatory diagram showing a schematic configuration of the watching device 10. As described above, the camera unit 140 can be integrated with the watching device 10. The driving mechanism 142 changes the imaging direction of the camera 141. The driving mechanism 142 can be configured to include a motor, gears, and a connecting member for connecting to the camera 141. The imaging direction of the camera 141 can be changed by rotating about a predetermined axis in the direction indicated by the arrow in FIG. 4, for example.

[0060] As shown in Fig. 4, camera unit 140 includes cover 144 that covers camera 141 and drive mechanism 142. Cover 144 may cover only drive mechanism 142. Cover 144 is colored gray in Fig. 4. Cover 144 is provided as necessary. Therefore, cover 144 does not necessarily have to be provided.

[0061] The cover 144 suppresses the transmission of visible light. By making it difficult for visible light to transmit through the cover 144, it becomes difficult for the movement of the drive mechanism 142 to be recognized from outside the monitoring device 10. The cover 144 can be made of a material with a visible light transmittance of 10% or less.

[0062] The human sensor 143 is a sensor for detecting the position of the care recipient 80. The human sensor 143 is configured, for example, by a stereo camera or a TOF (Time Of Flight) sensor. The human sensor 143 detects depth information for each pixel within a predetermined detection range to generate a distance image 800 (see FIG. 6 ; hereinafter, also simply referred to as “distance image 800”). The depth information is information on the distance from the camera unit 140 to an object (e.g., a person, a floor, a wall, a bed, and furniture). The depth information can be information on the distance in a direction perpendicular to the captured image 700 (normal direction). The distance image 800 is depth information for each pixel. Therefore, the distance image 800 can be said to be a three-dimensional image. The distance image 800 constitutes the detection information of the human sensor 143.

[0063] The detection direction and detection range of the depth information by the human presence sensor 143 may be fixed. The detection range of the depth information by the human presence sensor 143 may be wider than the shooting range of each frame of the captured image 700 by the camera 141. Note that the shooting range by the camera 141 is expanded by changing the shooting direction of the camera 141. The fixed detection range of the human presence sensor 143 may include the shooting ranges by the camera 141 (multiple shooting ranges before and after changing the shooting direction). Note that the distance image 800 does not include texture information including patterns, colors, and the like.

[0064] The motion sensor 143 may be replaced by other devices that detect the location of the care recipient 80. Other devices include, for example, beacons.

[0065] The function of the control unit 11 will now be described.

[0066] 5 is a block diagram showing the functions of the control unit 11. The control unit 11 functions as a position detection unit 111, a drive control unit 112, a recognition unit 113, an estimation unit 114, a determination unit 115, a malfunction detection unit 116, a specification unit 117, and a notification unit 118. The malfunction detection unit 116 constitutes a detection unit.

[0067] The position detection unit 111 acquires a distance image 800 from the human presence sensor 143 and detects the position of the care recipient 80 from the distance image 800 .

[0068] Fig. 6 is a diagram showing a distance image 800. In the distance image 800 of Fig. 6, depth information (the distance from the camera unit 140 to the object) is shown by color tones. Note that the distance image 800 shown in Fig. 6 is an image obtained by converting the distance image 800 in which the depth information is shown by differences in color into gray. For this reason, the tone density of the distance image 800 shown in Fig. 6 does not necessarily accurately reflect the depth information.

[0069] The position detection unit 111 may detect the position of the care recipient 80 by known techniques such as a time subtraction method (frame subtraction method) or a background subtraction method. The position detection unit 111 may detect the position of the care recipient 80 by using a trained model of a neural network that has been trained to detect a person from the distance image 800.

[0070] The drive control unit 112 rotates the camera 141 using the drive mechanism 142 so that the care recipient 80 detected by the position detection unit 111 is within the shooting range of the camera 141. Specifically, the drive control unit 112 orients the optical axis direction of the camera 141 toward the care recipient 80 using the drive mechanism 142 so that the care recipient 80 is positioned at the center of the shooting range of the camera 141. This changes the shooting direction of the camera 141 so that the care recipient 80 detected by the position detection unit 111 is included in the captured image 700. The distance image 800 and the captured image 700 are calibrated to correspond to each other's pixels.

[0071] The recognition unit 113 recognizes (detects) the care recipient 80 from the captured image 700.

[0072] FIG. 7 is a diagram showing a human rectangle 710 detected in a captured image 700. The recognition unit 113 may detect a care recipient 80 (person) included in the captured image 700 as a human rectangle 710 that is an area including the care recipient 80. Specifically, the recognition unit 113 detects an area in which an object exists on the captured image 700. Then, the recognition unit 113 detects the care recipient 80 of the person category by estimating the category of the object included in the detected area. The area in which the object exists is detected as a candidate rectangle including the object on the captured image. Among the detected candidate rectangles, a candidate rectangle estimated to be a person in the object category is detected as the human rectangle 710 to detect the care recipient 80. The recognition unit 113 may use a trained model of a neural network reflecting a dictionary (parameters) for detecting the human rectangle 710 from the captured image 700. Then, the recognition unit 113 may detect the human rectangle 710 from the captured image 700 using the trained model. As the trained model, for example, a known model, Region Proposal Network (RPN), may be used. The recognition unit 113 may output a human rectangle 710 as a combination of the captured image 700 and the coordinates of two opposing vertices e1 and e2 of the human rectangle 710.

[0073] The recognition unit 113 may detect the care recipient 80 as a silhouette from the captured image 700 using known techniques such as a time subtraction method (frame subtraction method) or a background subtraction method.

[0074] The estimation unit 114 estimates joint points 720 (see FIG. 8) of the care recipient 80 recognized by the recognition unit 113. Specifically, the estimation unit 114 estimates the joint points 720 from the human rectangle 710.

[0075] FIG. 8 is a schematic diagram showing an articulation point 720.

[0076] 8, the joint points 720 include a head joint point 720a, a shoulder joint point 720b, a waist joint point 720c, and a trunk joint point 720d. The distance between the two head joint points 720a reflects the size of the head.

[0077] FIG. 9 is a schematic diagram showing articulation points 720 detected based on a human rectangle 710.

[0078] In the example of FIG. 9, joint points 720 of the care recipient 80 in a sitting position on a chair are shown. The estimation unit 114 may estimate the joint points 720 using a trained model of a neural network that has been trained to detect the joint points 720 from the human rectangle 710. As the trained model, a known model such as Deep Pose may be used. Details of DeepPose are described in a known document (Alexander Toshev, et al. "DeepPose: Human Pose Estimation via Deep Neural Networks", in CVPR, 2014).

[0079] The determination unit 115 determines the behavior of the care recipient 80 based on the joint points 720 estimated by the estimation unit 114. For example, when the care recipient 80 is detected in a lying position from a state in which the care recipient 80 is detected in an upright position from the joint points 720 in a short time, the determination unit 115 may determine that the behavior is a "fall". The behavior of the care recipient 80 may be determined using a trained model of a neural network. The trained model is a model trained to estimate behavior based on a plurality of joint points 720, each of which is estimated from a plurality of captured images 700 adjacent in time series. The behavior of the care recipient 80 may be determined using a relative positional relationship between the joint points 720 and an object (e.g., a bed 90). For example, when a lying position is detected from the joint points 720 outside the bed 90 and in a relatively close range from the bed 90 in the captured image 700, the behavior may be determined as a "fall".

[0080] The malfunction detection unit 116 detects a malfunction of the drive mechanism 142 or the human presence sensor 143. The malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 and the human presence sensor 143. Malfunctions of the drive mechanism 142 include not only malfunctions of the drive mechanism 142 itself but also abnormalities of the drive mechanism 142 that do not involve unnecessary operation, such as generation of abnormal noise from the drive mechanism 142 due to driving of the camera 141. Malfunctions of the human presence sensor 143 include not only malfunctions of the human presence sensor 143 itself, such as not outputting the distance image 800, but also abnormalities of the human presence sensor 143 that do not result in malfunction, such as reduced sensitivity of the sensor.

[0081] The malfunction detection unit 116 detects a malfunction of the drive mechanism 142, for example, as follows. The malfunction detection unit 116 detects a malfunction of the drive mechanism 142 by detecting that the torque of the motor of the drive mechanism 142 is outside a predetermined normal range. The torque of the motor is detected by a known torque sensor. For example, the torque sensor measures, as an electrical signal, the minute twist of the shaft when torque is applied to the motor shaft, and detects the torque by converting the electrical signal into torque. In this case, the content of the malfunction can be detected as "abnormality in the drive mechanism."

[0082] The malfunction detection unit 116 can detect a malfunction of the drive mechanism 142 by detecting that the current (power) supplied to the motor of the drive mechanism 142 is outside a predetermined normal range. The current supplied to the motor is detected by a known method. For example, the current supplied to the motor is detected using a known current monitor circuit. In this case, the nature of the malfunction can be detected as "abnormality in the drive mechanism."

[0083] The malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 based on the three-dimensional position and orientation of the camera 141. For example, the malfunction detection unit 116 detects a malfunction of the drive mechanism 142 by detecting that the three-dimensional position and orientation of the camera 141 does not change even though a signal for rotating the camera 141 is output by the drive control unit 112. The three-dimensional position and orientation of the camera 141 is detected by a known method. For example, the three-dimensional position and orientation of the camera 141 is detected using a gyro sensor. In this case, the content of the malfunction can be detected as a "malfunction of the drive mechanism."

[0084] The malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 using the captured image 700 as follows. For example, the malfunction detection unit 116 detects whether the care recipient 80 is not detected in a temporally earlier frame of the adjacent frames of the captured image 700 and the care recipient 80 is detected in a later frame when the camera 141 is not rotating. The care recipient 80 is detected as, for example, a human rectangle 710. The state in which the camera 141 does not rotate can be detected, for example, using a gyro sensor. This phenomenon occurs when the care recipient 80 passes in front of the camera 141 that has stopped rotating due to a malfunction of the drive mechanism 142. Therefore, the malfunction detection unit 116 can detect a malfunction of the drive mechanism 142 by detecting the above phenomenon. In this case, the content of the malfunction can be detected as a "malfunction of the drive mechanism".

[0085] Furthermore, if the power to the monitoring device 10 is turned off, the camera 141 may not rotate, and this may be detected as a "malfunction of the drive mechanism."

[0086] The malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 by detecting the generation of an abnormal sound in the drive mechanism 142. For example, the malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 when the loudness of a sound detected as an electric signal by a microphone arranged near the drive mechanism 142 exceeds a predetermined threshold. In this case, the content of the malfunction may be detected as an "abnormality in the drive mechanism." Note that a case in which the speed of movement of the camera 141 falls below a predetermined threshold may also be detected as an "abnormality in the drive mechanism."

[0087] The malfunction detection unit 116 may detect a malfunction of the drive mechanism 142 when the number of times the care recipient 80 is tracked per hour by the drive control unit 112 falls below a predetermined threshold. The number of times of tracking can be detected as the number of times the camera 141 is rotated by the drive control unit 112. In this case, the content of the malfunction can be detected as "abnormality in the drive mechanism."

[0088] The malfunction detection unit 116 detects a malfunction of the human presence sensor 143, for example, as follows. One method for detecting a malfunction of the human presence sensor may be by monitoring the output signal of the human presence sensor. For example, a standard value for the output signal is set, and if the output signal is always weak or has a lot of noise, it is detected that there is a malfunction in the human presence sensor. The malfunction detection unit 116 may detect a malfunction of the human presence sensor 143 by detecting that the distance image 800 is not output. In these cases, the content of the malfunction may be detected as a "malfunction of the human presence sensor."

[0089] The malfunction detection unit 116 may detect a malfunction of the human sensor 143 using the captured image 700 as follows. For example, the malfunction detection unit 116 detects whether the care recipient 80 is not detected in a temporally earlier frame of the adjacent frames of the captured image 700 in a state where the camera 141 is not rotating, and the care recipient 80 is detected in a later frame. The state where the camera 141 is not rotating can be detected, for example, using a gyro sensor. This phenomenon occurs when the care recipient 80 passes in front of the camera 141 that has stopped rotating because the care recipient 80 is not detected in the distance image 800 due to a malfunction of the human sensor 143. Therefore, the malfunction detection unit 116 can detect a malfunction of the human sensor 143 by detecting the above phenomenon. In this case, the content of the malfunction can be detected as a "malfunction of the human sensor 143". Note that the above phenomenon also occurs when a malfunction occurs in the drive mechanism 142. Therefore, when the above phenomenon is detected, the malfunction detection unit 116 can detect it as a malfunction of at least one of the human sensor 143 and the drive mechanism 142. In other words, the content of the malfunction can be detected as at least one of "malfunction of the human sensor" and "malfunction of the drive mechanism."

[0090] The malfunction detection section 116 may also detect a malfunction of the camera 141. The malfunction detection section 116 detects that the captured image 700 is not output, thereby detecting the malfunction of the camera 141. In this case, the content of the malfunction may be detected as a "camera malfunction."

[0091] The malfunction detection section 116 may detect a malfunction of the camera 141 by detecting that the same image appears consecutively as the captured image 700. The same image appears consecutively as the captured image 700 when a state in which an average value of all pixels of differences between corresponding pixels in adjacent frames of the captured image 700 continues to be equal to or less than a predetermined threshold value for a predetermined period of time. In this case, the content of the malfunction may be detected as a "camera malfunction."

[0092] The identification unit 117 identifies the cause of a malfunction and a countermeasure method based on the detection result by the malfunction detection unit 116, using the correspondence relationship between the cause of the malfunction and the countermeasure method stored in the storage unit 13. The correspondence relationship between the cause of the malfunction and the countermeasure method can be registered by being stored in the storage unit 13 in advance.

[0093] Causes of the malfunction include, for example, the following: (1) The drive mechanism 142 is worn, broken, or deformed. (2) Dust or debris has accumulated in the drive mechanism 142. (3) The program that drives the drive mechanism 142 is damaged. The above cause (1) can occur in parts due to repeated use of the drive mechanism 142 over a long period of time. Cause (1) can cause the drive mechanism 142 to malfunction, produce abnormal noise, or reduce the drive speed. The above cause (2) causes dust or debris to interfere with the smooth operation of the drive mechanism 142. The above cause (3) can occur, for example, when the power supply to the monitoring device 10 is interrupted. Cause (3) can make it impossible for the drive mechanism 142 to rotate the camera 141.

[0094] Countermeasures for the above cause (1) include, for example, the following. If the malfunction detection unit 116 detects a "malfunction in the drive mechanism," a possible countermeasure is to "check the drive mechanism and then replace it." If the malfunction detection unit 116 detects an "abnormality in the drive mechanism," a possible countermeasure is to "check the drive mechanism and then replace it." Countermeasures for the above cause (2) include, for example, "check the drive mechanism and then remove dust and debris." Countermeasures for the above cause (3) include, for example, "restart the program."

[0095] Fig. 10 is a diagram showing a correspondence relationship table in which correspondence relationships between the causes of malfunctions and countermeasures related to the detection results by the malfunction detection unit 116 are registered. The correspondence relationship table may be stored in advance in the storage unit 13. For ease of explanation, Fig. 10 shows only some of the "detection results," "causes of malfunctions," and "countermeasures."

[0096] In the example of FIG. 10, "wear and damage to the drive mechanism" and "program corruption" are registered as causes related to the detection result "malfunction of the drive mechanism." In addition, "check the drive mechanism and replace it" and "restart the program" are registered as countermeasures corresponding to the causes "wear and damage to the drive mechanism" and "program corruption," respectively. In addition, "accumulation of dust and debris" and "deformation, wear and damage to the drive mechanism" are registered as causes related to the detection result "abnormality in the drive mechanism." In addition, "check the drive mechanism and remove the dust and debris" is registered as a countermeasure corresponding to the cause "accumulation of dust and debris." In addition, "check the drive mechanism and replace the drive mechanism" is registered as a countermeasure corresponding to the cause "deformation, wear and damage to the drive mechanism."

[0097] 10, "failure of the human sensor" is registered as a cause related to the detection result "malfunction of the human sensor." As a countermeasure for the cause "failure of the human sensor," "replace the human sensor." As a cause related to the detection result "abnormality of the human sensor," "adhesion of dust or debris" is registered. As a countermeasure for the cause "adhesion of dust or debris," "check the human sensor and remove the dust or debris."

[0098] The identifying unit 117 identifies the cause of the malfunction and a countermeasure based on the detection result of the malfunction detection unit 116 by using the correspondence relationship table.

[0099] For example, when the result of the detection of the malfunction by the malfunction detection unit 116 is "malfunction of the drive mechanism", "wear and damage of the drive mechanism" or "damage to the program driving the drive mechanism" is identified as the cause of the malfunction. In addition, when the damage of the program is detected by firmware or the like, "damage to the program driving the drive mechanism" may be identified. On the other hand, when the damage of the program is not detected, "wear and damage to the drive mechanism" may be identified. The identification unit 117 identifies a countermeasure corresponding to the identified cause of the malfunction. For example, when "wear and damage of the drive mechanism" or "damage to the program driving the drive mechanism" is identified, the countermeasures of "check the drive mechanism and then replace the drive mechanism" and "restart the program" are identified for each cause. When the cause of "wear and damage of the drive mechanism" is identified, the identification unit 117 identifies the countermeasure of "check the drive mechanism and then replace the drive mechanism". When the cause of "damage to the program driving the drive mechanism" is identified, the identification unit 117 identifies the countermeasure of "restart the program".

[0100] When the result of the malfunction detection by the malfunction detection unit 116 is "abnormality in the drive mechanism," the identification unit 117 identifies "accumulation of dust or debris" or "deformation, wear, or damage to the drive mechanism" as the cause of the malfunction. The identification unit 117 identifies "remove dust or debris after checking the drive mechanism" and "replace the drive mechanism after checking the drive mechanism" as countermeasures corresponding to the identified causes of the malfunction.

[0101] When the result of the malfunction detected by the malfunction detection unit 116 is "malfunction of the human presence sensor", the identification unit 117 identifies "failure of the human presence sensor" as the cause of the malfunction and "replacement of the human presence sensor" as the countermeasure.

[0102] If the result of the malfunction detection by the malfunction detection unit 116 is "abnormality in the human presence sensor," the identification unit 117 identifies "adhesion of dust or debris" as the cause of the malfunction and "remove the dust or debris after checking the human presence sensor" as the countermeasure.

[0103] The notification unit 118 notifies the plurality of mobile terminals 40 of the judgment result of the behavior of the care recipient 80 by the judgment unit 115 via the communication unit 12. Specifically, the notification unit 118 notifies the plurality of mobile terminals 40 of the behavior of the care recipient 80 judged by the judgment unit 115. The notification unit 118 can notify the judgment result of the behavior of the care recipient 80 by the judgment unit 115 to at least one terminal of the mobile terminal 40, the server 20, the fixed terminal 30, and other terminals. The notification unit 118 constitutes a first notification unit. Notification includes transmission to a terminal, transmission to a terminal and displaying the notification content on a display unit of the terminal, etc.

[0104] The notification unit 118 may notify only those judgment results of the behavior of the care recipient 80 by the judgment unit 115 that satisfy a predetermined notification criterion. The notification criterion may be set for each care recipient 80. For example, the notification unit 118 may include "notifying the behavior of going out" as a notification criterion for a care recipient 80 with dementia who may wander. On the other hand, for a care recipient 80 without dementia, "notifying the behavior of going out" may be excluded from the notification criterion.

[0105] The notification unit 118 notifies the multiple mobile terminals 40 of the detection result of the malfunction of the drive mechanism 142 and / or the human presence sensor 143 by the malfunction detection unit 116 via the communication unit 12. Specifically, the notification unit 118 notifies the multiple mobile terminals 40 of the contents of the malfunction of the drive mechanism 142 and / or the human presence sensor 143 detected by the malfunction detection unit 116. The notification unit 118 can notify the detection result of the malfunction of the drive mechanism 142 and / or the human presence sensor 143 by the malfunction detection unit 116 to one or more terminals, including at least one of the mobile terminal 40, the server 20, the fixed terminal 30, and other terminals. The notification unit 118 constitutes a second notification unit.

[0106] As described above, the first notification unit and the second notification unit may be configured by notification unit 118. In this manner, the first notification unit and the second notification unit may be the same unit, or may be different units.

[0107] The notification section 118 may notify the malfunction of the camera 141 detected by the malfunction detection section 116 to one or more terminals, including at least one of the mobile terminal 40, the fixed terminal 30, and other terminals.

[0108] The notification unit 118 may notify whether the malfunction of the drive mechanism 142 or the malfunction of the human sensor 143 is distinguishable. Specifically, the notification unit 118 notifies, for example, the contents of the malfunction including whether the malfunction is detected in the drive mechanism 142, the human sensor 143, or both. The notification unit 118 transmits the contents of the malfunction including whether the malfunction of the drive mechanism 142 or the human sensor 143 is a malfunction to the mobile terminal 40 or the like. As a result, the notification unit 118 may display on the display unit of the mobile terminal 40 whether the malfunction of the drive mechanism 142 or the human sensor 143 is a malfunction distinguishable. The notification unit 118 notifies, for example, the contents of the detected malfunction and the identified cause and countermeasure for the malfunction by a text such as "The drive mechanism is malfunctioning. The cause is thought to be wear and damage to the drive mechanism. Check the drive mechanism and then replace the drive mechanism." As a result, the detection result (content of the malfunction) of "malfunction of the drive mechanism" by the malfunction detection unit 116 is notified together with the cause of the malfunction, "wear and damage of the drive mechanism" and the countermeasure, "replace the drive mechanism after checking." In addition, the notification is made in such a way that it can be distinguished whether the malfunction is in the drive mechanism 142 or in the human sensor 143.

[0109] The notification unit 118 can notify the detection result to at least one of the terminals of the care facility, the seller of the monitoring device 10, and the company that maintains the monitoring device 10. The terminals of the care facility, the seller of the monitoring device 10, and the company that maintains the monitoring device 10 can be communicatively connected to the monitoring device 10 or the server 20 that is communicatively connected to the monitoring device 10.

[0110] The notification unit 118 may notify the occurrence of a malfunction by generating a notification signal from a notification device that notifies the occurrence of a malfunction. The notification signal includes a beep sound generated from a speaker that is built into the monitoring apparatus as a notification device.

[0111] When a malfunction is detected by malfunction detection unit 116, drive control unit 112 limits the drive control of camera 141 by drive mechanism 142. Drive control unit 112 can limit the drive control of camera 141 from the viewpoint of preventing the malfunction of drive mechanism 142 from becoming any worse. Alternatively, drive control unit 112 can limit the drive control of camera 141 from the viewpoint of continuing monitoring somehow even in a state in which a malfunction is occurring in drive mechanism 142.

[0112] (Limitation of drive control from the viewpoint of preventing further deterioration of the drive mechanism 142) When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 controls the drive mechanism 142 to fix the imaging direction of the camera 141 to a fixed direction. That is, when the malfunction of the drive mechanism 142 is detected, the drive control unit 112 changes the imaging direction of the camera 141 to a predetermined fixed direction and fixes it by the drive mechanism 142. The fixed direction can be set according to the state of each care recipient 80 and used by being stored in the storage unit 13 in advance. This suppresses the operation of the drive mechanism 142, so that the degree of wear and damage of the drive mechanism 142 can be maintained without further deterioration. For example, when an abnormality of the drive mechanism 142 is detected, such as abnormal noise caused by damage to the drive mechanism 142, the degree of damage to the drive mechanism 142 can be maintained, and malfunction (e.g., breakdown) of the drive mechanism 142 can be suppressed. The fixed direction can be a direction in which the bed 90 of the care recipient 80 is included in the captured image 700. For example, for a care recipient 80 who is often in bed 90 due to left-sided paralysis, the imaging direction of the camera 141 is fixed to a fixed direction in which the captured image 700 includes the bed 90. This makes it possible to prevent important behavior such as falling from the bed 90 from going undetected even if a malfunction occurs in the drive mechanism 142. For a care recipient 80 who has dementia and needs to be watched to make sure they do not go outside their room, the imaging direction of the camera 141 is fixed to a fixed direction in which the captured image 700 includes the door to the room. This makes it possible to prevent important behavior such as wandering of the dementia care recipient 80 from going undetected even if a malfunction occurs in the drive mechanism 142.

[0113] When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 may control the drive mechanism 142 to limit (reduce) the changeable range of the imaging direction of the camera 141. This suppresses the operation of the drive mechanism 142, which may enable the wear and tear of the drive mechanism 142 to be maintained without being further deteriorated.

[0114] When a malfunction of the drive mechanism 142 is detected by the malfunction detection unit 116, the drive control unit 112 limits the speed of movement of the camera 141 when the imaging direction of the camera 141 is changed by the drive mechanism 142. That is, when a malfunction of the drive mechanism 142 is detected, the drive control unit 112 limits the speed of movement of the camera 141 when the imaging direction of the camera 141 is changed by the drive mechanism 142 to a speed slower than the normal speed. This suppresses the operating speed of the drive mechanism 142 for moving the camera 141, so that the degree of wear and tear of the drive mechanism 142 can be maintained without further deterioration. In addition, the monitoring function of the monitoring device 10 can be maintained.

[0115] (Limitation of drive control from the viewpoint of continuing monitoring when a malfunction occurs in the drive mechanism 142) When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 controls the drive mechanism 142 to fix the imaging direction of the camera 141 to a fixed direction. This allows the monitoring function of the monitoring device 10 to continue for important behaviors of the care recipient 80, such as falling from the bed 90 or wandering around, even if a malfunction occurs in the drive mechanism 142.

[0116] When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 may control the drive mechanism 142 to limit (reduce) the range within which the imaging direction of the camera 141 can be changed. This suppresses the operation of the drive mechanism 142, which may prevent the wear and tear of the drive mechanism 142 from worsening any further. Furthermore, although the range of monitoring is reduced, the monitoring function of the monitoring device 10 can be maintained.

[0117] When malfunction detection section 116 detects a malfunction of drive mechanism 142, drive control section 112 limits the speed of movement of camera 141 when drive mechanism 142 changes the imaging direction of camera 141. This suppresses the operating speed of drive mechanism 142 for moving camera 141, so that the monitoring function can be maintained without further worsening the degree of wear and tear on drive mechanism 142.

[0118] When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 expands the imaging range of other imaging units (cameras of other devices) other than the camera 141 to overlap with the imaging range of the camera 141. This makes it possible to maintain the monitoring function of the monitoring device 10 before the malfunction occurred, even if the malfunction of the drive mechanism 142 is a malfunction. The other imaging units are preferably installed on the ceiling or the like of the same room as the monitoring device 10. For example, the other imaging units are cameras of other monitoring devices that are installed on the ceiling or the like together with the monitoring device 10 in a single multi-bed room and are communicably connected to the monitoring device 10.

[0119] When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the control unit 11 may start recording the captured image 700 captured by the camera 141 or recording the sound by the microphone. When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the control unit 11 may be configured to record or record the captured image 700 at all times. Usually, when a predetermined important behavior of the care recipient 80 is detected, the control unit 11 may record or record the captured image 700. This makes it possible to keep a record of important behavior of the care recipient 80 even if a malfunction occurs in the drive mechanism 142. Also, even if the drive mechanism 142 malfunctions, it is possible to maintain a minimum level of monitoring of the care recipient 80.

[0120] When the malfunction detection unit 116 detects a malfunction in the drive mechanism 142, the drive control unit 112 can perform a process to restore the drive mechanism 142 from the state in which the malfunction occurred.

[0121] (Processing for recovering from a state in which a malfunction has occurred in the drive mechanism 142) When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142, the drive control unit 112 restarts the program that drives the drive mechanism 142. If the malfunction of the drive mechanism 142 is caused by a corrupted program, the malfunction can be resolved by restarting the program that drives the drive mechanism 142.

[0122] When the malfunction detection section 116 detects a malfunction in the human sensor 143, the drive control section 112 can perform drive control from the viewpoint of continuing the monitoring.

[0123] (Drive control from the viewpoint of continuing monitoring when a malfunction of the human sensor 143 occurs) When the malfunction detection unit 116 detects a malfunction of the human sensor 143, the drive control unit 112 changes the imaging direction based on the movement of the care recipient 80 in the captured image 700 instead of the detection information of the human sensor 143. Specifically, the drive control unit 112 changes the imaging direction based on the movement of the human rectangle 710 or the joint point 720 in the captured image 700. The drive control unit 112 causes the camera 141 to capture the captured image 700 while changing the imaging direction of the camera 141 to the maximum extent within the range in which the imaging direction can be changed. When the care recipient 80 is detected as the human rectangle 710 or the joint point 720 in the captured image 700, the drive control unit 112 calculates the moving direction and moving speed of the care recipient 80. The drive control unit 112 controls the drive mechanism 142 so that the care recipient 80 is detected in the captured image 700 based on the moving direction and moving speed of the care recipient 80. As a result, even if the human sensor 143 malfunctions, the image capturing direction of the camera 141 is changed to continue tracking the care recipient 80, and the monitoring function of the monitoring device 10 is maintained.

[0124] When the malfunction detection unit 116 detects a malfunction in the drive mechanism 142 or the human sensor 143, the determination unit 115 can make a behavior determination from the perspective of continuing the watching over.

[0125] (Action determination from the viewpoint of continuing monitoring in a state where a malfunction of the driving mechanism 142 or the human sensor 143 occurs) The determination unit 115 changes the sensitivity of the behavior determination when the malfunction detection unit 116 detects a malfunction of the drive mechanism 142 or the human sensor 143. For example, the determination unit 115 lowers the sensitivity of the behavior determination when a malfunction of the drive mechanism 142 or the human sensor 143 is detected. Specifically, for example, when the determination unit 115 detects a lying posture within a predetermined time from a state in which the standing posture of the care recipient 80 is detected from the joint point 720, the determination unit 115 recognizes the behavior of "falling". Then, the predetermined time is set to a relatively short time. This makes it difficult to determine "falling". That is, the sensitivity of the behavior determination of "falling" is lowered. Therefore, erroneous detection of a predetermined behavior of the care recipient 80 is suppressed. The determination unit 115 may increase the sensitivity of the behavior determination when a malfunction of the drive mechanism 142 or the human sensor 143 is detected. In the example of the behavior determination of "falling" described above, the predetermined time is set to a relatively long time. This makes it easy to determine "falling". That is, the sensitivity of the behavior determination of “falling” is increased. As a result, even if only a part of the care recipient 80 is included in the captured image 700, for example, important behavior of the care recipient 80 can be detected.

[0126] When the malfunction detection unit 116 detects a malfunction of the drive mechanism 142 or the human sensor 143, the notification unit 118 changes the notification criteria of the determination result by the determination unit 115. For example, when a malfunction of the drive mechanism 142 or the human sensor 143 is detected, the notification unit 118 lowers the notification criteria. Specifically, for example, if "notify the behavior of getting out of bed" is excluded from the notification criteria, the notification criteria include "notify the behavior of getting out of bed". In this way, for example, when only a part of the care recipient 80 is included in the captured image 700, even if "fall" is erroneously detected as "getting out of bed", important behavior of the care recipient 80 can be notified. The notification unit 118 may raise the notification criteria when a malfunction of the drive mechanism 142 or the human sensor 143 is detected. Specifically, for example, when "notify the behavior of getting out of bed" is included in the notification criteria, the notification unit 118 excludes "notify the behavior of getting out of bed" from the notification criteria. This prevents erroneous detection of a predetermined behavior of the care recipient 80 caused by a malfunction of the drive mechanism 142 or the human sensor 143.

[0127] (Server 20) 11 is a block diagram showing a hardware configuration of the server 20. The server 20 includes a control unit 21, a communication unit 22, and a storage unit 23, which are interconnected by a bus.

[0128] The control unit 21 includes a CPU, a RAM, a ROM, etc. The control unit 21 controls each part of the server 20 and performs calculation processing according to a program.

[0129] The communication unit 22 is configured with an interface circuit for communicating with, for example, the watching device 10, the fixed terminal 30, and the mobile terminal 40, etc., via a LAN.

[0130] The storage unit 23 is configured with a HDD or SSD, and stores various programs and various data.

[0131] The control unit 21 may receive the judgment result of the behavior of the care recipient 80 from the watching device 10 via the communication unit 22. The control unit 21 may transmit the received judgment result of the behavior of the care recipient 80 to multiple mobile terminals 40 instead of the watching device 10. The control unit 21 may also transmit the received judgment result of the behavior of the care recipient 80 to the fixed terminal 30 instead of the watching device 10.

[0132] The control unit 21 may receive a detection result of a malfunction of the drive mechanism 142 and / or the human presence sensor 143 of the watching device 10 from the watching device 10 via the communication unit 22. The control unit 21 may transmit the received detection result of a malfunction to multiple mobile terminals 40 instead of the watching device 10. The control unit 21 may also transmit the received detection result of a malfunction to the fixed terminal 30 instead of the watching device 10.

[0133] (Fixed terminal 30) 12 is a block diagram showing a hardware configuration of fixed terminal 30. Fixed terminal 30 is, for example, a PC (Personal Computer), and includes a control unit 31, a communication unit 32, a storage unit 33, a display unit 34, and an input unit 35. The basic configurations of control unit 31, communication unit 32, and storage unit 33 are similar to the configurations of the corresponding elements of server 20, so duplicated explanations will be omitted.

[0134] The display unit 34 is configured, for example, by a liquid crystal display, and displays various information.

[0135] The input unit 35 is configured with a keyboard or a touch panel, and accepts operational input of various information by an administrator or the like.

[0136] The control unit 31 may receive the judgment result of the behavior of the care recipient 80 from the server 20 or the monitoring device 10 via the communication unit 32. The control unit 31 may receive the detection result of a malfunction of the drive mechanism 142 and / or the human sensor 143 of the monitoring device 10 from the server 20 or the monitoring device 10 via the communication unit 32.

[0137] The control unit 31 causes the display unit 34 to display the received malfunction detection result. Based on the received malfunction detection result, the control unit 31 may cause the display unit 34 to display whether the malfunction is in the drive mechanism 142 or in the human presence sensor 143 in a distinguishable manner. Based on the received malfunction detection result, the control unit 31 may output the details of the malfunction as sound from a speaker built into the fixed terminal 30. Based on the received malfunction detection result, the control unit 31 may output the occurrence of the malfunction as a beep from the speaker.

[0138] The control unit 31 causes the display unit 34 to display the received judgment result of the behavior of the care recipient 80.

[0139] (Mobile terminal 40) 13 is a block diagram showing the hardware configuration of the mobile terminal 40. The mobile terminal 40 includes a control unit 41, a wireless communication unit 42, a storage unit 43, an input display unit 44, and an audio input / output unit 45, which are connected to each other via a bus. The mobile terminal 40 may be configured by a portable communication terminal device such as a tablet computer, a smartphone, or a mobile phone. The basic configuration of the control unit 41 is similar to that of the control unit 21 of the server 20, and therefore a duplicated description will be omitted.

[0140] The wireless communication unit 42 wirelessly communicates with each device directly or via an access point 51 using wireless communication standards such as Wi-Fi and Bluetooth (registered trademark).

[0141] The storage unit 43 is configured with an SSD or an SD card, and stores various programs and various data.

[0142] The input display unit 44 is configured, for example, with a touch panel in which a touch sensor is superimposed on a display surface of a liquid crystal, etc. The input display unit 44 displays various information and accepts various inputs.

[0143] The audio input / output unit 45 is, for example, a speaker and a microphone, and enables the care staff 70 to make a voice call with another mobile terminal 40 via the wireless communication unit 42. The audio input / output unit 45 can enable the care staff 70 to make a voice call with the care recipient 80 between the monitoring device 10 and the server 20 via the wireless communication unit 42.

[0144] The control unit 41 may receive the judgment result of the behavior of the care recipient 80 from the server 20 or the watching device 10 via the wireless communication unit 42. The control unit 41 may receive the detection result of a malfunction of the drive mechanism 142 and / or the human presence sensor 143 of the watching device 10 from the server 20 or the watching device 10 via the wireless communication unit 42. The control unit 41 may receive the cause of the malfunction and a method for dealing with it together with the detection result.

[0145] The control unit 41 causes the input display unit 44 to display the received malfunction detection result. The control unit 41 may cause the input display unit 44 to display, based on the received malfunction detection result, whether the malfunction is in the drive mechanism 142 or in the human presence sensor 143 in a distinguishable manner. The control unit 41 may output the details of the malfunction as sound from the speaker of the audio input / output unit 45 based on the received malfunction detection result. The control unit 41 may also output the occurrence of the malfunction as a beep from the speaker of the audio input / output unit 45 based on the received malfunction detection result. The control unit 41 may cause the input display unit 44 to display the cause of the malfunction and the countermeasure method together with the detection result.

[0146] FIG. 14 is a diagram showing the fault detection results displayed on the mobile terminal 40. As shown in FIG.

[0147] In the example of Fig. 14, the text "A malfunction has occurred in the drive mechanism. The cause is thought to be wear and damage to the drive mechanism. Check the drive mechanism and then replace it." is displayed. This displays the detection result "malfunction of the drive mechanism" (details of the malfunction) together with the cause of the malfunction, "wear and damage to the drive mechanism," and the countermeasure, "check and then replace the drive mechanism." In addition, it is displayed so that it is possible to distinguish whether the malfunction is in the drive mechanism 142 or in the human sensor 143.

[0148] The control unit 41 causes the input display unit 44 to display the received judgment result of the behavior of the care recipient 80.

[0149] The operation of the system 1 will now be described.

[0150] 15 is a flowchart showing the operation of the system 1. This flowchart can be executed by the control unit 11 of the watching device 10 in accordance with a program.

[0151] The control unit 11 captures a captured image 700 by the camera 141 (S101).

[0152] The control unit 11 detects the position of the care recipient 80 from the distance image 800 acquired by the human sensor 143 (S102).

[0153] The control unit 11 changes the imaging direction of the camera 141 by the driving mechanism 142 based on the position of the care recipient 80 (S103). For example, the control unit 11 directs the optical axis direction of the camera 141 toward the care recipient 80 by the driving mechanism 142 so that the care recipient 80 is located at the center of the imaging range of the camera 141.

[0154] The control unit 11 determines the behavior of the care recipient 80 based on the captured image 700 and notifies the result of the determination (S104).

[0155] The control unit 11 detects a malfunction of the driving mechanism 142 or the human sensor 143 (S105).

[0156] When no malfunction is detected (S106: NO), the control unit 11 returns to step S101 and continues the process.

[0157] When a malfunction is detected (S106: YES), the control unit 11 identifies the cause of the malfunction and a countermeasure (S108). The cause of the malfunction and the countermeasure are identified based on the malfunction detection result, using the correspondence between the malfunction, the cause, and the countermeasure stored in the storage unit 13. In parallel with step S108, the control unit 11 limits the drive control of the drive mechanism 142 (S109).

[0158] The control unit 11 notifies the plurality of mobile terminals 40, etc. of the malfunction determination result, the cause of the malfunction, and a countermeasure method for the malfunction (S109). As a result, the control unit 11 causes the mobile terminals 40 to display the malfunction determination result in a manner that allows the user to distinguish whether the malfunction is in the drive mechanism 142 or in the human presence sensor 143.

[0159] The control unit 11 determines whether or not an instruction to end watching has been received by the communication unit 12 (S110). The instruction to end watching can be input at the fixed terminal 30 and transmitted to the watching device 10.

[0160] When the control unit 11 determines that there is no instruction to end the monitoring (S110: NO), the control unit 11 returns to step S101 and continues the process. When the control unit 11 determines that there is an instruction to end the monitoring (S110: YES), the control unit 11 ends the process.

[0161] (Modification) As described above, the estimation unit 114 of the monitoring device 10 detects the joint points 720 from the human rectangle 710. Since the human rectangle 710 is a two-dimensional image, the joint points 720 also have two dimensions. The determination unit 115 determines the behavior of the care recipient 80 based on the two-dimensional joint points 720 estimated by the estimation unit 114.

[0162] In this modification, a projector is included in the configuration of the camera unit 14. Then, the projector irradiates an infrared dot pattern in a range overlapping with the imaging range of the camera 141. Then, the camera 141 acquires an image of the dot pattern irradiated on the care recipient 80 together with the captured image 700. The depth information of the care recipient is calculated by utilizing the fact that the geometric shape of the captured dot pattern changes depending on the distance to the care recipient 80 on which the dot pattern is irradiated. Then, the two-dimensional joint points 720 are converted into three-dimensional joint points using the depth information corresponding to each joint point 720. The determination unit 115 determines the behavior of the care recipient 80 based on the three-dimensional joint points 720. This can improve the estimation accuracy of the behavior of the care recipient 80.

[0163] The two-dimensional joint points 720 may be converted into three-dimensional joint points using the distance image 800 acquired by the motion sensor 143. In this case, calibration is required to associate each pixel of the captured image 700 with each pixel of the distance image 800.

[0164] The embodiment provides the following advantages.

[0165] The device is equipped with a drive unit that changes the imaging direction of the imaging unit that captures an image of the care recipient based on detection information from a human presence sensor that detects the position of the care recipient.The device notifies the result of the determination of the care recipient's behavior based on the image and the detection result of a malfunction of the drive unit or the human presence sensor.This makes it possible to grasp the malfunction that has occurred in the drive unit that changes the detection direction, thereby improving the reliability of the determination of the care recipient's behavior.

[0166] Furthermore, the second notification unit, which notifies the result of detection of a malfunction of the drive unit or the human presence sensor, notifies in a distinguishable manner whether the malfunction is in the drive unit or the human presence sensor, thereby making it possible to easily and accurately grasp whether the malfunction is in the drive unit or the human presence sensor.

[0167] In addition, the system stores a correspondence relationship between the malfunction and the cause and the countermeasure for the malfunction. The system then uses the stored correspondence relationship to identify the cause and the countermeasure for the malfunction based on the malfunction detection result. The system then notifies the user of the cause and the countermeasure for the malfunction together with the detection result. This allows the system to promptly respond to the malfunction.

[0168] In addition, when a malfunction is detected, the drive control for the imaging unit is limited, which makes it possible to maintain monitoring without worsening the malfunction.

[0169] Furthermore, based on the result of the malfunction detection, the imaging direction is fixed so that the bed of the person requiring care is included in the image. This makes it possible to maintain monitoring within an appropriate monitoring range without worsening the malfunction any further.

[0170] In addition, a fixed direction to which the imaging direction is fixed when a malfunction is detected is stored. Then, when a malfunction is detected, the imaging direction is fixed to the fixed direction. This makes it possible to maintain monitoring in a more appropriate monitoring range without worsening the malfunction any further.

[0171] In addition, if a problem is detected, the speed of movement of the imaging unit when changing the imaging direction is limited, which prevents the problem from worsening any further and allows appropriate monitoring to be maintained.

[0172] Furthermore, when a malfunction is detected, the imaging range of the imaging unit other than the imaging unit is expanded to overlap with the imaging range of the imaging unit, so that monitoring can be continued even if the degree of the malfunction is relatively large.

[0173] In addition, when a malfunction is detected, the image captured by the imaging unit is started to be recorded or sound is made to be heard. Alternatively, when a malfunction is detected, the image captured or sound is made to be continuously recorded. This makes it possible to keep a record of important actions of the person requiring care even if a malfunction occurs.

[0174] In addition, a cover is provided to cover the drive mechanism of the drive unit that changes the imaging direction of the imaging unit. This makes the driving operation of the drive mechanism invisible to the care recipient, reducing the degree of psychological pressure on the care recipient who is conscious of being monitored. In addition, even if a malfunction of the drive mechanism becomes difficult to notice due to the cover, it is easy to grasp the malfunction of the drive unit.

[0175] The system also notifies the detection results to at least one of the terminals of the nursing care facility, the seller of the monitoring system for the care recipient, and the company that maintains the monitoring system for the care recipient. This allows the system to quickly arrange for parts replacement and other procedures to be carried out, allowing the malfunction to be resolved early.

[0176] Furthermore, the occurrence of a malfunction is notified by generating a notification signal from a notification device that notifies the occurrence of the malfunction, which makes it possible to easily notice the occurrence of the malfunction.

[0177] Furthermore, when a malfunction is detected, the program that drives the drive unit is restarted, which makes it possible to easily resolve malfunctions that can be resolved by restarting the program.

[0178] Furthermore, when a malfunction is detected, the range in which the imaging direction can be changed is limited by the driving unit, so that monitoring can be maintained within an appropriate monitoring range without worsening the malfunction any further.

[0179] In addition, when a malfunction of the motion sensor is detected, the imaging direction is changed based on the movement of the person in need of care in the image captured by the imaging unit, instead of the detection information of the motion sensor. This makes it possible to continue watching over the person even if the degree of malfunction of the motion sensor is relatively large.

[0180] In addition, when a malfunction is detected, the sensitivity of the behavior determination is changed. This allows appropriate monitoring to be continued according to the condition of the person requiring care, even if a malfunction occurs.

[0181] In addition, if a malfunction is detected, the notification criteria for the results of the judgment of the behavior of the care recipient are changed. This allows appropriate monitoring to be continued according to the condition of the care recipient, even if a malfunction occurs.

[0182] The above-described configurations of the system for monitoring a person requiring care, the method for monitoring a person requiring care, and the program for monitoring a person requiring care are the main configurations described in order to explain the features of the above-described embodiments. Therefore, the system for monitoring a person requiring care, the method for monitoring a person requiring care, and the program for monitoring a person requiring care are not limited to the above-described configurations, and can be modified in various ways within the scope of the claims. In addition, the configurations of a general system for monitoring a person requiring care, the method for monitoring a person requiring care, and the program for monitoring a person requiring care are not excluded.

[0183] For example, some steps may be omitted from the above-described flowchart, other steps may be added, some of the steps may be executed simultaneously, and one step may be divided into multiple steps and executed.

[0184] The means and methods for performing various processes in the above-mentioned system can also be realized by a dedicated hardware circuit. The means and methods for performing various processes in the above-mentioned system can also be realized by a programmed computer. The above-mentioned program may be provided by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded in the computer-readable recording medium is usually transferred to and stored in a storage unit such as a hard disk. The above-mentioned program may also be provided as a standalone application software, or may be incorporated as a function into the software of a device such as an information processing device. [Explanation of symbols]

[0185] 1 system, 10 Monitoring devices, 11 Control section, 111 Position detection unit, 112 Drive control unit, 113 Recognition Department, 114 Estimation Department, 115 Judgment Department, 116 Malfunction detection unit, 117 Specific Department; 118 Notification Department; 12 Communications Department, 13 storage section, 14 camera unit, 141 Camera, 142 Drive mechanism, 143 Human presence sensor, 30 fixed terminals, 40 mobile terminals, 50 communication networks, 51 access points, 700 captured images, 710 people rectangle, 720 articulation points, 80 Care recipients, 90 beds.

Claims

1. An imaging unit that captures an image of the person in need of care; A determination unit that determines a behavior of the care-requiring person based on at least an image of the imaging unit; A human presence sensor that detects the position of the person requiring care; A drive unit that changes an imaging direction of the imaging unit based on detection information of the human presence sensor; A detection unit that detects a malfunction of the drive unit or the human sensor; a first notification unit that notifies a plurality of terminals of a determination result of the determination unit; A second notification unit that notifies a plurality of terminals of a detection result of the detection unit; A care monitoring system for persons in need of care, which is installed in a care facility and used to monitor persons in need of care.

2. The care-requiring person monitoring system according to claim 1 , wherein the second notification unit notifies the user in a manner that distinguishes between the malfunction of the drive unit and the malfunction of the human presence sensor.

3. a storage unit that stores a correspondence relationship between the malfunction, the cause of the malfunction, and a countermeasure method for the malfunction; an identification unit that identifies a cause of the malfunction and the countermeasure based on a detection result by the detection unit by using the stored correspondence relationship; The care-requiring person monitoring system according to claim 1 , wherein the second notification unit notifies the identified cause of the malfunction and the countermeasure method together with the detection result.

4. The care-requiring person monitoring system according to claim 1 , wherein the driving unit limits driving control to the imaging unit when the malfunction is detected by the detection unit.

5. The care-requiring person monitoring system according to claim 1 , wherein the driving unit fixes the imaging direction to a direction in which a bed of the care-requiring person is included in the image based on a detection result of the detection unit.

6. a storage unit that stores a fixed direction in which the imaging direction is fixed when the defect is detected by the detection unit, The care-requiring person monitoring system according to claim 4 , wherein the drive unit fixes the imaging direction to the fixed direction when the malfunction is detected by the detection unit.

7. The care-requiring person monitoring system according to claim 4 , wherein the drive unit limits a speed of movement of the imaging unit when changing the imaging direction when the malfunction is detected by the detection unit.

8. The system for monitoring a person requiring care as described in claim 1, further comprising a control unit that, when the malfunction is detected by the detection unit, expands the imaging range of an imaging unit other than the imaging unit to overlap with the imaging range of the imaging unit.

9. The system for monitoring a person in need of care as described in claim 1, further comprising a control unit that starts recording or audio recording of the image captured by the imaging unit when the malfunction is detected by the detection unit, or that continues recording or audio recording at all times.

10. The system for monitoring a person requiring care according to claim 1 , further comprising a cover for covering a drive mechanism of the drive unit.

11. The care-requiring person monitoring system of claim 1, wherein the second notification unit notifies the detection result to a terminal of at least one of a care facility, a seller of the care-requiring person monitoring system, and a company that performs maintenance on the care-requiring person monitoring system.

12. The care-requiring person monitoring system according to claim 1 , wherein the second notification unit notifies the occurrence of the malfunction by generating an alarm signal from an alarm device that notifies the occurrence of the malfunction.

13. The care-requiring person monitoring system according to claim 4 , wherein the driving unit restarts a program that drives the driving unit when the malfunction is detected by the detection unit.

14. The care-requiring person monitoring system according to claim 4 , wherein the driving unit limits a range within which the imaging direction of the driving unit can be changed when the malfunction is detected by the detection unit.

15. The care-requiring person monitoring system of claim 1, wherein when the detection unit detects a malfunction of the human presence sensor, the driving unit changes the imaging direction based on the movement of the person in need of care in the image captured by the imaging unit, instead of the detection information of the human presence sensor.

16. The care-requiring person monitoring system according to claim 1 , wherein the determination unit changes a sensitivity of behavior determination when the malfunction is detected by the detection unit.

17. The care-requiring person monitoring system according to claim 1 , wherein the first notification unit changes a notification criterion for the determination result when the malfunction is detected by the detection unit.

18. A method executed by a care-requiring person monitoring system installed in a care facility and used to monitor a care-requiring person, comprising: A step (a) of acquiring an image of a person requiring care captured by an imaging unit; (b) determining a behavior of the care-requiring person based on at least an image of the imaging unit; (c) changing an imaging direction of the imaging unit by a driving unit based on detection information of a human presence sensor that detects a position of the care-requiring person; (d) detecting a malfunction of the drive unit or the human presence sensor; a step (e) of notifying a plurality of terminals of the determination result in the step (b); a step (f) of notifying a plurality of terminals of the detection result in the step (d); A method for monitoring a person requiring care.

19. The method for monitoring a person requiring care according to claim 18, wherein in the step (f), notification is given in a manner that distinguishes between the malfunction of the drive unit and the malfunction of the human presence sensor.

20. a step (g) of storing a correspondence relationship between the malfunction and a cause and a countermeasure for the malfunction, and identifying the cause and the countermeasure for the malfunction based on the detection result in the step (d) using the stored correspondence relationship; The method for monitoring a person requiring care according to claim 18, wherein in said step (f), the identified cause of the malfunction and the countermeasure method are notified together with the detection result.

21. The method for monitoring a person requiring care according to claim 18, further comprising the step (h) of limiting drive control to the imaging unit when the malfunction is detected in the step (d).

22. The method for monitoring a person requiring care according to claim 18, further comprising causing the drive unit to fix the imaging direction to a direction in which a bed of the person requiring care is included in the image, based on the detection result in step (d).

23. A method for monitoring a person requiring care as described in claim 21, wherein a fixed direction in which the imaging direction is fixed when the malfunction is detected in step (d) is stored, and the driving unit is caused to fix the imaging direction to the fixed direction when the malfunction is detected in step (d).

24. The method for monitoring a person requiring care according to claim 21 , further comprising causing the drive unit to limit the speed of movement of the imaging unit when changing the imaging direction when the malfunction is detected in step (d).

25. The method for monitoring a person requiring care according to claim 18, further comprising the steps of: enlarging an imaging range of an imaging unit other than the imaging unit so as to overlap the imaging range of the imaging unit when the malfunction is detected in step (d).

26. A method for monitoring a person requiring care as described in claim 18, wherein, when the malfunction is detected in step (d), recording or audio recording of the image captured by the imaging unit is started, or the recording or audio recording is continued continuously.

27. The method for monitoring a person requiring care according to claim 18 , wherein the monitoring system has a cover for covering a drive mechanism of the drive unit.

28. The method for monitoring a person requiring care as described in claim 18, wherein in step (d), the detection result is notified to a terminal of at least one of a care facility, a seller of the care monitoring system, and a company that performs maintenance on the care monitoring system.

29. 20. The method for monitoring a person requiring care according to claim 18, wherein in the step (d), the occurrence of the malfunction is notified by generating an alarm signal from an alarm device that notifies the occurrence of the malfunction.

30. The method for monitoring a person requiring care according to claim 21, further comprising a step (i) of restarting a program that drives the drive unit when the malfunction is detected in the step (d).

31. The method for monitoring a person requiring care according to claim 21 , further comprising the step of: causing the driving unit to limit a range within which the imaging direction of the driving unit can be changed when the malfunction is detected in step (d).

32. A method for monitoring a person requiring care as described in claim 18, wherein when a malfunction of the human presence sensor is detected in step (d), the driving unit changes the imaging direction based on the movement of the person requiring care in the image captured by the imaging unit instead of the detection information of the human presence sensor.

33. 20. The method for monitoring a person requiring care according to claim 18, further comprising the step (j) of changing a sensitivity of the behavior determination in the step (b) when the malfunction is detected in the step (d).

34. The method for monitoring a person requiring care according to claim 18, further comprising changing a criterion for notifying the result of the determination made in step (b) when the malfunction is detected in step (d).

35. A program for monitoring a person requiring care, for causing a computer to execute the method for monitoring a person requiring care according to any one of claims 18 to 34.

Images