Control device, control method, control program, and information processing system

Abstract

To provide a control device, control method, control program and information processing system which can properly perform state detection of a care-receiver with an image even when a bed is moved due to tilting and the like.SOLUTION: A control device comprises: an image acquisition unit 111; an image analysis unit 112; and a control unit 114. The image acquisition unit 111 acquires an image captured by an imaging unit 103 installed in a bed 90. The image analysis unit 112 analyzes the image. The control unit 114 controls the imaging unit 103 or the image analysis unit 112 on the basis of the movement or state of the bed 90.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a control device, a control method, a control program, and an information processing system. [Background technology]

[0002] In nursing care facilities such as nursing homes for the elderly and hospitals, residents or hospitalized care recipients (hereinafter referred to as "care recipients") may fall while walking, fall out of bed and injure themselves, or abnormalities may be detected in their biometric information. Therefore, to enable nurses, caregivers, etc. (hereinafter referred to as "caregivers") to rush to the scene immediately when a care recipient falls into such a state, monitoring systems that constantly monitor the condition of care recipients are being developed.

[0003] The following Patent Document 1 describes, as a monitoring system, an image recognition unit that recognizes body parts of a patient as objects or parts from an image captured by a monitor camera attached to an electric bed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-134278 Summary of the Invention [Problem to be solved by the invention]

[0005] The technology of Patent Document 1 has the advantage that by installing a camera on an electric bed, the camera does not need to be aligned with the electric bed or the surrounding environment, improving convenience for caregivers. However, the technology of Patent Document 1 may cause problems such as frequent false detections based on images. After careful consideration, the inventors found that problems can arise due to the operation of the electric bed. Specifically, electric beds have functions for adjusting the bed height, back lift angle, knee lift angle, foot lift angle, and tilt angle. However, by installing a camera on the electric bed, the orientation of the camera changes in response to movements such as tilting of the electric bed, and it was found that this can cause problems in detecting the condition of the care recipient based on images.

[0006] The present invention has been made to solve these problems, and aims to provide a control device, a control method, a control program, and an information processing system that can appropriately detect the condition of a care recipient using images even if the bed moves due to tilting or other reasons. [Means for solving the problem]

[0007] The above-mentioned problems of the present invention are solved by the following means.

[0008] (1) An image acquisition unit that acquires an image captured by an imaging unit installed in a bed, an image analysis unit that analyzes the image, and a control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, The bed is an electric bed, and the control unit determines the operation or state of the bed based on a signal from the electric bed. A control device.

[0009] (2) An image acquisition unit that acquires an image captured by an imaging unit installed on the bed, an image analysis unit that analyzes the image, and a control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed; The image analysis unit detects an object included in the image captured by the imaging unit, compares the position of the top end of a bed object corresponding to the bed in the image with the position of the top end of a human object corresponding to the person in the image, and determines that the person has not fallen if the position of the top end of the human object is higher than the position of the top end of the bed object, but determines that the person has fallen if the position of the top end of the human object is lower than the position of the top end of the bed object. , control device.

[0010] (3) an image acquisition unit that acquires an image captured by an imaging unit installed on the bed, and an image analysis unit that analyzes the image; The bed includes a sensor that detects the operation of the bed, including the operation of tilting the mattress of the bed and the operation of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted with respect to the floor surface, and a control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, and the control unit controls the orientation or position of the imaging unit based on the operation or state of the bed., control device.

[0011] (4) an image acquisition unit that acquires an image captured by an imaging unit installed on the bed, and an image analysis unit that analyzes the image; The bed includes a sensor that detects the bed's motion, including the motion of tilting the mattress of the bed and the motion of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted relative to the floor surface; a control unit that controls the imaging unit or the image analysis unit based on the bed's motion or state; and a receiving unit that receives a detection result from the sensor that detects the bed's motion or state, and the control unit determines the bed's motion or state based on the detection result. , control device.

[0012] (5) an image acquisition unit that acquires an image captured by an imaging unit installed on the bed, and an image analysis unit that analyzes the image; a sensor that detects the operation of the bed, including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or the state of the bed, including a state in which the mattress is tilted with respect to a floor surface; A control device comprising: a control unit that controls the imaging unit based on the operation or state of the bed.

[0013] (6) The control unit controls the image analysis unit to change a state determination threshold used to determine the state of the care recipient or a posture determination standard used to determine the posture of the care recipient based on the movement or state of the bed. The posture determination criteria include a normal state criterion that is set when the bed is in a normal state, and a reclined state criterion that is set when the bed is in a reclined state. The normal state criterion is a criterion for determining that the care recipient is not in a fallen state when the care recipient's posture is determined to be standing, and for determining that the care recipient is in a fallen state when the care recipient's posture is determined to be sitting. The reclined state criterion is a criterion for determining that the care recipient is in a fallen state when the care recipient's posture is determined to be standing. The control device according to any one of (1) to (4) above.

[0014] (7) The bed includes an image acquisition unit that acquires an image taken by an imaging unit installed in the bed, an image analysis unit that analyzes the image, and a control unit that controls the imaging unit or the image analysis unit based on the movement or state of the bed, and the control unit controls the orientation or position of the imaging unit based on the movement or state of the bed, The control unit converts the position of each pixel in the image acquired by the image acquisition unit to the position of the image that would be obtained if the bed were in a normal state based on the operation or state of the bed, and controls the image analysis unit to correct the image acquired by the image acquisition unit by performing calculations to interpolate between the pixels.

[0015] (8) The above For determining status The image analysis unit detects an object included in an image captured by the imaging unit, and calculates a position of an upper end of a human object corresponding to a person in the image. For determining status The control device according to (6) above, which determines the state of the person based on the result of comparison with a threshold value.

[0016] (9) The image analysis unit, based on the movement or state of the bed, For determining status The control device according to (8) above, which changes the threshold value.

[0017] (10) The above For posture determinationThe image analysis unit detects a skeleton and joint points of a human object corresponding to a person included in an image captured by the imaging unit, and compares the detected skeleton and joint points with the For posture determination The control device according to (6) above, which determines the posture of the person and the state of the person corresponding to the posture based on a standard.

[0018] (11) The image analysis unit performs the following based on the movement or state of the bed: For posture determination The control device according to (10) above, which changes the criteria.

[0019] (12) A control method comprising: a control step of controlling an imaging unit based on the operation or state of a bed on which the imaging unit is installed; a photographing step of photographing an object to be photographed, including the bed, by the imaging unit; an acquisition step of acquiring, by an image acquisition unit, an image photographed by the imaging unit; and an analysis step of analyzing the image.

[0020] (13) A control method comprising: an imaging step of imaging an object including the bed using an imaging unit installed on the bed; an acquisition step of acquiring the image captured by the imaging unit using an image acquisition unit; a step of receiving a detection result from a sensor that detects the movement or state of the bed; and a control step of determining the movement or state of the bed based on the detection result and controlling an image analysis unit that analyzes the image based on the movement or state of the bed.

[0021] (14) a detecting step of detecting a bed movement including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or a state of the bed including a reclining state in which the mattress is tilted relative to a floor surface; a control step of controlling the imaging unit based on the movement or state of the bed in which the imaging unit is installed; a photographing step of photographing an object including the bed by the imaging unit; an acquisition step of acquiring an image photographed by the imaging unit by an image acquisition unit; and an analysis step of analyzing the image. A control method comprising the steps of:

[0022] (15) an imaging step of imaging an object including the bed by an imaging unit installed on the bed; and an acquisition step of acquiring the image captured by the imaging unit by an image acquisition unit; A detection step of detecting, by a sensor, the operation of the bed, including the operation of tilting the mattress of the bed and the operation of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted relative to a floor surface; a step of receiving a detection result from the sensor that detects the operation or state of the bed; and a control step of determining the operation or state of the bed based on the detection result and controlling an image analysis unit that analyzes the image based on the operation or state of the bed. A control method comprising the steps of:

[0023] (16) A control program for causing a computer to execute the control method according to any one of (12) to (15) above.

[0024] (17) An information processing system comprising the control device according to any one of (1) to (11) above. [Effects of the Invention]

[0025] According to the present invention, the imaging unit or image analysis unit is controlled based on the movement or state of the bed, so that even if the bed moves due to tilting, etc., the state of the person being cared for can be appropriately detected using images. [Brief explanation of the drawings]

[0026] [Figure 1]1 is a diagram showing the overall configuration of a monitoring system according to a first embodiment. [Figure 2] 2 is a schematic diagram illustrating a bed for a care recipient on which the detection device shown in FIG. 1 is installed. [Figure 3] FIG. 2 is a block diagram showing a hardware configuration of the detection device shown in FIG. [Figure 4] 4 is a block diagram illustrating the main functions of the arithmetic and control unit shown in FIG. 3. [Figure 5] FIG. 2 is a block diagram showing the hardware configuration of the server shown in FIG. [Figure 6] 4 is a flowchart illustrating a processing procedure of a control method for the detection device according to the first embodiment. [Figure 7A] FIG. 2 is a schematic side view of the bed in a normal state and the detection device. [Figure 7B] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a non-falling state. [Figure 7C] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a fallen state. [Figure 8A] FIG. 2 is a schematic side view of the bed in a reclining state and the detection device. [Figure 8B] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a non-falling state. [Figure 8C] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a fallen state. [Figure 9A] FIG. 2 is a schematic side view of the bed in a reclining state and the detection device. [Figure 9B] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a non-falling state. [Figure 9C] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a fallen state. [Figure 10] 10 is a flowchart illustrating a processing procedure of a control method for a detection device according to a second embodiment. [Figure 11] FIG. 10 is a schematic diagram illustrating an image after distortion correction including a bed and a care recipient in a non-falling state. [Figure 12] 10 is a schematic diagram illustrating an example of an image after distortion correction, including a bed and a care recipient in a fallen state; FIG. [Figure 13] 10 is a flowchart illustrating a processing procedure of a control method for a detection device according to a third embodiment. [Figure 14] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a non-falling state. [Figure 15] FIG. 10 is a schematic diagram illustrating an example of an image including a bed and a care recipient in a fallen state. [Figure 16] 10 is a schematic diagram illustrating skeletal detection when the bed is in a normal state and the care recipient is not in a fallen state. FIG. [Figure 17] 10 is a schematic diagram illustrating skeletal detection when the bed is in a normal state and the care recipient has fallen. FIG. [Figure 18] 10 is a schematic diagram illustrating skeletal structure detection when the bed is in a reclining position and the care recipient is not in a fallen position. FIG. [Figure 19] 10 is a schematic diagram illustrating skeletal structure detection when the bed is in a reclining position and the care recipient has fallen. FIG. [Figure 20] FIG. 11 is a schematic side view of a bed in a normal state and a detection device in the fifth embodiment. [Figure 21] 10 is a schematic diagram of a bed and a detection device viewed from the side in a state where the mattress on the foot side is inclined. FIG. [Figure 22] FIG. 13 is a schematic side view of a bed in a normal state and a detection device in the sixth embodiment. [Figure 23] FIG. 2 is a schematic side view of the bed in a reclining state and the detection device. DETAILED DESCRIPTION OF THE INVENTION

[0027] Hereinafter, a control device, a control method, a control program, and an information processing system according to an embodiment of the present invention will be described with reference to the drawings. In the drawings, identical elements are designated by the same reference numerals, and duplicated explanations will be omitted. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation, and may differ from the actual proportions.

[0028] (First embodiment) FIG. 1 is a diagram showing the overall configuration of a monitoring system 1 according to a first embodiment. In this embodiment, the monitoring system 1 constitutes an information processing system and is used in a nursing care facility or the like. The monitoring system 1 includes a detection device 10, a server 20, multiple mobile terminals 30, and a wireless AP (access point) 31. The detection device 10, the server 20, and the mobile terminals 30 are connected to each other via a local area network (LAN) and the wireless AP 31 so that they can communicate with each other. The mobile terminal 30 is carried by a caregiver 70 who is caring for or caring for a care recipient 80. The detection device 10 constitutes a control device either alone or together with an arithmetic and control unit 201 (see FIG. 5) of the server 20. The following description will mainly focus on the case where the detection device 10 constitutes a control device alone. The care recipient 80 constitutes a subject to be photographed. In the following description, the subject to be photographed is the care recipient 80, but the subject to be photographed also includes clinical trial participants and the like.

[0029] (Detection device 10) Fig. 2 is a schematic diagram illustrating a bed 90 for a care recipient 80, on which the detection device 10 shown in Fig. 1 is installed. Fig. 3 is a block diagram illustrating a hardware configuration of the detection device 10 shown in Fig. 1.

[0030] As shown in Fig. 2, bed 90 includes a frame unit 91, a headboard 92, a footboard 93, a mattress 94, and a sensor unit 95. In this embodiment, bed 90 is, for example, an electric bed, and is equipped with a first drive device (not shown) that partially tilts mattress 94 corresponding to the back, knees, and feet of care recipient 80 when he or she lies supine on bed 90. For example, by tilting the portion of mattress 94 corresponding to the back of care recipient 80, a tilted (reclining) state can be created. The operation of the first drive device is controlled by a control signal from a control device (not shown) of the electric bed.

[0031] Furthermore, the frame section 91 has an upper frame 911 and a lower frame 912, and the height (floor height) of the mattress 94 relative to the floor surface can be adjusted by moving the lower frame 912 in a direction approaching or away from the upper frame 911 (see Figures 2, 7A, etc.). Also, by using the first drive device to independently move the head side and foot side of the lower frame 912 in a direction approaching or away from the upper frame 911, the upper frame 911 can also be tilted relative to the floor surface.

[0032] The sensor unit 95 has a sensor that detects the movement or state of the bed. The sensor can be a known acceleration sensor, tilt sensor, angle sensor, etc. The detection result by the sensor is sent to the arithmetic and control unit 101.

[0033] As shown in FIG. 3, the detection device 10 includes an arithmetic and control unit 101, a communication unit 102, and a camera 103, which are interconnected by a bus. The detection device 10 is installed, for example, on the upper end of a mattress 94 of a bed 90, and captures an image of the monitored area, which includes the bed 90 and the care recipient 80 as the subject of the image capture. The detection device 10 also includes a rotation drive unit that rotates a housing that houses the arithmetic and control unit 101, the communication unit 102, and the camera 103 around the Z-axis and the Y-axis as rotation axes, and can freely change the orientation of the housing in response to a command from the arithmetic and control unit 101 (see FIG. 7A for coordinate axis indication). This allows the arithmetic and control unit 101 to control the direction in which the camera 103 captures images. The detection device 10 can also include a mechanism that moves the housing in the X-axis direction or the Z-axis direction.

[0034] The arithmetic and control unit 101 includes a CPU, RAM, ROM, etc. The arithmetic and control unit 101 may also include an HDD or SSD (Solid State Drive). The arithmetic and control unit 101 controls each unit of the detection device 10 and performs arithmetic processing according to a control program. The main functions of the arithmetic and control unit 101 will be described later. The RAM stores state determination thresholds (first threshold, second threshold) and posture determination standards (standard for the normal state, standard for the reclined state), which will be described later.

[0035] The communication unit 102 is an interface circuit (for example, a LAN card, a wireless communication circuit, etc.) for communicating with, for example, the server 20 via a LAN.

[0036] The camera 103 captures images within the monitoring area and outputs the captured images (image data). The camera 103 constitutes an imaging unit. The captured images include still images and video. Video may include multiple still images captured intermittently in time series. The camera 103 is a near-infrared camera, but a visible light camera may be used instead, or a combination of these may be used. The camera 103 is fixed in the housing of the detection device 10 so that the optical axis of the lens faces the foot side of the bed 90, and the camera 103 also rotates and moves in accordance with the rotation and movement of the housing.

[0037] Fig. 4 is a block diagram illustrating the main functions of the arithmetic and control unit 101 shown in Fig. 1. The arithmetic and control unit 101 functions as an image acquisition unit 111, an image analysis unit 112, a reception unit 113, and a control unit 114.

[0038] The image acquisition unit 111 acquires images captured by the camera 103 installed on the bed 90. The image analysis unit 112 analyzes the images acquired by the image acquisition unit 111. In this embodiment, the image analysis unit 112 determines whether the care recipient 80 has fallen or is not falling, based on the images. The reception unit 113 receives the detection results from the sensors of the sensor unit 95 and a control signal from the control device of the bed 90.

[0039] The control unit 114 determines the movement or state of the bed based on the detection results of the sensors in the sensor unit 95 and / or a control signal from the control device of the bed 90, and controls the camera 103 based on the determined movement or state. The movement of the bed 90 includes, for example, an operation of tilting a part of the mattress 94 relative to the upper frame 911, an operation of tilting the upper frame 911 relative to the floor surface, etc. The state of the bed 90 also includes, for example, a state in which the mattress 94 is tilted relative to the floor surface, a state in which it is not tilted, etc.

[0040] (Server 20) 5 is a block diagram showing the hardware configuration of the server 20. The server 20 has an arithmetic and control unit 201, a communication unit 202, and a storage unit 203, which are interconnected by a bus.

[0041] The arithmetic and control unit 201 includes a CPU, RAM, ROM, etc. The arithmetic and control unit 201 controls each unit of the server 20 and performs arithmetic processing in accordance with a program. The arithmetic and control unit 201 also constitutes a control device together with the detection device 10, and can be configured to cooperate with the image analysis unit 112 to execute image analysis processing.

[0042] The communication unit 202 is configured with an interface circuit for communicating with, for example, the mobile terminal 30 via a LAN. The communication unit 202 receives the determination result of the condition of the care recipient 80 and / or the captured image from the detection device 10.

[0043] The storage unit 203 is configured by an HDD or SSD, and stores various programs and various data.

[0044] (Control method) FIG. 6 is a flowchart illustrating the processing steps of the control method of the control device in the first embodiment. The processing of the flowchart in FIG. 6 is realized by the CPU of the arithmetic and control unit 101 executing a control program. FIG. 7A is a schematic diagram of a bed 90 with a mattress 94 horizontal to the floor, and a detection device 10, viewed from the side. FIG. 7B is a schematic diagram illustrating an image including the bed 90 and a care recipient 80 in a non-falling state. FIG. 7C is a schematic diagram illustrating an image including the bed 90 and a care recipient 80 in a fallen state. FIG. 8A is a schematic diagram of a bed 90 in a reclining state, and a detection device 10, viewed from the side. FIG. 8B is a schematic diagram illustrating an image including the bed 90 and a care recipient 80 in a non-falling state. FIG. 8C is a schematic diagram illustrating an image including the bed 90 and a care recipient 80 in a fallen state. Also, Figure 9A is a schematic diagram of the bed 90 in a reclining state and the detection device 10 viewed from the side, Figure 9B is a schematic diagram illustrating an image including the bed 90 and the cared-for person 80 in a non-falling state, and Figure 9C is a schematic diagram illustrating an image including the bed 90 and the cared-for person 80 in a falling state.

[0045] 7A, 8A, and 9A, the direction from left to right on the paper (the direction from the head side to the foot side of the bed 90) is defined as the X-axis direction, the direction from the front to the back of the paper (the width direction of the bed 90) is defined as the Y-axis direction, and the direction from bottom to top of the paper (the height direction of the bed 90) is defined as the Z-axis direction. Also, to simplify the explanation, objects corresponding to the footboard 93 are not shown in Figures 7B, 7C, 8B, 8C, 9B, and 9C.

[0046] First, the direction or position of the camera 103 is controlled to set a monitoring area (step S101). The control unit 114 controls the direction or position of the camera 103 based on the state of the bed 90 to set the monitoring area. For example, as shown in FIG. 7A, when the mattress 94 of the bed 90 is horizontal with respect to the floor surface, i.e., not tilted (hereinafter referred to as the "normal state"), the control unit 114 sets the parameters of the camera 103 to initial values. In this embodiment, the parameters include, for example, pan P, tilt T, and magnification M (or zoom Z). The control unit 114 sets the parameters (P, T, M) to initial values ​​(P0, T0, M0). The state of the camera 103 in which the parameters are set to their initial values ​​is referred to as the initial state. The pan P0 and tilt T0 in the initial state are set so that the optical axis of the camera 103 is directed toward the foot side of the mattress 94. Furthermore, the magnification M0 in the initial state is set so that at least the foot side portion of the mattress 94 is included in the monitoring area of ​​the camera 103. By setting the parameters in this way, the monitoring area of ​​the camera 103 is set.

[0047] Alternatively, instead of setting the pan P, tilt T, and magnification M based on the state of bed 90, the position of camera 103 may be controlled. For example, when bed 90 is in a normal state, moving the position of camera 103 in the X-axis direction can achieve an effect equivalent to changing the magnification M.

[0048] Next, camera 103 starts capturing images of the area being monitored (step S102). Camera 103 starts (continues) capturing images of the area being monitored that has been set based on the parameters. As an example, assume that care recipient 80, who has been lying in bed 90, gets up from bed 90, gets off the bed, and attempts to walk. Camera 103 captures images of care recipient 80 and bed 90 within the area being monitored.

[0049] Next, the captured image is acquired (step S103). The image acquisition unit 111 acquires the image captured by the camera 103 and transmits it to the image analysis unit 112.

[0050] Next, objects are detected in the image (step S104). The image analysis unit 112 detects objects included in the image acquired by the image acquisition unit 111. For example, as shown in FIG. 7B, the image analysis unit 112 detects an object 940 corresponding to a bed 90 (hereinafter referred to as a "bed object") and an object 800 corresponding to a care recipient 80 who is trying to walk (hereinafter referred to as a "person object") 800. Note that detection of objects in an image can be achieved using a known algorithm, and therefore a detailed description of the object detection method will be omitted.

[0051] Next, the state of the care recipient 80 is determined (step S105). The image analysis unit 112 determines the state of the care recipient 80 based on the position of the upper end of the detected person object 800. More specifically, the position of the upper end of the bed object 940 is set as a first threshold TH1 as the state determination threshold, and the position B of the upper end of the person object 800 is compared with the state determination threshold. For example, as shown in FIG. 7B, the image analysis unit 112 determines that the care recipient 80 is in a non-falling state when the position B of the upper end of the person object 800 is greater than the state determination threshold. On the other hand, for example, as shown in FIG. 7C, the image analysis unit 112 determines that the care recipient 80 is in a falling state when the position B of the upper end of the person object 800 is smaller than the state determination threshold. Note that the first threshold TH1 may be a value pre-stored in the RAM of the arithmetic and control unit 101.

[0052] Next, the determination result is output (step S106). The arithmetic and control unit 101 outputs the determination result obtained by the image analysis unit 112. The determination result is transmitted to the mobile terminal 30 by the communication unit 202.

[0053] In this way, when the bed 90 is in a normal state, i.e., when the mattress 94 is not tilted relative to the floor surface, the camera 103 takes an image of the monitoring area with its parameters set to the initial values, and the state of the person being cared for 80 (fall / not fall) can be determined based on the acquired image.

[0054] However, when the bed 90 is in a reclining state as shown in Fig. 8A, the distance between the camera 103 and the mattress 94 in the monitoring area becomes closer as shown in Fig. 8B and Fig. 8C, and the shapes of the person object 800 and the bed object 940 change (for example, become larger) from the shapes when the bed 90 is in a normal state. As a result, the position B of the upper end of the person object 800 is not compared correctly with the state determination threshold, and there is a possibility that the state of the care recipient 80 cannot be determined appropriately.

[0055] Therefore, as shown in Figure 9A, the pan P and tilt T of camera 103 are changed (controlled) to P1 and T1, respectively, so that the angle formed between the optical axis of camera 103 and the head-side portion of mattress 94 corresponds to the inclination angle a1 of mattress 94, thereby making the optical axis of camera 103 approximately parallel to the foot-side portion of mattress 94. Also, as shown in Figures 9B and 9C, the magnification of camera 103 is set to an appropriate value M1 so that the shape of bed 90 in the captured image is equivalent to the shape of bed 90 in its normal state. By setting the parameters (P, T, M) in this way and capturing an image within the monitoring area, the condition of care recipient 80 can be appropriately determined based on the captured image.

[0056] Also, instead of controlling the pan P, tilt T, and magnification M, the position of the camera 103 may be controlled to move in the X-axis direction and / or the Z-axis direction.

[0057] Furthermore, although the case where the orientation or position of camera 103 is controlled based on the state of bed 90 (for example, normal state or reclining state) has been described, the orientation or position of camera 103 may also be controlled based on the movement of bed 90 (for example, movement of mattress 94). The movement of mattress 94 can be detected, for example, by an acceleration sensor of sensor unit 95 installed on mattress 94.

[0058] The detection device 10 of this embodiment described above controls the camera 103 based on the movement or state of the bed 90, so that even if the bed 90, i.e., the camera 103, moves due to the bed 90 tilting, for example, the state of the person being cared for can be appropriately detected from the image.

[0059] (Second embodiment) In the first embodiment, the parameters of the camera 103 are controlled based on the movement or state of the bed 90 to capture an image of a set monitoring area, and the state of the care recipient 80 is determined based on the captured image. In the second embodiment, the distortion (shape) of an image captured by the camera 103 is corrected based on the movement or state of the bed 90, and the state of the care recipient 80 is determined based on the corrected image. Note that the correction of "distortion" in this specification refers to correcting (converting) an image when the bed 90 is in a reclined position into an image equivalent to an image when the bed 90 is in a normal position, and is unrelated to distortion in the optical system (lens) of the camera 103. In the following, to avoid duplication, detailed descriptions of the same configuration as in the first embodiment will be omitted.

[0060] Fig. 10 is a flowchart illustrating the processing procedure of the control method of the detection device 10 in the second embodiment. The processing of the flowchart in Fig. 10 is realized by the CPU of the arithmetic and control unit 101 executing a control program. Fig. 11 is a schematic diagram illustrating an image after distortion correction, including a bed and a care recipient in a non-falling state. Fig. 12 is a schematic diagram illustrating an image after distortion correction, including a bed and a care recipient in a fallen state.

[0061] 10, first, camera 103 starts capturing images within a monitoring area (step S201). Control unit 114 sets initial values ​​(P0, T0, M0) to parameters (P, T, M) of camera 103. This puts camera 103 into an initial state (see FIG. 7A).

[0062] Next, the captured image is acquired (step S202). The image acquisition unit 111 acquires the image captured by the camera 103. As described above, when the state of the bed 90 changes from the normal state to the reclining state, the distance between the camera 103 and the mattress 94 in the monitoring area decreases, and the shapes of the person object 800 and the bed object 940 change from the shapes in the normal state (see FIGS. 8B and 8C).

[0063] Next, distortion of the acquired image is corrected (step S203). Image analysis unit 112 corrects distortion of the image acquired by image acquisition unit 111 based on the state of bed 90. Image analysis unit 112 performs predetermined arithmetic processing for correcting distortion on the image acquired by image acquisition unit 111, thereby generating an image equivalent to the image obtained when bed 90 is in a normal state. For example, as shown in FIGS. 11 and 12, image analysis unit 112 uses existing morphing technology or the like to convert the position of each pixel in the image acquired by image acquisition unit 111 to the position of the image obtained when bed 90 is in a normal state, and performs arithmetic processing to interpolate between pixels.

[0064] Next, an object is detected in the image (step S204). The image analysis unit 112 detects an object included in the image whose distortion has been corrected.

[0065] Next, the state of the care recipient 80 is determined (step S205). Since the image with corrected distortion is equivalent to an image captured under normal conditions, the same method as the method for determining the state of the care recipient 80 in the first embodiment can be applied, and therefore detailed description thereof will be omitted.

[0066] Next, the determination result is output (step S206). The arithmetic and control unit 101 outputs the determination result obtained by the image analysis unit 112. The determination result is transmitted to the mobile terminal 30 by the communication unit 202.

[0067] 10, the camera 103 captures an image of the monitored area and acquires the image. Then, based on the movement or state of the bed 90, distortion of the image is corrected, and based on the distortion-corrected image, the state of the care recipient 80 is determined and the determination result is output.

[0068] The detection device 10 of this embodiment described above determines the condition of the care recipient 80 based on an image in which distortion has been corrected, and therefore can appropriately detect the condition of the care recipient from the image even if the bed 90 moves, for example, by tilting.

[0069] (Third embodiment) In the third embodiment, a case will be described in which the image analysis method is changed based on the movement or state of the bed 90. In the following, to avoid duplication of explanation, detailed explanation of the same configuration as in the first embodiment will be omitted.

[0070] Fig. 13 is a flowchart illustrating the processing procedure of the control method of the detection device 10 in the third embodiment. The processing of the flowchart shown in Fig. 13 is realized by the CPU of the arithmetic and control unit 101 executing a control program. Fig. 14 is a schematic diagram illustrating an example of changing the state determination threshold when the bed 90 is in a reclining state and the care recipient 80 is not in a fallen state, and Fig. 15 is a schematic diagram illustrating an example of changing the state determination threshold when the bed 90 is in a reclining state and the care recipient 80 is in a fallen state.

[0071] Hereinafter, a case will be considered in which bed 90, which was initially in a normal state, changes to a reclining state. First, as shown in Fig. 13, camera 103 starts capturing images of the monitoring area (step S301). Initially, bed 90 is in a normal state, and the parameters of camera 103 are set to initial values ​​(P0, T0, M0) (see Fig. 7A).

[0072] Next, the captured image is acquired (step S302). The image acquisition unit 111 acquires the image captured by the camera 103. As described above, when the state of the bed 90 changes from the normal state to the reclining state, the distance between the camera 103 and the mattress 94 in the monitoring area decreases, and the shapes of the person object 800 and the bed object 940 change from the shapes in the normal state (see FIGS. 14 and 15).

[0073] Next, objects are detected in the image (step S303). The image analysis unit 112 detects objects included in the image acquired by the image acquisition unit 111. For example, as shown in Figs. 14 and 15, the image analysis unit 112 detects a bed object 940 corresponding to the bed 90 and a person object 800 corresponding to the care recipient 80.

[0074] Next, the image analysis method is changed (step S304). When the state of the bed 90 changes from the normal state to the reclining state, the image analysis unit 112 changes the state determination threshold from the first threshold TH1 to the second threshold TH2. The first threshold TH1 is the upper end of the bed object 940 when the bed 90 is in the normal state, and the second threshold TH2 is the upper end of the bed object 940 when the bed 90 is in the reclining state.

[0075] Next, the state of the care recipient 80 is determined (step S305). The image analysis unit 112 compares the position B of the top end of the person object 800 with the state determination threshold. For example, as shown in FIG. 14, if the position B of the top end of the person object 800 is larger than the state determination threshold, the image analysis unit 112 determines that the care recipient 80 is in a non-falling state. Note that because the person object 800 exceeds the range of the monitoring area, the position B of the top end is set to the top end of the image of the person object 800 for convenience. On the other hand, for example, as shown in FIG. 15, if the position B of the top end of the person object 800 is smaller than the state determination threshold, the image analysis unit 112 determines that the care recipient 80 is in a falling state. Note that the second threshold TH2 may use a value pre-stored in the RAM of the arithmetic and control unit 101.

[0076] Next, the determination result is output (step S306). The arithmetic and control unit 101 outputs the determination result obtained by the image analysis unit 112. The determination result is transmitted to the mobile terminal 30 by the communication unit 202.

[0077] 13, the camera 103 captures an image of the monitored area and acquires the image. Then, based on the movement or state of the bed 90, the condition determination threshold value used to determine the condition of the care recipient 80 as an image analysis method is changed. The changed condition determination threshold value is then used to determine the condition of the care recipient 80, and the determination result is output.

[0078] The detection device 10 of this embodiment described above determines the state of the person being cared for 80 using the changed state determination threshold, so that even if the bed 90 moves, for example, by tilting, the state of the person being cared for 80 can be appropriately determined from the image.

[0079] (Fourth embodiment) In the third embodiment, a case was described in which the state determination threshold used to determine the state of the care recipient 80 is changed as an image analysis method based on the movement or state of the bed 90, but in the fourth embodiment, a case will be described in which the posture determination standard used to determine the posture of the care recipient 80 is changed as an image analysis method based on the movement or state of the bed 90. In the following, to avoid duplication, detailed descriptions of the same configuration as in the first embodiment will be omitted.

[0080] Fig. 16 is a schematic diagram illustrating skeletal detection when the bed 90 is in a normal position and the care recipient 80 has not fallen, Fig. 17 is a schematic diagram illustrating skeletal detection when the bed 90 is in a normal position and the care recipient 80 has fallen, Fig. 18 is a schematic diagram illustrating skeletal detection when the bed 90 is in a reclining position and the care recipient 80 has not fallen, and Fig. 19 is a schematic diagram illustrating skeletal detection when the bed 90 is in a reclining position and the care recipient 80 has fallen.

[0081] The posture of the care recipient 80 can be determined using, for example, a human posture estimation technology that uses machine learning. Human posture estimation technology, such as Open Pose (see, for example, https: / / arxiv.org / abs / 1611.08050) and Deep Pose (see, for example, https: / / arxiv.org / abs / 1312.4659), is a technology that recognizes posture by estimating the positions of a person's skeleton and joints from two-dimensional video data. These technologies recognize the skeleton that makes up a person, such as the head, neck, shoulders, arms, waist, and legs, and further obtain coordinate values ​​as position information for the joints that are the joints and ends of these.

[0082] For example, in FIG. 16, thick lines indicate a skeleton 801, and the parts connecting the skeletons and the ends of the skeletons (circled parts in the figure) indicate joint points 802. The head 810 also includes joint points. Since the skeletons and joint points are displayed in the same way in FIGS. 17 to 19, descriptions and reference numerals will be omitted. In this embodiment, the positions of the joint points roughly coincide with the anatomical positions of the joints of the human body. The image analysis unit 112 acquires coordinate data of each joint point of an object recognized as a person, and analyzes the posture, etc. of the recognized person from the coordinate data.

[0083] The processing procedure of the control method for the detection device 10 in this embodiment is the same as the processing procedure of the third embodiment (flowchart in FIG. 13) except for step S304 (i.e., steps S301 to S303, S305, and S306). Therefore, in the following, explanations of steps S301 to S303, S305, and S306 in this embodiment will be omitted, and only the processing of step S304 will be explained.

[0084] The image analysis unit 112 changes the posture determination criterion used to determine the posture of the care recipient 80, for example, as an image analysis method. More specifically, as shown in FIGS. 16 and 17 , when the bed 90 is in a normal state, the image analysis unit 112 sets the posture determination criterion to the normal state criterion. The normal state criterion is a criterion for determining that a standing posture is not a fall and that a sitting posture is a fall, and is pre-stored in the RAM of the calculation control unit 101. In the example shown in FIG. 16 , since the bed 90 is in a normal state, the image analysis unit 112 uses the normal state criterion to determine that the care recipient 80 is in a standing posture, for example, based on the positions of the joint points of the head, neck, shoulders, arms, waist, and feet of the care recipient 80. In this case, since the posture is standing, the image analysis unit 112 determines that the care recipient 80 is in a non-fall state. 17, the image analysis unit 112 similarly determines that the care recipient 80 is in a sitting position. In this case, since the position is a sitting position, the image analysis unit 112 determines that the care recipient 80 has fallen.

[0085] On the other hand, when the state of the bed 90 changes from the normal state to the reclined state, the image analysis unit 112 changes the posture determination criterion from the normal state criterion to the reclined state criterion. The reclined state criterion is a criterion for determining that the care recipient 80 is in a fallen state when the care recipient 80's posture is determined to be standing, and is pre-stored in the RAM of the calculation and control unit 101. In the example shown in FIG. 18 , the bed 90 is in a reclined state, so the image analysis unit 112 determines the posture of the care recipient 80 using the reclined state criterion. Because the head and neck of the care recipient 80 are outside the monitoring area, the image analysis unit 112 determines the posture of the care recipient 80 based on the positions of the joint points of the shoulders, arms, waist, and feet. In this case, the image analysis unit 112 may determine that the posture is other than standing, but in either posture, the image analysis unit 112 determines that the care recipient 80 is not in a fallen state. 19, since the bed 90 is in a reclined position, the image analysis unit 112 similarly uses the reclined position as a criterion to determine the posture of the care recipient 80. The image analysis unit 112 determines that the care recipient 80 has fallen because the positions of the joint points of the head, neck, shoulders, arms, waist, and feet of the care recipient 80 are similar to the posture of the normal standing position in terms of skeletal balance.

[0086] In this manner, in this embodiment, the posture determination criterion used to determine the posture of the care recipient 80 as an image analysis method is changed based on the movement or state of the bed 90. Then, the posture of the care recipient 80 is determined using the changed posture determination criterion, and the state of the care recipient 80 is determined based on the determination result.

[0087] The detection device 10 of this embodiment described above determines the state of the person being cared for 80 based on the results of determining the posture of the person being cared for 80, so that even if the bed 90 moves, for example by tilting, the state of the person being cared for 80 can be appropriately determined from the image.

[0088] (Fifth embodiment) In the first to fourth embodiments, the detection device 10 is installed on the mattress 94. In the fifth embodiment, the detection device 10 is installed on the headboard 92.

[0089] Fig. 20 is a schematic side view of the bed 90 in a normal state and the detection device 10 in the fifth embodiment. Fig. 21 is a schematic side view of the bed 90 and the detection device 10 in a state where the foot-side mattress 94 is inclined.

[0090] 20, the detection device 10 is installed on, for example, a headboard 92, and sets an area including the bed 90 as the monitoring area of ​​the camera 103, and captures an image of the monitoring area. When the bed 90 is in a normal state, the parameters of the camera 103 are set to initial values ​​to capture an image of the monitoring area, and the state of the care recipient 80 (fall / not fall) can be determined based on the captured image.

[0091] 21, for example, if the portion of the mattress 94 corresponding to the waist of the care recipient 80 is tilted, the distance between the camera 103 and the mattress 94 in the monitoring area becomes shorter, and the shapes of the person object 800 and the bed object 940 change from their normal shapes. As a result, the image analysis unit 112 may not be able to correctly determine the state of the care recipient 80.

[0092] Therefore, in this embodiment, the pan P and tilt T of the camera 103 are changed (controlled) to P2 and T2, respectively, and the optical axis of the camera 103 is set to an angle corresponding to the inclination angle a2 of the mattress 94, thereby making the optical axis of the camera 103 approximately parallel to the foot side of the mattress 94. Also, the magnification of the camera 103 is set to an appropriate value M2 so that the shape of the bed 90 in the captured image is similar to the shape of the bed 90 in its normal state. By setting the parameters (P, T, M) in this way and capturing an image within the monitoring area, the condition of the care recipient 80 can be appropriately determined based on the captured image.

[0093] (Sixth embodiment) In the sixth embodiment, a case where the detection device 10 is installed on a footboard 93 will be described.

[0094] Fig. 22 is a schematic side view of the bed 90 in a normal state and the detection device 10 in the sixth embodiment, and Fig. 23 is a schematic side view of the bed 90 in a reclining state and the detection device 10.

[0095] 22, the detection device 10 is installed on, for example, a footboard 93, and sets an area including the bed 90 as the monitoring area of ​​the camera 103, and captures an image of the monitoring area. When the bed 90 is in a normal state, the parameters of the camera 103 are set to initial values ​​to capture an image of the monitoring area, and the state of the care recipient 80 (fall / not fall) can be determined based on the captured image.

[0096] 23, for example, if the portion of the mattress 94 corresponding to the back of the care recipient 80 is tilted, the distance between the camera 103 and the mattress 94 in the monitoring area becomes shorter, and the shapes of the person object 800 and the bed object 940 change from their normal shapes. As a result, the image analysis unit 112 may not be able to correctly determine the state of the care recipient 80.

[0097] Therefore, in this embodiment, the pan P and tilt T of camera 103 are changed (controlled) to P3 and T3, respectively, and the optical axis of camera 103 is set to an angle corresponding to the inclination angle a3 of mattress 94, thereby making the optical axis of camera 103 approximately parallel to the head end portion of mattress 94. Also, the magnification M of camera 103 is set to an appropriate value M3 so that the shape of bed 90 in the captured image is equivalent to the shape of bed 90 in its normal state. By setting the parameters (P, T, M) in this way and capturing an image within the monitoring area, the condition of care recipient 80 can be appropriately determined based on the captured image.

[0098] The configuration of the monitoring system 1 described above is a main configuration for explaining the features of the above embodiment, but is not limited to the above configuration and can be modified in various ways within the scope of the claims.

[0099] For example, in the first to sixth embodiments described above, the detection device 10 is installed on the mattress 94, the headboard 92, or the footboard 93, but the detection device 10 may be installed in a location on the bed 90 other than the mattress 94, the headboard 92, or the footboard 93. Furthermore, the detection device 10 is not limited to being installed on the upper end of the headboard 92 or the footboard 93, and may also be installed, for example, in a plane facing the center of the bed 90.

[0100] Furthermore, in the above-described first to sixth embodiments, the case where the state of the bed 90 changes due to the tilt of the mattress 94 has been mainly described, but the present invention can also be applied to a case where the state of the bed 90 changes due to the tilt of the upper frame 911. It can also be applied to a case where the state of the bed 90 changes due to a change in the bed height. The control unit 114 controls the camera 103 or the image analysis unit 112 according to a change in the orientation or position of the camera 103.

[0101] Furthermore, in the above-described first to sixth embodiments, the bed 90 is described as an electric bed, but the bed 90 may be configured to tilt the mattress 94 or the upper frame 911 not only by electric power but also by hydraulic pressure or the like.

[0102] Furthermore, the above-described flowcharts may include steps other than those shown in the flowcharts, or some steps may not be included. The order of the steps is not limited to the above-described embodiment. Furthermore, each step may be combined with other steps and executed as a single step, may be included in other steps and executed, or may be divided into multiple steps and executed.

[0103] The means and methods for performing various processes in the above-described embodiments can be realized by either dedicated hardware circuits or a programmed computer. The program may be provided, for example, by a computer-readable recording medium such as a USB memory or a DVD (Digital Versatile Disc)-ROM, or may be provided online via a network such as the Internet. In this case, the program recorded on the computer-readable recording medium is typically transferred and stored in a storage unit such as a hard disk. The program may also be provided as standalone application software, or may be incorporated as a function into the software of a device such as a detection unit. [Explanation of symbols]

[0104] 1. Surveillance system, 10 detection device, 101 arithmetic and control unit, 102 Communications Department, 103 cameras, 111 image acquisition unit, 112 Image analysis unit, 113 Reception Department, 114 control section, 20 servers, 201 arithmetic and control unit, 202 Communications Department, 203 Memory section, 30 mobile devices, 31 access points, 90 beds, 91 Frame part, 911 upper frame, 912 lower frame, 92 headboard, 93 footboard, 94 mattresses, 95 Sensor section.

Claims

1. an image acquisition unit that acquires an image captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; A control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, the bed is an electric bed, The control unit determines the operation or state of the electric bed based on a signal from the electric bed.

2. an image acquisition unit that acquires an image captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; A control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, The image analysis unit detects objects included in the image captured by the imaging unit, compares the position of the top end of a bed object corresponding to the bed in the image with the position of the top end of a person object corresponding to the person in the image, and determines that the person has not fallen if the position of the top end of the person object is higher than the position of the top end of the bed object, but determines that the person has fallen if the position is lower.

3. an image acquisition unit that acquires an image captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; a sensor that detects the operation of the bed, including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted relative to a floor surface; A control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, The control unit is a control device that controls the orientation or position of the imaging unit based on the movement or state of the bed.

4. an image acquisition unit that acquires an image captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; a sensor that detects the operation of the bed, including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted relative to a floor surface; A control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed; a reception unit that receives a detection result from the sensor that detects the movement or state of the bed, The control unit determines the operation or state of the bed based on the detection result.

5. an image acquisition unit that acquires an image captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; a sensor that detects the operation of the bed, including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or the state of the bed, including a state in which the mattress is tilted with respect to a floor surface; A control device comprising: a control unit that controls the imaging unit based on the operation or state of the bed.

6. the control unit controls the image analysis unit to change a state determination threshold used to determine the state of the care recipient or a posture determination standard used to determine the posture of the care recipient based on the movement or state of the bed; the posture determination criteria include a normal state criterion that is set when the bed is in a normal state, and a reclined state criterion that is set when the bed is in a reclined state, The standard for the normal state is a standard for determining that the care recipient is not in a fallen state when the posture of the care recipient is determined to be standing, and determining that the care recipient is in a fallen state when the posture of the care recipient is determined to be sitting, A control device as described in any one of claims 1 to 4, wherein the criterion for the reclining state is a criterion for determining that the person being cared for is in a fallen state when the posture of the person being cared for is determined to be upright.

7. An image acquisition unit that acquires images captured by an imaging unit installed on the bed; an image analysis unit that analyzes the image; A control unit that controls the imaging unit or the image analysis unit based on the operation or state of the bed, The control unit controls the orientation or position of the imaging unit based on the movement or state of the bed, The control unit converts the position of each pixel in the image acquired by the image acquisition unit to the position of the image that would be obtained if the bed were in a normal state based on the operation or state of the bed, and controls the image analysis unit to correct the image acquired by the image acquisition unit by performing calculations to interpolate between the pixels.

8. further comprising a storage unit that stores the state determination threshold value; The control device according to claim 6, wherein the image analysis unit detects an object included in an image captured by the imaging unit, and determines the state of the person based on the result of comparing the position of the top end of a human object corresponding to the person in the image with the state determination threshold.

9. The control device according to claim 8 , wherein the image analysis unit changes the state determination threshold value based on the movement or state of the bed.

10. a storage unit that stores the posture determination criteria; 7. The control device according to claim 6, wherein the image analysis unit detects a skeleton and joint points of a human object corresponding to a person included in an image captured by the imaging unit, and determines a posture of the person and a state of the person corresponding to the posture based on the detected skeleton and joint points and the posture determination criterion.

11. The control device according to claim 10 , wherein the image analysis unit changes the posture determination criterion based on a movement or state of the bed.

12. an imaging step of imaging a subject including the electric bed by an imaging unit installed in the electric bed; an acquisition step of acquiring the image captured by the imaging unit by an image acquisition unit; A control method comprising a control step of determining the operation or state of the electric bed based on a signal from the electric bed, and controlling an image analysis unit that analyzes the image based on the operation or state of the electric bed.

13. an imaging step of imaging an object including the bed by an imaging unit installed on the bed; an acquisition step of acquiring the image captured by the imaging unit by an image acquisition unit; A control step of controlling an image analysis unit that analyzes the image based on the operation or state of the bed, In the control step, an object included in the image captured by the imaging unit is detected, and the position of the top end of a bed object corresponding to the bed in the image is compared with the position of the top end of a person object corresponding to the person in the image.If the position of the top end of the person object is higher than the position of the top end of the bed object, it is determined that the person has not fallen, but if it is lower, it is determined that the person has fallen.

14. A detection step of detecting a bed operation including an operation of tilting a mattress of the bed and an operation of moving the mattress of the bed, or a state of the bed including a reclining state in which the mattress is tilted relative to a floor surface; a control step of controlling the imaging unit based on an operation or a state of the bed on which the imaging unit is installed; an imaging step of imaging an object including the bed by the imaging unit; an acquisition step of acquiring the image captured by the imaging unit by an image acquisition unit; and analyzing the image.

15. an imaging step of imaging an object including the bed by an imaging unit installed on the bed; an acquisition step of acquiring the image captured by the imaging unit by an image acquisition unit; a detection step of detecting, by a sensor, the operation of the bed, including the operation of tilting the mattress of the bed and the operation of moving the mattress of the bed, or the state of the bed, including a reclining state in which the mattress is tilted with respect to a floor surface; receiving a detection result from the sensor that detects the movement or state of the bed; A control step of determining the movement or state of the bed based on the detection result and controlling an image analysis unit that analyzes the image based on the movement or state of the bed; A control method comprising:

16. A control program for causing a computer to execute the control method according to any one of claims 12 to 15.

17. An imaging unit installed on the bed, A control device according to any one of claims 1 to 11; An information processing system having:

Images