DETECTION SYSTEM, DETECTION DEVICE, TERMINAL DEVICE, DETECTION METHOD, AND PROGRAM

Abstract

To provide a technique capable of flexibly meeting various requests of a use.SOLUTION: A detection system comprises: a terminal device which receives a setting operation of a detection object event concerning an observation object person; and a detection device which detects the detection object event on the basis of a setting content by the setting operation. For example, by a setting operation using a setting screen 200, the detection object event is set as an event satisfying conditions concerning a three-dimensional position of a specific portion in the observation object person, variation in the three-dimensional position, and / or the variation speed of the three-dimensional position and the like.SELECTED DRAWING: Figure 20

Description

[Technical field]

[0001] The present invention relates to a detection device that detects a detection target event related to an observed person, and to a technique related thereto. [Background technology]

[0002] There is technology that detects target events (such as falls) related to a target person (such as a person being watched).

[0003] For example, Patent Document 1 describes a technology for monitoring the condition of a resident of a nursing home (also called a person to be observed) who is a monitoring target (a person to be watched over). The monitored conditions (also called a detection target event) include a fall of a person receiving care (a person requiring care) in the room, leaving the room (abnormal leaving), etc.

[0004] In the technology of Patent Document 1, a program for detecting each specific behavior (event to be detected) to be monitored is prepared (see paragraph 0038 of Patent Document 1). Then, each specific behavior is detected by running each program. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2017-228042 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology of the above-mentioned Patent Document 1, a program needs to be created in advance for each operation to be detected, so that the events to be detected are, so to speak, fixed.

[0007] However, various demands exist in actual nursing care sites, etc. With the above-mentioned fixed method, it is difficult to flexibly respond to such various demands.

[0008] Therefore, the present invention provides a technique that can flexibly respond to various requests from users. Impose The subject is: [Means for solving the problem]

[0012] The above issues In order to solve the above problem, the detection system according to the present invention includes a terminal device that receives a setting operation of a detection target event related to an observed person, and a detection device that detects the detection target event based on the setting content of the setting operation. The terminal device accepts a setting operation that sets each detection target event related to a plurality of rooms for each room, the setting operation including an operation of setting height information of the person to be observed for each room, and the detection device detects the detection target event by excluding people who do not match the height information from the detection target people.

[0013] In order to solve the above problem, the detection method of the present invention is a detection method for detecting a detectable event related to an observed person, comprising: a) a step of accepting a setting operation of a detectable event related to the observed person; and b) a step of detecting the detectable event based on the setting contents set by the setting operation, wherein the step a) comprises a step of accepting a setting operation which is a setting operation for setting each detectable event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of the observed person for each room; and the step b) comprises a step of excluding persons who do not match the height information from the detected persons, and detecting the detectable event.

[0014] In order to solve the above problem, the program of the present invention is a program for causing a computer to execute the steps of a) accepting a setting operation of a detection target event related to an observed person, and b) detecting the detection target event based on the setting contents set by the setting operation, wherein the step a) includes a step of accepting a setting operation which is a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of the observed person for each room, and the step b) includes a step of detecting the detection target event by excluding persons who do not match the height information from the detection target persons.

[0015] In order to solve the above problems, the detection device of the present invention comprises a reception unit that receives setting operations of detection target events related to a person to be observed, and a control unit that detects the detection target events based on the setting contents of the setting operations, wherein the reception unit receives setting operations that set each detection target event related to a plurality of rooms for each room, the setting operations including an operation of setting height information of the person to be observed for each room, and the control unit detects the detection target events by excluding persons that do not match the height information from the people to be detected.

[0016] In order to solve the above problem, the terminal device of the present invention comprises a reception unit that receives a setting operation of a detectable event related to a person to be observed, and a control unit that transmits setting information based on the setting operation to a detection device that detects the detectable event, wherein the reception unit receives a setting operation that sets each detectable event related to a plurality of rooms for each room, the setting operation including an operation of setting height information of the person to be observed for each room, and the control unit transmits the setting information including the height information in order to have the detection device detect the detectable event after excluding persons that do not match the height information from the people to be detected.

[0017] In order to solve the above problem, the program of the present invention is a program for causing a computer to execute the steps of a) accepting a setting operation of a detectable event related to a person to be observed, and b) transmitting setting information based on the setting operation to a detection device that detects the detectable event, wherein the step a) includes a step of accepting a setting operation that sets each detectable event related to a plurality of rooms for each room, the setting operation including an operation of setting height information of the person to be observed for each room, and the step b) includes a step of transmitting the setting information including the height information in order to cause the detection device to detect the detectable event after excluding persons that do not match the height information from the people to be detected. Effect of the Invention

[0037] The present invention This makes it possible to flexibly respond to various user requests. [Brief description of the drawings]

[0039] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of a detection system. [Diagram 2] FIG. 2 is a functional block diagram showing a schematic configuration of a detection device (image processing device). [Diagram 3] FIG. 2 is a functional block diagram showing a schematic configuration of a management device. [Figure 4] FIG. 2 is a functional block diagram showing a schematic configuration of a mobile terminal device. [Diagram 5] 4 is a flowchart showing a process flow in the detection device. [Figure 6] 1 is a conceptual diagram showing the generation of three-dimensional position information regarding each specific part of a person. [Figure 7] FIG. 13 is a diagram showing "getting up from bed." [Figure 8] FIG. 13 is a diagram showing the “edge sitting position.” [Figure 9] FIG. [Figure 10] FIG. 13 illustrates a "fall on the floor." [Figure 11] FIG. 1 shows a diagram showing "lying on the floor." [Figure 12] This is a diagram showing "full-body sliding off the bed." [Figure 13] This is a diagram showing "the lower body sliding off the bed." [Figure 14] FIG. 13 is a diagram showing "one arm reaching out from the bed." [Figure 15] FIG. 13 is a diagram showing a selection screen for selecting a room to be displayed. [Figure 16] FIG. 2 shows an observation screen of an observed person in a certain room. [Figure 17] FIG. 2 shows an observation screen of an observed person in a certain room. [Figure 18] 11A to 11C are diagrams for explaining a generation process of an observation image, etc.; [Figure 19] FIG. 13 is a diagram showing a setting screen (registration screen) for detecting events to be detected. [Figure 20] FIG. 13 is a diagram showing a setting screen for a detection target event "getting up (on the bed)." [Figure 21] FIG. 13 is a diagram showing a setting screen for the detection target event “sitting on the edge of bed.” [Figure 22] FIG. 13 is a diagram showing a setting screen for the detection target event “sitting on edge of bed.” [Diagram 23] FIG. 13 is a diagram showing a setting screen for the detection target event “getting out of bed.” [Figure 24] FIG. 13 is a diagram showing a setting screen for a detection target event "fall (on floor)." [Diagram 25] FIG. 13 is a diagram showing a setting screen for the detection target event "lying down (on the floor)." [Figure 26]FIG. 13 is a diagram showing a setting screen for the detection target event “full-body slippage from bed.” [Figure 27] FIG. 13 is a diagram showing a setting screen for the detection target event “lower body slipping off the bed.” [Figure 28] FIG. 13 is a diagram showing a setting screen for the detection target event “one hand reaching out from the bed.” [Figure 29] FIG. 13 is a diagram showing a setting screen for the detection target event “one hand reaching out from the bed.” [Diagram 30] 13 is a flowchart showing a process in the management device. [Diagram 31] FIG. 13 is a diagram showing "falling on a bed." [Diagram 32] FIG. 13 is a diagram showing a setting screen for a detection target event “falling over.” DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0040] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0041] <1. System Overview> FIG. 1 is a diagram showing a detection system 1 according to the present invention. As shown in FIG. 1, the detection system 1 includes a plurality of detection devices 10 and a plurality of terminal devices 70, 80. The terminal device 70 is also referred to as a management device 70, and the terminal device 80 is also referred to as a mobile terminal device 80. The detection system 1 is a system that detects a detection target event (such as "falling") related to an observed person (such as a person receiving care). Note that the detection device and the detection system are also referred to as a monitoring device and a monitoring system, etc., since they monitor the behavior of a person, etc.

[0042] Each detection device 10 and each terminal device 70, 80 are connected to each other via a network 108. The network 108 is configured by a LAN (Local Area Network) and the Internet, etc. The connection to the network 108 may be a wired connection or a wireless connection. For example, the management device 70 is connected to the network 108 by a wire, and each detection device 10 and each mobile terminal device 80 are connected to the network 108 by wireless. Alternatively, all devices 10, 70, 80 may be connected to the network 108 by wireless.

[0043] In the detection system 1, various input / output processes (operation input processes, display processes, etc.) related to each detection device 10 are performed using the management device 70 and the mobile terminal device 80. That is, the management device 70 and the mobile terminal device 80 each function as a terminal device related to the multiple detection devices 10.

[0044] The following mainly illustrates an example in which the detection system 1 is used in a nursing facility. However, the detection system 1 is not limited to this, and may be used in a nursing facility (such as a hospital) or a factory.

[0045] Each detection device 10 is disposed in each room 90 (e.g., a private room for each care recipient) of each person to be observed (here, a care recipient). Each detection device 10 is a device that detects a detection target event (such as "falling") related to the person to be observed based on a captured image of the person to be observed. As will be described later, the "detection target event" is set (defined) by a setting operation on the terminal device 70. Then, the detection device 10 detects the detection target event based on the settings made by the setting operation.

[0046] Each detection device 10 includes a camera unit 20 and a processing unit 30. The camera unit 20 is installed on the ceiling of a room 90 or the like, and is capable of capturing images of the interior of the room 90. The processing unit 30 is capable of acquiring three-dimensional positions (particularly time-series information) of a plurality of specific parts (chest, waist, shoulders, elbows, wrists, knees, ankles, etc.) of a person to be observed, based on images (particularly moving images) captured by the camera unit 20. Note that, although the camera unit 20 and the processing unit 30 are provided separately here, this is not limiting, and the camera unit 20 and the processing unit 30 may be provided integrally.

[0047] The camera unit 20 is a three-dimensional camera. The camera unit 20 acquires a captured image with depth information. Specifically, the camera unit 20 captures a captured image (such as an infrared image) 110 (see FIG. 6) of a subject object (such as a person, a wall, a floor 91, a bed 92, etc.), and acquires depth information (depth distance information) 120 of each pixel in the captured image 110. The depth information 120 of each pixel in the captured image is information on the distance (distance from the camera unit 20) to the subject object (such as a person, a wall, a floor, a bed, etc.) corresponding to each pixel in the captured image 110, and is information on the distance in a direction perpendicular to the captured image 110. In other words, the depth information is distance information (depth distance information) in the normal direction of the captured image plane.

[0048] For example, the camera unit 20 includes a sensor section 23 (FIG. 2), which includes an infrared camera (such as an infrared image sensor) and an infrared projector. The infrared camera acquires a captured image 110 (such as an infrared image) of a subject object. The infrared projector also irradiates a predetermined dot pattern by infrared rays toward the subject object, and the infrared camera also acquires an infrared image of the dot pattern irradiated on the subject object. The distance (depth distance information) 120 to the subject object is calculated (acquired) by utilizing the fact that the geometric shape of the captured dot pattern changes depending on the distance to the subject object. The calculation process of the depth distance information is executed by a controller 21 or the like incorporated in the camera unit 20. The controller 21 has the same hardware configuration as a controller 31 (described later) or the like. Note that the sensor section 23 may be provided with an RGB image sensor or the like that captures a visible light image (such as a color image), and a color image may be captured.

[0049] In this way, the camera unit 20 acquires a captured image (captured image information) 110 (see FIG. 6) of the subject person and depth information of each pixel in the captured image (information on the distance to the object corresponding to each pixel (depth distance information)) 120.

[0050] FIG. 6 is a conceptual diagram showing that information on the three-dimensional position of each specific part of a subject person is acquired (calculated) based on a photographed image 110 and depth information 120. In FIG. 6, the photographed image 110 and the depth information 120 are each shown in a schematic manner. For example, in the actual photographed image 110, a person is captured as is, whereas in the photographed image 110 of FIG. 6, the person is represented (simplified) by a figure in which ellipses are connected. The same is true for the other depth information 120. Moreover, while the actual depth information 120 has information on the distance to an object corresponding to each pixel, in the depth information 120 of FIG. 6, the distance to an object corresponding to each pixel is expressed by converting it into the density of each pixel. In detail, the magnitude of the distance is expressed by the magnitude of density (shade).

[0051] First, the processing unit 30 (particularly the controller 31) analyzes the photographed image 110 and obtains skeletal information (skeletal model information) 140 (see the lower left part of FIG. 6) of the photographed person. The skeletal information 140 is information that expresses (in a simplified manner) the skeleton of the photographed person using a plurality of specific parts (specifically, chest, nose, shoulders, elbows, wrists, waist, knees, ankles, eyes, ears, etc.) (mainly joints) of the photographed person and skeletal lines (links) connecting the plurality of specific parts. In the skeletal information 140 of FIG. 6, each specific part is indicated by a "point", and each skeletal line (connecting line) is indicated by a "line segment". The skeletal information 140 based on the photographed image has information (such as planar position information of each part in the image) on a plurality of specific parts (here, B0 to B17).

[0052] In addition, the processing unit 30 acquires planar position information (two-dimensional position information within the captured image in the camera coordinate system) of each specific part (for example, each representative position) of the captured person in the captured image 110 based on the skeletal information 140 of the captured person.

[0053] Furthermore, the processing unit 30 also acquires distance information (depth position information in the camera coordinate system) from the camera unit 20 to each specific part based on the depth information 120 of one or more pixels at a planar position (within the photographed image 110) corresponding to each specific part. The distance information is also expressed as depth information in the normal direction of the photographed image plane.

[0054] In this way, the processing unit 30 obtains information regarding the planar position of each specific part of the photographed person in the photographed image (two-dimensional position information in the photographed image) and distance information (depth information) to each specific part of the photographed person.

[0055] Then, the processing unit 30 acquires three-dimensional position information 150 (three-dimensional position information in the living space) of each specific part of the subject person based on information on the planar position of each specific part in the photographed image and distance information (depth information) to each specific part by coordinate conversion or the like. Specifically, the processing unit 30 converts the position information in the camera coordinate system Σ1 (planar position in the photographed image and depth position in the normal direction of the photographed image) into three-dimensional position information 150 in a coordinate system Σ2 fixed with respect to the living room. The camera coordinate system Σ1 is, for example, a three-dimensional orthogonal coordinate system based on three orthogonal axes, namely, two orthogonal axes parallel to the photographed image plane and one axis extending in a direction perpendicular to the photographed image plane. The coordinate system Σ2 after conversion is, for example, a three-dimensional orthogonal coordinate system based on three orthogonal axes, namely, two orthogonal axes parallel to the horizontal plane and one axis extending in the vertical direction (height direction).

[0056] In this way, the processing unit 30 acquires (calculates) information on the three-dimensional position of each specific part of the subject person. Here, the subject person in the captured image is regarded as the observed person in principle. When there are multiple subject people within the camera's field of view, all of the subject people need only be identified as the observed person. However, this is not limited to this, and a desired observed object may be identified from among the multiple subject people using information such as height (or face recognition technology, etc.).

[0057] In this embodiment, the controller 31 of the processing unit 30 generates the skeletal information 140 and the three-dimensional position information 150 of each specific part, but this is not limiting.

[0058] For example, the controller 21 of the camera unit 20 may generate the skeletal information 140. Also, the controller 21 of the camera unit 20 may generate the three-dimensional position information 150 of each specific part. In detail, the controller 21 may obtain information on the position of each specific part of the subject person in the captured image and distance information (depth information) to each specific part, and obtain the three-dimensional position information 150 of each specific part by coordinate conversion or the like. Then, the controller 31 of the processing unit 30 may obtain the three-dimensional position information 150 of each specific part from the camera unit 20. Alternatively, both the controllers 21, 31 of the camera unit 20 and the processing unit 30 may cooperate to execute these various processes. In other words, the skeletal information 140 and the three-dimensional position information 150 may be generated by both or one of the controllers 21, 31.

[0059] Also, here, a three-dimensional camera using a pattern illumination method that acquires depth information of each pixel based on the illumination of a dot pattern is exemplified, but is not limited to this. The three-dimensional camera may be a stereoscopic camera or a TOF (Time of Flight) camera. Any type of three-dimensional camera may acquire a captured image of a person to be observed and depth information (such as depth information of each pixel) that is information on the distance to a subject object in the captured image.

[0060] In addition, the detection device 10 is also referred to as an image processing device since it is a device that executes image processing on captured images, etc. In addition, the detection system 1 is also referred to as an image processing system, etc.

[0061] <2.Detection device 10> FIG. 2 is a functional block diagram showing a schematic configuration of the detection device 10. As shown in FIG.

[0062] As shown in the functional block diagram of FIG. 2, the detection device 10 includes a camera unit 20 and a processing unit 30.

[0063] As described above, the camera unit 20 includes an infrared camera, an infrared projector, and the like.

[0064] The processing unit 30 also includes a controller 31, a storage unit 32, a communication unit 34, and an operation unit 35.

[0065] The controller 31 is a control device that is built into the processing unit 30 and controls the detection device 10 .

[0066] The controller 31 is configured as a computer system including a CPU (Central Processing Unit) (also referred to as a microprocessor or a hardware processor, etc.). The controller 31 realizes various processes by executing, in the CPU, a predetermined software program (hereinafter also simply referred to as a program) stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 32. The program (specifically, a group of program modules) may be recorded in a portable recording medium such as a USB memory, read from the recording medium, and installed in the detection device 10. Alternatively, the program may be downloaded via a communication network or the like and installed in the detection device 10.

[0067] The controller 31 acquires captured image information on the observed person from the camera unit 20, and also acquires distance information (depth information) to each specific part of the observed person (subject person) from the camera unit 20. Then, the controller 31 acquires three-dimensional position information of each specific part based on the captured image information (particularly, planar position information of each specific part of the subject in the captured image) and the distance information (depth information of the subject).

[0068] Furthermore, the controller 31 detects a "detection target event" (described later) based on the three-dimensional position information of each specific part. In detail, the controller 31 sets a "detection target event" (described later) in response to a setting operation received by the terminal devices 70, 80. Then, the controller 31 detects the detection target event based on the setting contents of the setting operation.

[0069] Moreover, the controller 31 controls the process of displaying a monitoring screen or the like related to the observed person on the terminal device 70, 80. In detail, the controller 31 generates image data or the like of an observed image 330 (FIG. 18) as described below, and transmits the image data or the like to the terminal device 70, 80. In response to this, the observed image 330 based on the image data or the like is displayed on the terminal device 70, 80.

[0070] The storage unit 32 is configured with a storage device such as a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage unit 32 stores (records) time-series data of the three-dimensional positions of a plurality of specific parts of the observed person, etc.

[0071] The communication unit 34 is capable of performing network communication via the network 108. In this network communication, various protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) are used. By using this network communication, the detection device 10 can transmit and receive various data to and from a desired destination (for example, the terminal devices 70 and 80).

[0072] The operation unit 35 includes an operation input unit 35a that receives operation input for the detection device 10, and a display unit 35b that displays and outputs various information. In the present system 1, the input / output functions (operation input reception function and output function (display function, etc.)) for the detection device 10 are mainly performed by the terminal devices 70, 80. Therefore, the operation unit 35 may be of a type that is attached to the processing unit 30 only during maintenance, etc.

[0073] <3. Terminal Devices 70, 80> Next, the configuration of the terminal devices 70 and 80 will be described.

[0074] Of the terminal devices 70 and 80, the management device 70 is a device that manages the entire detection system 1, and is mainly operated by an administrator. On the other hand, the mobile terminal device 80 functions as a display device or the like that displays various information in the detection system 1. The mobile terminal device 80 is carried by a person (caregiver) who cares for a care recipient, and displays various information related to the care recipient, etc. The management device 70 also functions as a display device that displays various information in the detection system 1. Furthermore, the management device 70 also functions as a setting reception device that receives setting operations by a user regarding detection target events (described later) and the like.

[0075] The terminal devices 70 and 80 are information input / output terminal devices (information processing devices) capable of network communication with other devices (such as 10). The terminal devices 70 and 80 are configured as smartphones, tablet terminals, or personal computers (which may be fixed (desktop) or portable). For example, the terminal device 70 is a fixed personal computer, and the terminal device 80 is a portable device (portable terminal device), more specifically, a smartphone.

[0076] First, the management device 70 will be described.

[0077] FIG. 3 is a functional block diagram showing a schematic configuration of the management device 70. As shown in FIG.

[0078] As shown in the functional block diagram of FIG. 3, the management device 70 includes a controller 71, a memory unit 72, a communication unit 74, an operation unit 75, etc., and realizes various functions by operating these units in a combined manner.

[0079] The controller (control unit) 71 is a control device that controls the management device 70.

[0080] The controller 71 has the same hardware configuration as the controller 31. The controller 71 executes, in a CPU, a predetermined software program stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 72, thereby realizing various processes. The program (specifically, a group of program modules) may be recorded in a portable recording medium such as a USB memory, read from the recording medium, and installed in the management device 70. Alternatively, the program may be downloaded via a communication network or the like and installed in the management device 70.

[0081] The controller 71 executes the program and performs the following various processes.

[0082] The controller 71 cooperates with the operation unit 75 to accept various setting operations and the like. For example, the controller 71 accepts a setting operation of a detection target event related to an observed person. More specifically, the controller 71 generates a setting screen 200 of a detection target event related to an observed person, displays the setting screen 200 on the display unit 75b, and accepts a user operation using the setting screen 200. In addition, the controller 71 transmits the setting contents based on the setting operation to the detection device 10, and causes the detection device 10 to execute a registration process (setting process) for registering the detection target event.

[0083] In addition, the controller 71 causes the display unit 75b to display information about the observed person (such as a skeleton image 325 (described later)) based on the image information for display (such as an observed image 330 (see Figures 16 and 17) (described later)) transmitted from the detection device 10.

[0084] The storage unit 72 has the same hardware configuration as the storage unit 32. The storage unit 72 (particularly a non-volatile storage unit) stores the settings (such as settings related to the detection target event) made by the management device 70. The storage unit 72 may store (record) three-dimensional position information and the like (such as time-series data on the three-dimensional positions of a plurality of specific parts related to the observed person) acquired by the detection device 10 and transferred from the detection device 10.

[0085] The communication unit 74 has the same hardware configuration as the communication unit 34. By utilizing the network communication by the communication unit 74, the management device 70 can transmit and receive various data to and from a desired counterpart (e.g., each of the devices 10 and 80).

[0086] The operation unit 75 includes an operation input unit 75a that receives operation input to the management device 70, and a display unit 75b that displays and outputs various information. A mouse, a keyboard, or the like is used as the operation input unit 75a, and a display (such as a liquid crystal display) is used as the display unit 75b.

[0087] Next, the mobile terminal device 80 will be described.

[0088] FIG. 4 is a functional block diagram showing a schematic configuration of the mobile terminal device 80. As shown in FIG.

[0089] As shown in the functional block diagram of FIG. 4, the mobile terminal device 80 includes a controller 81, a memory unit 82, a communication unit 84, an operation unit 85, etc., and various functions are realized by operating these units in a combined manner.

[0090] The controller 81, the storage unit 82, the communication unit 84, and the operation unit 85 have the same hardware configurations as the controller 71, the storage unit 72, the communication unit 74, and the operation unit 75, respectively.

[0091] The controller (control unit) 81 is a control device (also called a controller) that controls the mobile terminal device 80.

[0092] The controller 81 realizes various processes by executing, in the CPU, a predetermined software program stored in a storage unit (a non-volatile storage unit such as a ROM and / or a hard disk) 82. The program (specifically, a group of program modules) may be recorded in a portable recording medium such as a USB memory, read from the recording medium, and installed in the mobile terminal device 80. Alternatively, the program may be downloaded via a communication network or the like and installed in the mobile terminal device 80.

[0093] The controller 81 executes the program and performs the following various processes.

[0094] For example, the controller 81 causes the display unit 85b to display information about the observed person (such as a skeleton image 325 (described later)) based on image information for display (such as an observed image 330 (see Figures 16 and 17) (described later)) transmitted from the detection device 10.

[0095] The mobile terminal device 80 can use network communication by the communication unit 84 to send and receive various data to and from a desired destination (eg, each of the devices 10 and 70).

[0096] The mobile terminal device 80 is provided with a touch panel 87 (see FIG. 1). The touch panel 87 is configured by embedding various sensors in a liquid crystal display panel, and is capable of displaying various information and receiving various operational inputs from an operator (user). The touch panel 87 functions as a part of the operation input unit 85a and also functions as a part of the display unit 85b.

[0097] <4. Events to be detected> Next, the detection target events will be described.

[0098] In the detection system 1, it is possible to detect various detection target events related to the observed person. A "detection target event" is an event that should be detected related to the observed person (an event that is the target of the detection process). Specifically, a "detection target event" includes at least one of the state of the person and the behavior of the person.

[0099] Examples of "detection target events" include "getting up from bed," "sitting on the edge of bed," "getting out of bed," "falling on the floor," "lying on the floor," "sliding the whole body off the bed," "sliding the lower body off the bed," and "reaching one arm out from the bed." Figures 7 to 14 are diagrams showing these detection target events (events E1 to E8), respectively.

[0100] Event E1 “rising from bed” (see FIG. 7) is a movement in which the upper body of the observed person (who was lying down) rises up on the bed. Event E1 is detected when, for example, the condition that the chest height of the observed person (chest height from the top surface of the bed) becomes 30 cm or higher in the bed area R1 (planar area in which the bed is located) is satisfied.

[0101] Event E2 "sitting on the edge of the bed" (see FIG. 8) is a state in which the observed person is sitting (sitting) on ​​the edge of the bed. "Sitting on the edge" is one of the "boundary positions" events that occur near the boundary between the bed area R1 and an adjacent area. "Sitting on the edge" is detected, for example, when the following conditions are met: the distal part of the observed person's lower body (ankles, etc.) moves from the bed area R1 to the bedside area R2, and other parts of the observed person (waist, etc.) remain in the bed area R1. "Sitting on the edge" can also be expressed as a composite event of the movement of both ankles of the observed person from the bed area R1 to the bedside area R2, and the state in which the observed person's waist is present within the bed area R1.

[0102] Event E3 “leaving the bed” (see FIG. 9) is the movement of the observed person leaving the bed and moving to a place other than the bed. “Leaving the bed” is detected, for example, when the condition that the observed person’s whole body (all specific parts) moves from the bed area R1 to the bedside area R2 is satisfied.

[0103] Event E4 “Fall on the floor” (see FIG. 10) is an action of the observed person falling on the floor. “Fall on the floor” is detected, for example, when the condition is satisfied that a specific part (chest, etc.) of the observed person enters a space below a predetermined height (e.g., 50 cm) above the floor surface at a predetermined speed (e.g., 1 m / s) or more in a flat area other than the bed area R1.

[0104] Incidentally, the events E1, E2, E3, and E4 may occur in this order. In order to prevent the occurrence of the most dangerous event E4 among these events E1 to E4, it is preferable to detect the events E1, E2, and E3, which occur before the event E4, respectively (sequentially). By detecting each of the events E1, E2, and E3 and notifying a caregiver or the like, it is possible to prevent the occurrence of the dangerous event E4.

[0105] Event E5 "lying on the floor" (see FIG. 11) is a state in which the observed person lies on the floor (has a supine position) for a predetermined period of time or more. "Lying on the floor" is detected, for example, when the following condition is met: a specific part of the upper body (nose, chest, etc.) remains in a space outside the range of the bed area R1 at a predetermined height (e.g., 50 cm) or less from the floor surface for a predetermined period of time (e.g., 5 seconds) or more.

[0106] Event E6 "whole body sliding off the bed" (see FIG. 12) is the movement of the observed person's whole body sliding off the bed to the bedside. "Whole body sliding off the bed" is detected when the following condition is met. The condition is, for example, that the position of multiple main parts (upper body chest, lower body waist, etc.) moves from the bed area R1 to the bedside area R2 at a predetermined speed (e.g., 1 m / s) or more to a space at a predetermined height (e.g., 50 cm) or less.

[0107] Event E7 "lower body sliding off bed" (see FIG. 13) is the movement of the observed person's lower body sliding off the bed to the bedside. "Lower body sliding off bed" is detected when the following condition is met. The condition is, for example, that the position of the distal part (waist, etc.) of the observed person's lower body moves from the bed area R1 to the bedside area R2 at a predetermined speed (e.g., 1 m / s) or more into a space at a predetermined height (e.g., 50 cm) or less.

[0108] Event E8 “one hand reaching out from the bed” (see Figure 14) is a state in which the observed person has one hand reaching out toward the bedside. “One hand reaching out from the bed” is one of the “boundary positions” events that occur near the boundary between the bed area and other areas.

[0109] "One hand reaching out from the bed" is detected, for example, when the conditions are met that a part of one hand (wrist) of the observed person moves from the bed area R1 to the bedside area R2 and the waist of the observed person remains in the bed area R1. Note that "one hand reaching out from the bed" can also be expressed as a composite event of the action of one wrist of the observed person moving from the bed area R1 to the bedside area R2 and the state in which the waist of the observed person is present in the bed area R1.

[0110] Event E8 is an event that may lead to the occurrence of dangerous events E5 to E7, and is preferably detected before the occurrence of these events E5 to E7. By detecting event E8 and notifying a caregiver or the like, it is possible to prevent the occurrence of dangerous events E5 to E7.

[0111] <5. Processing flow> 5 is a flowchart showing the flow of processing in the detection device 10 (particularly the processing unit 30). The processing in the detection system 1 will be described with reference to FIG.

[0112] First, the management device 70 accepts a setting operation of a detection target event related to an observed person. Specifically, the management device 70 displays a setting screen 200 on its display unit 75b, and accepts an operation input from an operating user (administrator, etc.). The management device 70 transmits setting information related to the operation input from the operating user to the detection device 10. The detection device 10 sets (registers) a "detection target event" based on the received setting information (step S1). The setting operation using the management device 70 will be described later.

[0113] Thereafter, when an instruction to start an observation process (detection process, etc.) is given by the user, the detection device 10 repeatedly executes the processes (detection process, etc.) of steps S11 to S15 at short time intervals (for example, every tens of milliseconds to hundreds of milliseconds) as shown in Fig. 5. Specifically, the detection process, etc. is executed to detect a detection target event related to the observed person. In other words, the detection target event is detected based on information on the three-dimensional positions of a plurality of specific parts of the observed person in time series, etc.

[0114] In step S11, the detection device 10 acquires skeletal information 140 of the observed person and three-dimensional position information 150 of a plurality of specific parts of the observed person based on the captured images (110, 120) with depth information of the observed person.

[0115] For example, as described above, the detection device 10 acquires skeletal information 140 of the observed person based on the captured images with depth information (particularly, captured images of the observed person) (110, 120) acquired by the camera unit 20. The detection device 10 also acquires planar position information (information on two-dimensional position) of each specific part of the observed person in the captured images and distance information (depth information) to each specific part of the observed person. Then, based on this information, the detection device 10 acquires three-dimensional position information 150 of each specific part (three-dimensional position information within the living space) by coordinate conversion or the like.

[0116] In step S12, the detection device 10 generates an image for monitoring display (also referred to as an observation image). Specifically, the detection device 10 generates a skeleton image 325 (see FIG. 18) (described later) of the observed person from the photographed image 110 of the observed person, and generates an observation image 330 regarding the observed person based on the skeleton image 325. The observation image 330 is an image that does not have a photographed image of the observed person itself but has a skeleton image of the observed person.

[0117] Then, the detection device 10 transmits image data relating to the observation image 330 to the terminal device 70, 80 (more specifically, the device that sent the monitoring display request (described later) to the detection device 10), and causes the observation image 330 to be displayed on the display unit of the terminal device 70 or the like (step S12). In this way, the photographed image of the observed person is not displayed as is, but the observation image 330 of the observed person is displayed. Therefore, it is possible to realize a display that fully considers the privacy of the observed person.

[0118] Furthermore, in step S12, the time-series observed images 330 are stored in the storage unit 32 as log images (images for recording).

[0119] In step S13, the detection device 10 executes detection processing for each of a plurality of detection target events (for example, events E1 to E8) based on the three-dimensional position information of a plurality of specific parts of the observed person. The detection device 10 detects each detection target event based on the setting contents (the contents set in step S1) by the setting operation received by the management device 70 or the like.

[0120] In step S14, it is determined whether any of the detection target events has been detected.

[0121] If none of the detection target events is detected, the process returns to step S11.

[0122] On the other hand, if any of the detection target events is detected, the process proceeds to step S15. In step S15, processing (such as an alert process) is performed according to the detection result. For example, when event E4 "Fall on floor" is detected, an "alert" is issued to the terminal devices 70, 80, etc. For example, an alarm sound is output from each of the terminal devices 70, 80, and an alarm screen including text such as "Fall in Room 101!" is displayed on the display units 75b, 85b. This allows the caregiver, etc., to know that event E4 "Fall on floor" has occurred.

[0123] <6. Setting operations, etc.> 30 is a flowchart showing the processing (such as the processing for accepting setting operations related to events to be detected) of the management device 70. The processing of step S1 (FIG. 5) in the detection device 10 is performed in accordance with the processing of FIG.

[0124] In step S31, the management device 70 displays a setting screen 200 (see FIG. 19) on the display unit 75b in response to a predetermined operation by an operating user (administrator, etc.). The setting screen (also referred to as a registration screen) 200 is a screen for setting (registering) a detection target event related to an observed person. Then, the management device 70 uses the setting screen 200 to accept a setting operation of a detection target event related to an observed person.

[0125] Next, in step S32, the management device 70 transmits setting information based on the setting operation accepted in step S31 to the detection device 10 (processing unit 30). The detection device 10 sets (registers) detection target events based on the setting information received from the management device 70 (see step S1 in FIG. 5). In other words, the detection target events are customized and registered according to the user's request. Note that the setting information transmitted from the management device 70 to the detection device 10 may be the information accepted by the management device 70 itself, or may be information processed by the management device 70.

[0126] In such processing, a certain event to be detected is set as an event that satisfies a condition regarding the three-dimensional position (planar position (particularly planar areas R1, R2 (described later) etc.) and / or height etc.) of a specific part of the observed person. In particular, a certain event to be detected is set as an event that satisfies a condition regarding the positional relationship between a specific part of the observed person and a predetermined object (such as a bed) other than the observed person.

[0127] Moreover, a certain event to be detected is set as an event that satisfies a condition regarding a change in the three-dimensional position of a specific part of the observed person (movement of planar position (particularly movement between planar areas) and / or a change in height, etc.). Moreover, a certain event to be detected is set as an event that satisfies a condition regarding the speed of change (movement speed, etc.) of the three-dimensional position of a specific part of the observed person.

[0128] Furthermore, each detection target event may be set as an event that satisfies conditions regarding any combination of "three-dimensional position", "change in three-dimensional position", and "speed of change in three-dimensional position".

[0129] The setting operation (step S31) using the setting screen 200 will be described below.

[0130] FIG. 19 is a diagram showing the setting screen 200.

[0131] The setting screen 200 includes a room designation area 201 , a person information registration button 202 , an area designation area 210 , a skeleton information display area 220 , a rule designation area 230 , a rule editing area 240 , an alarm level designation area 280 , a registration button 291 , and a cancel button 292 .

[0132] The room designation area 201 is an area for designating a room (living room) to be set. The operating user can designate a desired room (for example, "Room 101") from among a plurality of rooms ("Room 101", "Room 102", "Room 103", ...) displayed in a drop-down list. The operating user can then set one or more detection target events related to the designated room.

[0133] The person information registration button 202 is a button for registering information on a person (observed person) residing in a room. When the person information registration button 202 is pressed, a screen (not shown) for registering information on the observed person for each room is displayed as a pop-up. By using the information registration screen, a setting operation for setting information on the observed person (for example, height information, etc.) is accepted.

[0134] The area setting area 210 includes an area addition button 211 , an area deletion button 212 , an area selection area 213 , and a camera image display area 215 .

[0135] Here, the planar position (more specifically, a part or all of it) in the room (more specifically, within the detectable space) is divided into a plurality of areas (planar areas) R1, R2, ... by a setting operation using the area setting region 210. As will be described later, the conditions related to the planar position among the three-dimensional positions of each specific part of the observed person are set by specifying a desired area among the plurality of areas.

[0136] The camera image display area 215 is an area for displaying an image captured by the camera unit 20 regarding the space to be detected as an area designation image. For example, an image captured in a situation where the care recipient is not present is displayed. However, the present invention is not limited to this, and a schematic graphic image or the like obtained by processing the captured image may be displayed as the area designation image.

[0137] The add area button 211 is a button for adding an area, and the delete area button 212 is a button for deleting an area. The area selection area 213 is an area in which the area names ("R1", "R2", etc.) of one or more areas that have been generated are displayed. The operating user can select an area to be edited from among the areas displayed in the area selection area 213.

[0138] For example, when the area addition button 211 is pressed for the first time, a new name "R1" is added to the area selection area 213, and a hatched area indicating the area R1 is displayed in the camera image display area 215. The hatched area is displayed superimposed on the camera image (captured image). The operating user can set the position and size of the area R1 by appropriately shaping the position, size, etc. of the hatched area by operating the mouse, etc.

[0139] The height of each area from the floor surface (a constant value within the area) is set for each area. In detail, the height of each area is automatically set to the height of the three-dimensional position of the object corresponding to the position of the center of gravity (center) of the hatched area. For example, the height of area R1 placed on a bed is automatically set to the height (measured value) of the upper surface of the bed (e.g., 50 cm from the floor surface). However, without being limited to this, the height of area R1 may be specified by the operating user.

[0140] In addition, the boundary position (planar position) of each area is automatically converted to a coordinate position in the coordinate system Σ2 in the three-dimensional space fixed within the room and registered. For example, area R1 is registered as a rectangular area (with two points (0,100) and (80,280) as diagonal vertices) in a horizontal plane parallel to the floor.

[0141] However, without being limited to this, the coordinate values ​​of the diagonal vertices (two points) of each area having a rectangular region may be input by the operating user to specify the position and size of each area. For example, the coordinate values ​​(0,100,50) and (80,280,50) of the two points in the three-dimensional coordinate system Σ2 fixed to the room may be input. The Z coordinate value ("50") may or may not be specified.

[0142] The skeleton information display area 220 is an area for displaying an image showing skeleton information of a person (human). The skeleton information of a person includes information on a plurality of specific body parts of the person (chest, waist, shoulders, elbows, wrists, knees, ankles, etc.). In the skeleton information display area 220, each specific body part of a human is shown together with its designated number. Specifically, each body part of "nose", "chest", "right shoulder", "right elbow", "right wrist", "left shoulder", "left elbow", and "left wrist" is shown together with its respective body part numbers "B0" to "B7". In addition, each body part of "right waist", "right knee", "right ankle", "left waist", "left knee", and "left ankle" is shown together with its respective body part numbers "B8" to "B13". In addition, each body part of "right eye", "left eye", "right ear", and "left ear" is shown together with its respective body part numbers "B14" to "B17".

[0143] Such skeletal information (body part information) is used for operations such as designating a desired body part with a designated number (including numbers and / or letters, etc.) when creating rules (creating detection rules for events to be detected) as described below. The operating user can designate a desired specific part among multiple specific parts while referring to the image (image showing skeletal information) in the skeletal information display area 220. The image in the skeletal information display area 220 is an image for designating multiple specific parts of a person (an image displaying a list of multiple specific parts), and is also referred to as a "body part designation image" (or a "body part list display image").

[0144] For example, a desired specific part is specified by a designation number (designation number shown in association with a corresponding part in the part designation image) (such as "B1") corresponding to each specific part. Note that this is not limited to this, and the specific part (desired specific part) corresponding to the display part (such as a dot part corresponding to B1) of each specific part in the part designation image may be specified by clicking the mouse (or touching on the screen).

[0145] The rule specification area 230 includes an add rule button 231 , a delete rule button 232 , and a rule selection area 233 .

[0146] The add rule button 231 is a button for adding a detection rule for a detection target event (also referred to as a detection condition for the detection target event), and the delete rule button 232 is a button for deleting a detection rule. The rule selection area 233 is an area for displaying the names of one or more generated rules as options. The rule selection area 233 is an area for selecting a desired rule (from among multiple rules) as a rule to be edited. The content of the rule selected in the rule selection area 233 is specified (set) in detail in the rule editing area 240. When the add rule button 231 is pressed, a "new rule" is additionally displayed as an option in the rule selection area 233, as shown in FIG. 19.

[0147] The rule editing area 240 is an area for editing the details of the rule to be edited. By inputting appropriate content into various input fields in the rule editing area 240, the rule (detection rule (detection condition) to be edited) is specified (defined).

[0148] The rule editing area 240 includes a rule name input field 241 and a detailed condition specification field 250 .

[0149] An arbitrary rule name is input in the rule name input field 241. The "rule name" is also expressed as the name of the event to be detected. The name input in the rule name input field 241 (for example, "getting up (on the bed)") is determined as the rule name, and the name is displayed as an option in the rule selection area 233 (see FIG. 20).

[0150] The detailed condition specification field 250 includes a part specification field 251 and an action specification field 252 .

[0151] The part designation field 251 is an input field for designating a specific part to focus on. The operating user can designate a desired option (for example, "B1") from among multiple options (B0 to B17, etc.) in the first drop-down list in the part designation field 251. That is, the operating user can designate a desired specific part from among multiple specific parts. For example, if the operating user wants to designate "chest" as a specific part, he or she can designate the designated number B1 corresponding to "chest" by referring to the display contents in the skeleton information display area 220.

[0152] Furthermore, the operating user can specify two or more specific parts by pressing an "Add part" button in part specification field 251. For example, in response to pressing the Add part button, a drop-down list for specifying a second part and a drop-down list for specifying an AND / OR condition are additionally displayed (see FIG. 21). Then, the operating user can specify a second specific part in the drop-down list for specifying the second part, and specify the relationship between the two or more specific parts (AND condition / OR condition) in the drop-down list for specifying the AND / OR condition. Furthermore, a third or subsequent specific part can be specified in the same manner.

[0153] The action etc. specification field 252 is an input field for specifying the action or state etc. of the specific part specified in the part specification field 251. For example, a drop-down list in the action etc. specification field 252 displays multiple options such as "move" and "stay". The operating user can specify a desired option from the multiple options.

[0154] The detailed condition designation field 250 further includes an area designation field 253 , a height designation field 254 , a speed designation field 255 , and a period designation field 256 .

[0155] The operating user can specify detailed contents of a desired specification field by checking the check box at the top of the desired specification field among these specification fields 253 to 256. Note that the contents of the specification field of the check box that is not checked is invalidated.

[0156] The area designation field 253 is an input field for designating a detection area (planar position) related to the action or state of a specific part. The operating user can designate an area (e.g., "R1") in the first area designation drop-down list, and can also designate an associated condition (e.g., "inside") in the associated condition designation drop-down list related to the area (see FIG. 20). There are multiple options for the associated condition, such as "inside," "outside," "into," "outside," and "from."

[0157] Furthermore, when "FROM" is specified in this drop-down list for specifying an accompanying condition, a second drop-down list for specifying an area and a drop-down list for specifying an accompanying condition related to that area are additionally displayed (see FIG. 21). This drop-down list for specifying an accompanying condition provides options such as "TO." This makes it possible to specify that a specific part moves "from area R1 to area R2," for example.

[0158] The height specification field 254 is an input field for specifying the height related to the action or state of a specific body part. The first drop-down list in the height specification field 25 contains options indicating the height standard (such as "current location" and "floor surface"). Furthermore, in other input fields in the height specification field 25, it is possible to specify "30 cm or more" (more specifically, "30" cm "or more"), etc. Note that any numerical value can be specified, and it is also possible to specify "less than" the numerical value, rather than "more than" the numerical value. The same applies to the other fields.

[0159] Also, here, in the height specification field 254, the height in a certain state or the height after movement is specified, but this is not limited to this, and for example, the height before the movement (before the state transition) and the height after the movement (after the state transition) may be specified.

[0160] The speed designation field 255 is an input field for designating the speed related to the movement of the specific body part. Specifically, it is possible to designate that the movement speed of the specific body part is "1" m / s "or more" (see FIG. 24).

[0161] The period designation field 256 is an input field for designating a period related to the state (or action) of a specific body part. Specifically, it is possible to designate that the duration of the state (or the duration of the action) is a predetermined length or longer (see FIG. 25).

[0162] By using this detailed condition specification field 250, the operating user can set the event to be detected.

[0163] For example, as a detection target event having the name of "getting up (on bed)", an event as shown in Fig. 20 can be set. In Fig. 20, an event is set (also expressed as a rule or definition) that satisfies the condition that "within area R1, part B1 (chest) moves so that its height from the current position (current height) is 30 cm or more."

[0164] Furthermore, by pressing the condition addition button 242 in the rule editing area 240, it is possible to specify a complex condition consisting of two or more conditions (see Figs. 21 and 22, etc.). For example, when the condition addition button 242 is pressed after a first condition is specified in the detailed condition specification field 250 of Fig. 21, a detailed condition specification field 260 (see Fig. 22) for specifying a second condition is displayed. The detailed condition specification field 260 has the same configuration as the detailed condition specification field 250. Also, in Fig. 22, a button 243 and a drop-down list 244 are provided (in place of the rule name input field 241). The relationship between the first condition and the second condition (AND / OR, etc.) is input in the drop-down list 244. Also, in response to pressing the button 243, the detailed condition specification field 250 for the first condition is displayed again.

[0165] The alarm level designation area 280 is an area for designating multiple alarm levels ("Warning", "Caution", "Daily Action"). The operating user can designate a desired level from among the multiple alarm levels by selecting a desired option (for example, "Warning") from a drop-down list in the alarm level designation area 280. For example, "Daily Action" is designated for the detection target event of "Getting up (on the bed)", and "Caution" is designated for the detection target event of "Sitting on the edge of the bed". Also, "Warning" is designated for the detection target event of "Falling (on the floor)". This makes it possible to designate (change) the alarm level for each detection target event. In particular, it is also possible to change the alarm level for each detection target event for each facility (or for each room) in accordance with the needs of the site, etc.

[0166] The register button 291 is a button for registering (confirming) the content edited using the rule edit area 240, the detailed condition specification field 250, and the alarm level specification area 280, etc. When the register button 291 is pressed, the edited content is confirmed.

[0167] The cancel button 292 is a button for discarding the edited contents. When the cancel button 292 is pressed, the edited contents are discarded and the settings are restored to the settings before the start of editing.

[0168] By using such a setting screen 200, the above events E1 to E8 and the like can be set.

[0169] Fig. 20 is a diagram showing a setting screen for the detection target event E1 "getting up (on the bed)". The setting screen 200 in Fig. 20 specifies a condition that the height (height from the current position) of the observed person's chest (part B1) in the bed area R1 (planar area in which the bed is located) is 30 cm or more. When this condition is met, the event E1 (motion of the observed person's upper body getting up on the bed) (see Fig. 7) is detected.

[0170] The detection target event E1 in Fig. 20 is defined as an event that satisfies a condition regarding the three-dimensional position (planar area R1 and / or height) of a specific part B1 of the observed person. In particular, the detection target event E1 is defined as an event that satisfies a condition regarding the positional relationship between the specific part B1 of the observed person and a predetermined object (here, a bed) other than the observed person ("the specific part B1 is present within the bed area R1"). In addition, the detection target event E1 is defined as an event that also satisfies a condition regarding a change in the three-dimensional position (change in height) of the specific part of the observed person.

[0171] 21 and 22 are diagrams showing the setting screen for the detection target event E2 "sitting on the edge of bed". In FIG. 21, a condition (first condition) that "the ankles (parts B10 and B13) of the observed person move from the bed area R1 to the bedside area R2" is preset. Also, in FIG. 22, a condition (second condition) that "the waist (parts B8 and B11) of the observed person remains within the bed area R1" is specified. Furthermore, "AND" (with the previous page condition (first condition in FIG. 21)) is specified in the drop-down list 244. That is, it is also specified that the logical product (AND) condition of the first condition and the second condition is satisfied.

[0172] In this way, the condition is defined that the ankles (parts B10 and B13) of the observed person move from the bed area R1 to the bedside area R2 and the waist (parts B8 and B11) of the observed person remains within the bed area R1. When the condition is satisfied, the event E2 (the observed person is sitting at the edge of the bed) (see FIG. 8) is detected.

[0173] The detection target event E2 in Fig. 21 and Fig. 22 is defined as an event that satisfies a condition regarding a change in the three-dimensional position (movement from planar area R1 to planar area R2) of the specific parts B10, B13 (right ankle and left ankle) of the observed person. In particular, the detection target event E2 is defined as an event that satisfies a condition regarding a change in the positional relationship between the specific parts B10, B13 of the observed person and the bed. In detail, the condition is "the specific parts B10, B13 move from inside the bed area R1 to the bedside area R2 outside the bed area R1."

[0174] Furthermore, the detection target event E2 is defined as an event that satisfies a condition regarding the three-dimensional positions (planar area R1) of the specific parts B8 and B11 of the observed person. In particular, the detection target event E2 is defined as an event that satisfies a condition regarding the positional relationship between the specific parts B8 and B11 of the observed person and a predetermined object (bed) other than the observed person. In detail, the condition is that "the specific parts B8 and B11 are present within the bed area R1."

[0175] Fig. 23 is a diagram showing a setting screen for the detection target event E3 "leaving the bed". In the setting screen 200 of Fig. 23, a condition is specified that the whole body (all specific parts B0 to B17) of the observed person moves from the bed area R1 to another area R3 (area away from the bed area R1). When this condition is satisfied, the event E3 (the action of the observed person leaving the bed and moving to a place other than the bed) (see Fig. 9) is detected.

[0176] 23 is defined as an event that satisfies a condition regarding a change in the three-dimensional position of the specific parts B0 to B17 of the observed person (moving from planar area R1 to planar area R3). In particular, the detectable event E3 is defined as an event that satisfies a condition regarding a change in the positional relationship between the specific parts B0 to B17 of the observed person and the bed ("the specific parts B0 to B17 move from inside the bed area R1 to area R3 outside the bed area R1").

[0177] Fig. 24 is a diagram showing a setting screen for detection target event E4 "fall (on floor)". In the setting screen 200 of Fig. 24, a condition is specified that the chest (specific part B1) of the observed person enters a space of a predetermined height (50 cm) or less above the floor surface at a speed of 1 m / s or more outside the bed area R1. When this condition is satisfied, event E4 (motion of the observed person falling on the floor) (see Fig. 10) is detected.

[0178] The detection target event E4 in Fig. 24 is defined as an event that satisfies a condition regarding the three-dimensional position of the specific part B1 of the observed person ("outside the bed area R1"). The detection target event E4 is also defined as an event that satisfies a condition regarding the change in the three-dimensional position (height change) of the chest (specific part B1) of the observed person. Furthermore, the detection target event E4 is also defined as an event that satisfies a condition regarding the change speed (movement speed) of the three-dimensional position of the chest (specific part B1) of the observed person ("1 m / s or more").

[0179] In particular, a "fall" is an action in which the posture of the observed person (such as a care recipient) changes against his or her will. Therefore, the movement speed during a "fall" is usually above a certain level. In FIG. 24, such a characteristic is utilized to more appropriately set the detection target event. This makes it possible to appropriately distinguish between the action of the observed person (such as a care recipient) falling against his or her will (see FIG. 10) and the action of the observed person lying down slowly (at less than a certain speed) of his or her own will (not shown).

[0180] Fig. 25 is a diagram showing a setting screen for detection target event E5 "lying (on the floor)". In the setting screen 200 of Fig. 25, a condition is specified that the nose (specific part B0) and chest (specific part B1) of the observed person remain in a space of 50 cm or less from the floor surface outside the range of the bed area R1 for 5 seconds or more. When this condition is satisfied, event E5 (a state in which the observed person is lying on the floor (having a supine position) for a predetermined period or more) (see Fig. 11) is detected.

[0181] 25 is defined as an event that satisfies the conditions regarding the three-dimensional positions of the specific parts B0 and B1 of the observed person ("height is 50 cm or less from the floor surface" and "outside the bed area R1"). In addition, the detection target event E5 is defined as an event that also satisfies the condition regarding the continuity (non-change) of the three-dimensional positions of the specific parts B0 and B1 of the observed person ("5 seconds or more").

[0182] Fig. 26 is a diagram showing a setting screen for the detection target event E6 "whole body sliding off the bed". The setting screen 200 in Fig. 26 specifies a condition that the part B0 (nose) and part B8 (right hip) of the observed person move from the bed area R1 to the bedside area R2 at a speed of 1 m / s or more into a space below a predetermined height (50 cm) above the floor. When this condition is satisfied, the event E6 (the motion of the observed person's whole body sliding off the bed to the bedside) (see Fig. 12) is detected.

[0183] 26 is defined as an event that satisfies the conditions regarding the change in the three-dimensional position of the specific parts B0 and B8 of the observed person ("movement from bed area R1 to bedside area R2" and "change in height"). In addition, the detection target event E6 is defined as an event that satisfies the condition regarding the change speed (movement speed) of the three-dimensional position of the specific parts B0 and B8 of the observed person ("1 m / s or more").

[0184] Fig. 27 is a diagram showing a setting screen for the detection target event E7 "lower body slipping off the bed". The setting screen 200 in Fig. 27 specifies a condition that the right hip (site B8) and left hip (site B11) of the observed person move from the bed area R1 to the bedside area R2 at a speed of 1 m / s or more into a space below a predetermined height (50 cm) above the floor. When this condition is met, the event E7 (the motion of the observed person's lower body slipping off the bed to the bedside) (see Fig. 13) is detected.

[0185] Here, "whole body slippage" is defined by the movement of a specific part B0 (nose) of the upper body and a specific part B8 (right hip) of the lower body, whereas "lower body slippage" is defined by the movement of specific parts B8 and B11 (right hip and left hip) of the lower body. That is, "lower body slippage" differs from "whole body slippage" in that the movement of a specific part of the upper body is not a condition. Note that, based on this definition (setting), when a person's "whole body" actually slips off the bed, both "lower body slippage" and "whole body slippage" are detected. However, this is not limited to this, and the two may be detected separately by defining further conditions. For example, "lower body slippage" may be detected on the condition that the person's face (nose, etc.) is at a position above a predetermined height (for example, 50 cm) from the floor.

[0186] 28 and 29 are diagrams showing the setting screen for the detection target event E8 "one hand reaching out from the bed." In FIG. 28, a condition (first condition) is set that "one of the observed person's wrists (part B4 or part B7) moves from the bed area R1 to the bedside area R2." Also, in FIG. 29, a condition (second condition) is specified that "the observed person's waist (part B8 and part B11) remains within the bed area R1." Furthermore, "AND" (with the previous page condition (first condition in FIG. 28)) is specified in the drop-down list 244. That is, it is also specified that the logical product (AND) condition between the first condition and the second condition is satisfied.

[0187] In this way, the condition is defined that one wrist (site B4 or site B7) of the observed person moves from the bed area R1 to the bedside area R2 and the waist (site B8 and site B11) of the observed person remains within the bed area R1. When the condition is satisfied, the event E8 (one hand of the observed person is stretched out toward the bedside) (see FIG. 14) is detected.

[0188] 28 and 29 is defined as an event that satisfies a condition regarding a change in the three-dimensional position (movement from area R1 to area R2) of a specific part B4 or B7 (right wrist or left wrist) of the observed person. In particular, the detectable event E8 is defined as an event that satisfies a condition regarding a change in the positional relationship between the specific parts B4, B7 of the observed person and the bed. In detail, the condition is "the movement of the specific part B4 or B7 from within the bed area R1 to the bedside area R2 outside the bed area R1."

[0189] Furthermore, the detection target event E8 is defined as an event that satisfies a condition regarding the three-dimensional positions (planar area R1) of the specific parts B8 and B11 of the observed person. In particular, the detection target event E8 is defined as an event that satisfies a condition regarding the positional relationship between the specific parts B8 and B11 of the observed person and a predetermined object (bed) other than the observed person ("the specific parts B8 and B11 are present within the bed area R1").

[0190] As described above, it is possible to flexibly set (specify) each detection target event by using the setting operation using the setting screen 200, etc. Therefore, it is possible to flexibly respond to various user requests.

[0191] In detail, the user can easily set the detection target event itself (in other words, the detection rules (detection conditions) for the detection target event). For example, the user (facility staff, etc.) can easily customize and register the setting contents (part, action / state, three-dimensional position, speed, etc.) of each of the above events E1 to E8 according to the circumstances of each facility. In addition, the user can determine the events to be detected according to the circumstances of each facility, and appropriately register (set) the events to be detected as detection target events. Furthermore, when a new event to be detected increases, it is easy to additionally register the event as a detection target event.

[0192] In particular, the events to be detected can be specified by the state and / or movement of specific body parts of a person, and the specific body parts can be specified arbitrarily (specified appropriately according to needs, etc.). The combination of each specific body part and the content of the movement, etc. can also be specified arbitrarily. In short, it has high customization capabilities.

[0193] In particular, the detection target event is specified as an event that satisfies the conditions for one or more of the three elements of the "three-dimensional position" of a specific part of a person, "change in the three-dimensional position," and "speed of change of the three-dimensional position." Therefore, it is possible to specify events in real space (three-dimensional space) in a detailed and appropriate manner. Furthermore, since the detection target event can be specified as an event that satisfies the conditions for two or more of these three elements, it is possible to specify events in real space (three-dimensional space) in an even more detailed and appropriate manner. Furthermore, since the detection target event can be set (specified) including the speed of change of the three-dimensional position, it is possible to very appropriately set detection target events (events characterized by the speed of change), such as falls.

[0194] Furthermore, by using the "height" of the three-dimensional position of the observed person, large changes in the person's posture (such as large changes due to falling, etc.) can be easily detected appropriately.

[0195] Furthermore, the "planar position" of the three-dimensional position of the observed person may be specified by an area (R1, R2, R3, etc.) preset by the user. In detail, a planar area in real space is divided into a plurality of areas (R1, R2, R3, etc.) in advance based on a camera image (a photographed image of a room) based on the user's operation, etc. Then, an appropriate planar area may be specified from among the plurality of areas (R1, R2, R3, etc.) by the user to set a detection target event. This allows the user to create each planar area with a high degree of freedom, and to flexibly specify the planar area as the presence area, the origin area, and / or the destination area of ​​each specific part.

[0196] In addition, the name of the detection target event can be appropriately set to a common name for each facility, etc. Therefore, when introducing the above system, it is possible to prevent users (people providing care (caregivers), etc.) from feeling uncomfortable with the name of the detection target event.

[0197] Furthermore, an alert level (warning, caution, daily action) can be set for each detection target event. In other words, the terminal device 70 accepts an operation to set an alert level among a plurality of alert levels according to the detection target event. As a result, the detection target event is reported according to a level. In addition, the level can be flexibly set according to the circumstances of each facility.

[0198] The occurrence of each detection target event is notified in a different manner according to the notification level. For example, when an event of the "caution" level occurs, the name of the detected event (such as "sitting on the edge of the bed") is displayed in yellow and medium-sized characters on the observation screen 310 (more specifically, the character display area 312) (see FIG. 16) of the terminal device 70, 80. When an event of the "warning" level (a level higher than the "caution" level) occurs, the name of the detected event (such as "falling over") is displayed in red and large-sized characters on the observation screen 310 (see FIG. 17) of the terminal device 70, 80. In this case, an alarm sound is also output from the terminal device 70, 80. When an event of the "daily activity" level (a level lower than the "caution" level) occurs, the name of the detected event (such as "getting up") is displayed in black and small-sized characters on the observation screen 310 of the terminal device 70, 80.

[0199] Furthermore, one or more detection target events (detection conditions) are set for each room. In detail, first, the terminal device 70 accepts a setting operation for setting each detection target event related to a plurality of rooms for each room. Then, each detection device 10 provided for each room sets each detection target event for each room in accordance with the setting operation, and detects each detection target event based on the setting contents (setting contents for each room) set by the setting operation.

[0200] Therefore, detailed settings can be made according to the occupants of each room. For example, when the care level of the residents varies from room to room, it is possible to appropriately customize and set the detection target events (detection conditions) for each room (in other words, for each resident of each room).

[0201] Furthermore, when information on people residing in each room (such as the height of the observed person in each room) is registered in response to pressing the person information registration button 202 described above, it is possible to improve the detection accuracy as follows. Specifically, the detection device 10 may detect each detection target event in the room by excluding people who do not match the height information of the observed person in that room from the people to be detected.

[0202] More specifically, the detection device 10 measures the height of one or more subject persons in a captured image based on information (captured image, depth information, etc.) acquired by a three-dimensional camera. When the difference between the subject person's height (measured height) and the height of the observed person (pre-registered height) is equal to or greater than a predetermined value, the detection device 10 determines that the subject person is not the observed person and excludes the subject person from detection targets.

[0203] With this, when a person other than the person being observed (for example, a child whose height is shorter than the person being observed by a certain amount) is staying in the room to visit a sick relative or the like, even if the child (playing) falls, it will not be mistakenly determined that this is the person being observed having fallen.

[0204] Alternatively, when there are more people in a room than the number of people to be observed (a predetermined number (for example, 1 person)), an alert may not be issued even when a detection target event is detected. In a situation where a person (another person) other than the person to be observed is present in the room, it is often possible for that person (another person) to recognize the occurrence of a detection target event related to the person to be observed. In such a situation, it is possible to suppress an alert that is essentially unnecessary by considering the necessity for an alert to be lower than a predetermined level and not issuing an alert even when a detection target event is detected. It is also possible to suppress an increase in the load on the detection device 10 that accompanies the detection operation of multiple subject people.

[0205] In this embodiment, the setting operation is accepted by the management device 70, but is not limited to this. For example, the setting operation may be accepted by the mobile terminal device 80.

[0206] <7. Monitoring display using skeleton display> Next, the monitoring display of the observed person will be described.

[0207] FIG. 15 is a diagram showing a selection screen 300 for selecting a room to be displayed. A plurality of buttons 301 to 306 corresponding to a plurality of rooms (six in this example) are displayed on the selection screen 300. When a desired button is selected from the plurality of buttons (301 to 306), an observation screen 310 relating to the room corresponding to the desired button is displayed. For example, when the button 301 is pressed, an observation screen 310 (see FIG. 16 and the like) relating to "Room 101" corresponding to the button 301 is displayed. The selection screen 300 and the observation screen 310 are displayed on the display unit 75b of the management device 70, for example. However, the present invention is not limited to this, and the selection screen 300 and the observation screen 310 may be displayed on the display unit 85b (touch panel 87 and the like) of each mobile terminal device 80.

[0208] The display processing of these screens 300, 310 is controlled by cooperation between each terminal device 70 (or 80) and each detection device 10. More specifically, first, the terminal device 70 (or 80) sends a monitoring display request to the detection device 10 corresponding to the target room. In response to the monitoring display request, the detection device 10 generates display image data, etc. (display data for the observation screen 310 (including the observation image 330)) and transmits the display image data, etc. to the terminal device 70, etc. (step S12 in FIG. 5). The terminal device 70, etc. displays the observation screen 310 based on the received display data.

[0209] 16 and 17 are diagrams showing an observation screen 310 of an observed person in a certain room (here, "Room 101").

[0210] As shown in FIG. 16 etc., an observation screen 310 includes an image display area 311 for displaying an observation image 330 (observation video), and a character display area 312.

[0211] An observation image (video (moving image)) 330 of the observed person is displayed in real time in the video display area 311. As shown in Fig. 16 and other figures, the observation image 330 is an image that does not include a photographed image of the observed person itself but includes a skeleton image 325 of the observed person.

[0212] The skeleton image 325 and the observed image 330 are generated by the processing unit 30 based on the captured image 110 captured by the camera unit 20 .

[0213] FIG. 18 is a diagram for explaining the generation process of the observation image 330 and the like.

[0214] Specifically, the processing unit 30 of the detection device 10 first generates a skeletal image 325 of the observed person from the captured image 110 of the observed person (more specifically, the skeletal information 140 based on the captured image 110). As shown in Fig. 18 (and Fig. 6) and the like, the skeletal image 325 is an image in which, for example, each specific part of the observed person is represented by a "dot" and each skeletal line (connecting line) is represented by a "line segment."

[0215] Then, the processing unit 30 generates an observation image 330 of the observed person based on the skeleton image 325. The observation image 330 is generated, for example, by combining a background image 327 (also referred to as 327a) in which no person is present with the skeleton image 325. The skeleton image 325 is disposed in the background image 327 at the same position as the skeleton image 325 exists in the captured image 110.

[0216] The background image 327a in which no person is present is generated, for example, based on a captured image 110 (also referred to as 326) that was captured in advance and in which no person is present. In detail, when the detection device 10 determines that no person is present in the captured image 110 at a certain point in time (in detail, a captured image of the target room captured in advance), the detection device 10 extracts a still image from the captured image (moving image) 110. Then, the still image is generated as the background image 327a.

[0217] In this manner, an observed image 330 at a certain point in time is generated.

[0218] Thereafter, such an operation is repeated to sequentially generate the observed images 330 at each time point. In short, the observed images 330 (including the skeletal image 325) are generated in real time based on the captured images 110 and the like.

[0219] Note that the background image 327a may always be the same image. However, without being limited to this, for example, different background images 327a may be generated and used depending on the time of day (daytime / nighttime) or the like. In detail, during the daytime, a captured image 110 taken during the day and without a person may be used as the background image 327a. During the nighttime, a captured image 110 taken at night and without a person may be used as the background image 327a.

[0220] Furthermore, the background image 327 may be a fixed image 327b (a plain image or a three-dimensional schematic diagram of a room, etc.) that is prepared in advance.

[0221] Meanwhile, on a general monitoring screen (also referred to as a screen according to a comparative example), the image captured by the camera unit 20 itself is displayed, not the skeleton image 325. In detail, the image captured of a person (also referred to as a person to be observed) in the living room is always displayed as it is on the manager's screen, etc.

[0222] However, such monitoring display does not seem to give adequate consideration to the privacy of the person in question.

[0223] In contrast, in the monitoring display according to the above embodiment, a skeleton image 325 of the observed person is displayed as the observation image 330, rather than a captured image of the observed person itself. Therefore, it is possible to appropriately consider the privacy of the observed person. In particular, according to the skeleton information (skeleton image 325 (observation image 330)), information on the observed person's face is lost, and the amount of information is reduced to an extent that the observed person's body type (fat / thin) and gestures, etc., are almost unrecognizable. Therefore, it is possible to appropriately consider the privacy of the observed person. Therefore, it is also possible to reduce the psychological resistance of the person being watched (such as the person being cared for).

[0224] On the other hand, a caregiver or the like can visually recognize the skeletal image of the person being observed to adequately grasp the posture and the like of the person being observed.

[0225] <8. Examples of modifications to displays> In the above-described monitoring display, when a predetermined condition is met, the captured image 110 relating to the observed person may be displayed instead of the observed image 330 or together with the observed image 330. In short, the observed image 330 (skeleton image 325) may be switched to the captured image 110 as necessary.

[0226] <Prescribed Condition 1> An example of a predetermined condition for performing such switching is when a predetermined detection target event is detected.

[0227] For example, in a predetermined setting screen (200, etc.), an option (option regarding the image to be displayed) regarding whether a "skeleton image" or a "photographed image" is to be displayed when each detectable event occurs (when detected) is provided. The option is preferably provided for each detectable event. Then, the type of image set by the option ("skeleton image" / "photographed image") may be displayed when each detectable event occurs. For example, if "photographed image" is set as the image to be displayed when a certain detectable event (such as "fall") occurs, when the detectable event is detected, the photographed image 110 is displayed for monitoring. On the other hand, when the detectable event is not detected, the skeleton image 325 (observation image 330) is displayed.

[0228] According to this, when a highly dangerous event such as a "fall" occurs, the user can visually check the actual captured image 110 and confirm the situation in more detail. As a result, the user can more appropriately grasp the situation and take appropriate measures.

[0229] In this way, the skeleton image 325 (observed image 330) is displayed under normal circumstances, and the captured image 110 is displayed when a predetermined detection target event occurs. This allows the user (caregiver, etc.) to check the actual captured image in a predetermined situation while taking privacy into consideration. In other words, the observer (caregiver, etc.) can check the captured image related to the observed person as necessary. Furthermore, the observer (caregiver, etc.) can perform appropriate processing.

[0230] Furthermore, by setting whether or not to display the captured image for each detection target event, it is possible to realize a detailed response.

[0231] <Prescribed Condition 2> Further, an example of a predetermined condition under which the above-mentioned switching or the like should be performed is that the alarm level set for the detection target event among a plurality of alarm levels has an importance level equal to or higher than a predetermined level.

[0232] For example, a relationship between the image to be displayed (observation image 330) and multiple alarm levels may be set on a setting screen for alarms (see FIG. 20, etc.). Examples of the multiple alarm levels include "warning", "caution", and "daily activity". "Warning" is an alarm level with a higher level of importance than "caution", and "caution" is an alarm level with a higher level of importance than "daily activity". Note that one of the multiple alarm levels is assigned to each event to be detected.

[0233] For example, on the setting screen for alarm, a predetermined level (for example, "warning" or higher) at which switching from the skeleton image 325 (observation image 330) to the captured image 110 or the like should be performed is set among a plurality of alarm levels.

[0234] Based on such settings, when a detectable event is not detected, a skeleton image 325 (observation image 330) is displayed, whereas when a detectable event is detected, an image corresponding to the alarm level assigned to the detectable event is displayed.

[0235] Specifically, when a detectable event occurs, it is determined whether or not the alarm level of the detectable event has an importance equal to or greater than a predetermined level ("Warning"). If the alarm level of the detectable event does not have an importance equal to or greater than the predetermined level ("Warning"), the skeleton image 325 (observation image 330) continues to be displayed. On the other hand, if the alarm level of the detectable event has an importance equal to or greater than the predetermined level ("Warning"), the image to be displayed is changed from the skeleton image 325 (observation image 330) to the captured image 110.

[0236] In this way, the skeleton image 325 (observation image 330) is displayed under normal circumstances, and the captured image 110 is displayed when a detection target event with a predetermined level of importance or higher occurs. This makes it possible to check the actual captured image and perform appropriate processing as necessary, while taking privacy into consideration. In addition, since it is possible to set whether or not to display the captured image for each detection target event, it is possible to realize a detailed response.

[0237] In particular, depending on the importance of the event to be detected that has occurred, it is possible to change whether to display the skeleton image 325 (observed image 330) or the captured image 110. In particular, when an event that poses a high risk such as "falling" (an event that has an importance level equal to or higher than a predetermined level) has occurred, the user can visually check the actual captured image 110 and confirm the situation in more detail. As a result, it is possible to more appropriately grasp the situation and take appropriate measures.

[0238] In addition, according to this aspect, the user does not need to set a display target image for each detection target event, which reduces the effort required for setting, compared to when the user sets both an alarm level and a display target image for each detection target event.

[0239] <Other modification examples> In addition, in each of the above modified examples, when a predetermined condition is met, a photographed image of the observed person is displayed instead of the observed image 330 (skeleton image 325), but this is not limiting. When a predetermined condition is met, the photographed image 110 of the observed person may be displayed together with the observed image 330 (skeleton image 325).

[0240] In addition, in the above-mentioned embodiments, moving images are displayed as the observation image 330 and the skeleton image 325 on the terminal devices 70 and 80, but this is not limited to this, and still images may be displayed as the observation image 330 and the skeleton image 325 on the terminal devices 70 and 80.

[0241] In the above embodiment, in response to a monitoring display request sent from the terminal device 70 (or 80) to the detection device 10, the transfer of image data to the terminal device 70 or the like is started, and the display of an image (skeleton image, etc.) is started, but the present invention is not limited to this. For example, in response to the detection of each detection target event by the detection device 10, the detection of the detection target event may be notified to each terminal device 70, 80, and the display of an image (skeleton image, etc.) may be automatically started by the management device 70 or the like. The displayed image (skeleton image 325 and observed image 330, etc.) at this time may be a moving image or a still image.

[0242] In the above embodiment, the skeleton image 325 (observation image 330) is displayed during monitoring display, but the present invention is not limited to this. For example, when the observation image 330 (log image) stored in the storage unit 32 is subsequently played back, the skeleton image 325 may be displayed. Alternatively, when the recorded photographed image 110 (recorded image) is subsequently played back, the skeleton image 325 based on the photographed image 110 may be generated and displayed.

[0243] In addition, although the detection device 10 here executes the generation process (in other words, display control process) of the skeleton image 325 and the observation image 330, the present invention is not limited to this. For example, the terminal device 70 (, 80) may execute the generation process of the skeleton image 325 and / or the observation image 330. In detail, the captured image 110 and the depth information 120 may be transmitted from the detection device 10 to the terminal device 70, etc., and the skeleton image 325, etc. may be generated in the destination terminal device 70, etc. However, from the viewpoint of improving security or reducing the amount of data transmission, it is preferable that the detection device 10 on the transmitting side (rather than the management device 70 on the receiving side) executes the generation process.

[0244] In this manner, the detection device 10 may perform the display control process, or the terminal devices 70, 80 may perform the display control process. That is, the detection device 10 may operate as a display control device, or the terminal devices 70, 80 may operate as a display control device. In other words, the display control device may be provided as a device separate from the display device (70, 80), or may be provided integrally with the display device (70, 80).

[0245] <9. Reference heights by area, etc.> In the above, event E4 "falling on the floor" (see Figure 10) is shown as an example of one of the detection target events. In the detection target event E4, the condition for the "height" of the "chest" (part B1) of the observed person is that the height from the floor surface changes from a value of 50 cm or more to a value of 50 cm or less.

[0246] However, "falls" can occur not only on the floor but also on beds. If the same criteria as for falls on the floor are used to determine falls on a bed, it will not be possible to make an appropriate determination. Specifically, "chest height being 50 cm or more from the floor" is not always normal, so a different criterion must be used when falling on a bed.

[0247] Therefore, as another detection target event similar to the event E4, the event E12 "falling on the bed (the observed person falls on the bed)" (see FIG. 31) may be specified, and the detection target event E12 may be detected depending on the establishment of the following condition. The condition is, for example, that the height of the observed person's "chest" (part B1) from the "top surface of the bed" in the "bed area R1" changes from a value of 50 cm or more to a value of 50 cm or less, and the moving speed of the specific part B1 at that time is "1 m / s or more."

[0248] However, without being limited to this, the event E13 “fall” (an event combining the two events E4 and E12) may be detected without distinguishing between a fall on the floor and a fall on the bed.

[0249] The detection target event E13 is detected according to the establishment of the following condition. The condition is, for example, that the "height from the reference plane" of the "chest" (part B1) of the observed person changes from a value of 50 cm or more to a value of 50 cm or less, and the moving speed of the specific part B1 at that time is "1 m / s or more." In other words, the conditions for the two events E4 and E12 may both be specified by the height relative to the "reference plane."

[0250] Specifically, first, the detection device 10 predefines the height of the "reference plane" for each area (planar area) (before the processing in FIG. 5).

[0251] In detail, the detection device 10 divides a planar position (part or all) in a room (specifically, in the detectable space) into a plurality of areas R1, R2, ... in response to a setting operation by the user, etc. In other words, a plurality of planar areas R1, R2, ... are provided in the room. Then, the detection device 10 sets a "reference plane height" for each of the plurality of areas R1, R2, .... For example, "45 cm" is set as the reference plane height for the bed area R1, "0 cm" for the bedside area R2, and "0 cm" for the other area R3. In detail, the detection device 10 may set the reference plane height for each area in response to an operation input by the user, etc.

[0252] The height of the reference plane of each planar area (R1, R2, R3, etc.) may be set automatically. Specifically, the height of the reference plane may be automatically set to the height in three-dimensional space of an object (floor, bed, etc.) corresponding to a representative position of each planar area based on information from the camera unit 20 (three-dimensional camera). For example, the height of the reference plane of planar area R1 on which a bed is placed is automatically set to the height of the top surface of the bed (measurement value by a three-dimensional camera, etc.) (45 cm from the floor surface, etc.). In addition, the height of the reference plane of planar area R2 on which a bed is not placed is automatically set to the height of the floor surface (0 cm from the floor surface, etc.).

[0253] Then, the "reference height" for each area is determined based on the height of the "reference surface" for each area. For example, the reference height of the flat area (bed area) R1 is "95 cm", which is the sum of "50 cm" (height from the reference surface) and the height of the reference surface (top surface of the bed) set for the flat area R1, "45 cm". On the other hand, the reference height of the flat areas R2, R3, etc. (non-bed areas (floor surface areas)) is "50 cm", which is the sum of "50 cm" (height from the reference surface) and the height of the reference surface (floor surface) set for the flat area R2, "0 cm".

[0254] As described below, when the presence area of ​​the observed person is determined, the reference height of the presence area is calculated based on the "reference height" for each area. In other words, a "reference height" is set for each presence area. For example, if the bed area (area on the bed) R1 is the presence area of ​​the observed person, the reference height of the presence area is "95 cm." On the other hand, if the area on the floor (areas R2, R3, etc. other than the bed area R1) is the presence area of ​​the observed person, the reference height of the presence area is "50 cm." The reference height for each presence area differs depending on whether the presence area is on the bed or on the floor, etc.

[0255] The detection target event E13 may be set, for example, according to a setting operation using the setting screen 200 of Fig. 32. Fig. 32 is a diagram showing the setting screen 200 for the detection target event E13 "fall". The setting screen 200 of Fig. 32 (particularly, the detailed condition specification field 250) specifies a condition that the chest (specific part B1) of the observed person enters a space at a predetermined height (50 cm) or less from the "reference plane" at a speed of 1 m / s or more. Compared with the setting screen 200 (Fig. 24) for the detection target event E4, it differs in particular in that the planar area of ​​the detection target is not specified and that the height from the "reference plane" is specified.

[0256] Thereafter, when the process of FIG. 5 is started, in steps S11 to S15 (particularly S11 and S13), the detection device 10 determines the presence area of ​​the observed person, and obtains (sets) a reference height for the presence area according to the presence area of ​​the observed person. Then, the detection device 10 determines the condition regarding the height of each specific part based on the reference height for each presence area. Furthermore, the detection device 10 detects the detection target event based on these determination results. Note that the presence area of ​​the observed person may be determined based on the presence position (planar position) of a specific part (chest, waist, etc.) of the observed person.

[0257] In detail, the detection device 10 detects the detection target event based on the result of comparison between the reference height for each existence area and the height of the specific portion.

[0258] For example, when the observed person is on the bed (within the bed area R1), the detection target event E13 is detected based on the result of comparing the reference height of the bed area R1 ("95 cm") with the height of the observed person's specific part B1 ("chest") In detail, the detection target event E13 is detected on the condition that the height of the observed person's "chest" (part B1) from the floor surface changes from a value of 95 cm or more (reference surface height "45 cm" + "50 cm") to a value of 95 cm or less.

[0259] On the other hand, when the observed person is on the floor (for example, area R3), the detection target event E13 is detected based on the result of comparing the reference height of the area R3 ("50 cm") with the height of the observed person's specific part B1 ("chest") In detail, the detection target event E13 is detected on the condition that the height of the observed person's "chest" (part B1) from the floor surface changes from a value of 50 cm or more (reference surface height "0 cm" + "50 cm") to a value of 50 cm or less.

[0260] In this way, the detection target event E13 is detected based on the reference height for each existence area (in other words, the reference height set for each existence area). Note that it is preferable to detect the detection target event E13 by adding the above-mentioned condition related to the moving speed (such as a moving speed of "1 m / s or more") in addition to such a condition related to "height".

[0261] As described above, the detection device 10 (controller 31) may determine the presence area of ​​the observed person, and may determine the condition regarding the height of the specific part based on the reference height for each presence area. In detail, a different reference height may be set for each presence area of ​​the observed person, and the detection target event may be detected based on the comparison result between the reference height and the height of the specific part. In this way, even if the normal range of the height of the specific part differs for each presence area, an appropriate detection process may be performed based on an appropriate reference height for each presence area of ​​the observed person. Therefore, it is possible to appropriately detect the detection target event compared to the case where the detection process is performed based on a single reference height (a reference height that does not depend on the presence area). In addition, it is not necessary to separate and define the event of "falling" as multiple detection target events (E4, E12, etc.) for each area, and it is possible to unify the multiple detection target events (E4, E12, etc.) as one detection target event E13 ("falling").

[0262] <10.Other> Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described contents.

[0263] For example, the specific parts of the observed person are not limited to the specific parts mentioned above, but may be the person's head, neck, clavicle, etc.

[0264] In the above embodiment, the detection device 10 (particularly the processing unit 30) executes the detection process for each detection target event, but is not limited to this. For example, the management device 70 may execute the detection process for each detection target event.

[0265] In the above embodiment, the processing unit 30 (controller 31) generates the skeleton information 140 and the three-dimensional position information 150 of each specific part, but is not limited to this. For example, the management device 70 (controller 71) may generate the three-dimensional position information 150 of each specific part. Furthermore, the management device 70 (controller 71) may also generate the skeleton information 140.

[0266] In addition, in the above-described embodiments, the detection device 10 and the management device 70 are configured as separate devices, but the present invention is not limited to this. For example, the detection device 10 may be configured as a device that also executes the processes of the management device 70 (such as the process of accepting setting operations).

[0267] Further, the display unit that displays the observation image 330 may be provided in the terminal device 70 (, 80) as in the above-described embodiments, but is not limited to this. For example, the display unit may be provided in the detection device 10. [Explanation of symbols]

[0268] 1. Detection System 10. Detection Device 20 Camera unit 30 Processing Unit 70 Management device 80 Portable terminal device 110 Images 120 Depth information 140 Skeleton Information 150 3D location information 200 Setting screen 202 Person information registration button 210 Area setting region 220 Skeleton information display area 230 Rule Specification Area 240 Rules Editing Area 250 Detailed condition specification field 280 Alarm level designated area 300 Selection screen 310 Observation Screen 325 Skeleton Images 327 Background Image 330 Observation images

Claims

1. A terminal device that accepts a setting operation of a detection target event related to an observed person; a detection device that detects the detection target event based on the setting contents by the setting operation; Equipped with the terminal device accepts a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The detection system is characterized in that the detection device detects the detection target event by excluding people who do not match the height information from people to be detected.

2. A detection method for detecting a detection target event related to an observed person, comprising: a) receiving an operation for setting a detection target event related to an observed person; b) detecting the detection target event based on the setting contents by the setting operation; Equipped with The step a) includes a step of receiving a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The detection method, wherein the step b) comprises a step of detecting the detection target event by excluding a person who does not match the height information from detection target persons.

3. a) receiving an operation for setting a detection target event related to an observed person; b) detecting the detection target event based on the setting contents by the setting operation; A program for causing a computer to execute the above, The step a) includes a step of receiving a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The program, wherein the step b) comprises a step of excluding a person who does not match the height information from persons to be detected, and detecting the detection target event.

4. A reception unit that receives a setting operation of a detection target event related to an observed person; a control unit that detects the detection target event based on the setting contents by the setting operation; Equipped with the reception unit receives a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The control unit detects the detection target event by excluding people who do not match the height information from people to be detected.

5. A reception unit that receives a setting operation of a detection target event related to an observed person; a control unit that transmits setting information based on the setting operation to a detection device that detects the detection target event; Equipped with the reception unit receives a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The control unit transmits the setting information including the height information in order to have the detection device detect the detection target event after excluding people who do not match the height information from the detection target people.

6. a) receiving an operation for setting a detection target event related to an observed person; b) transmitting setting information based on the setting operation to a detection device that detects the detection target event; A program for causing a computer to execute the above, The step a) includes a step of receiving a setting operation for setting each detection target event related to a plurality of rooms for each room, the setting operation including an operation for setting height information of an observed person for each room; The program, wherein step b) includes a step of transmitting the setting information including the height information in order to have the detection device detect the detection target event after excluding persons who do not match the height information from the persons to be detected.

Images