Monitoring system

Abstract

To provide a monitoring system, a monitoring device, a monitoring method, and a monitoring program that make it easy to accurately grasp the situation within a monitored area while protecting the privacy of people in the monitored area.SOLUTION: A monitoring system includes: an imaging unit that captures an image of a monitoring target area; a skeletal information generation unit that generates skeletal information indicating a skeletal structure of a subject who is a person present in the monitoring target area, on the basis of a first image that is a captured image acquired by the imaging unit; an abstraction unit that generates a second image by performing abstraction processing on the first image; an image generation unit that generates a third image 13 that is an image in which the second image and the skeletal information are superimposed; and a display unit that can display the third image 13.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to a monitoring system Mu .

Background Art

[0002] Patent Document 1 describes a monitoring terminal that monitors a target person (referred to as a "monitoring target" in Patent Document 1) who is a person present in a monitoring target area. This monitoring terminal generates a notification image by superimposing information indicating the skeleton of the target person on a background image captured in advance. That is, the notification image does not include an image of the target person himself / herself. Thereby, the privacy of the target person is protected.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the monitoring terminal described in Patent Document 1, after the background image is captured, if a change occurs in the monitoring target area, for example, furniture within the monitoring target area is moved, the part of the notification image other than the skeleton of the target person will become different from the current situation. Thereby, a situation where the situation within the monitoring target area (for example, the positional relationship between the target person and the furniture) cannot be accurately grasped is assumed.

[0005] An object of the present invention is to provide a monitoring system, a monitoring device, a monitoring method, and a monitoring program that can accurately grasp the situation within the monitoring target area while protecting the privacy of a person present in the monitoring target area.

Means for Solving the Problems

[0006] The monitoring system according to the present invention is characterized by comprising: an imaging unit that images a target area to be monitored; a skeleton information generation unit that generates skeleton information indicating the skeleton of a target person present in the target area based on a first image, which is an image acquired by the imaging unit; an abstraction unit that generates a second image by performing an abstraction process on the first image to such an extent that the individual of the target person cannot be identified; an image generation unit that generates a third image, which is an image in which the second image and the skeleton information are superimposed; and a display unit capable of displaying the third image. The display unit is capable of displaying the first image and includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination by the image determination unit, either the first image or the third image is displayed on the display unit. The display unit also includes a second storage unit that stores authorization information, which is information indicating the authority of a monitor, who is a person who monitors the subject by looking at the display unit, regarding the content displayed on the display unit. Based on the authorization information stored in the second storage unit, the image determination unit determines which of the first image and the third image is displayed on the display unit. It is the matter.

[0007]

[0008]

[0009]

[0010] According to the present invention, a third image is generated by superimposing a second image, which is created by applying an abstraction process to a first image, with skeletal information. In this third image, the external details of the subject's face and body are obscured by the abstraction process. Therefore, the subject's privacy can be protected.

[0011] Furthermore, if a change occurs within the monitored area, for example, due to the movement of furniture within the monitored area, the third image, generated based on the first image acquired after the change, will show the situation within the monitored area after the change. This prevents the third image from being inaccurate. In addition, the skeletal information included in the third image allows for the determination of the subject's position and posture. Therefore, according to the present invention, it is possible to accurately grasp the situation within the monitored area (for example, the positional relationship between the subject and furniture).

[0012] In other words, according to the present invention, it is possible to accurately grasp the situation within a monitored area while protecting the privacy of people present in that area. Furthermore, this configuration allows the display to show either the first or third image, depending on the monitor's authority. This enables a system where, for example, if the monitored area is a resident's room in a nursing home and the monitor is a staff member of the nursing home, only a limited number of staff members (e.g., administrators) can view the first image.

[0013]

[0014]

[0015] Furthermore, in the present invention ,before The display unit is equipped with a first storage unit that stores intention information, which is information indicating the intention of the subject regarding the content to be displayed by the display unit. Preferably, the image determination unit determines which of the first image and the third image will be displayed on the display unit based on the intention information stored in the first storage unit.

[0016] This configuration allows the display unit to show either the first or third image, depending on the situation and necessity. In particular, this configuration allows the display unit to show either the first or third image, depending on the subject's wishes.

[0017] Furthermore, in the present invention ,before Preferably, the system includes a danger determination unit that determines whether the subject is in a dangerous state, and if the subject is determined to be in a dangerous state, the image determination unit decides to display the first image on the display unit, and if the subject is not determined to be in a dangerous state, the image determination unit decides to display the third image on the display unit.

[0018] In this configuration, the third image is displayed on the display unit during normal times, and the first image is displayed on the display unit when the subject is in a dangerous situation (for example, when the subject falls). Therefore, while protecting the subject's privacy during normal times, the first image makes it easier to understand the subject's condition in detail when the subject is in a dangerous situation.

[0019]

[0020]

[0021] Another feature of the monitoring system according to the present invention is that it comprises: an imaging unit that images a target area; a skeleton information generation unit that generates skeleton information indicating the skeleton of a target person present in the target area based on a first image which is an image acquired by the imaging unit; an abstraction unit that generates a second image by performing an abstraction process on the first image to such an extent that the individual of the target person cannot be identified; an image generation unit that generates a third image which is an image in which the second image and the skeleton information are superimposed; and a display unit capable of displaying the third image.The display unit can display the first image and includes an image determination unit that determines which of the first image and the third image is to be displayed on the display unit. According to the determination by the image determination unit, one of the first image and the third image is displayed on the display unit. The image determination unit determines which of the first image and the third image is to be displayed on the display unit based on the gender of a monitor who monitors the subject by looking at the display unit. It is 。

[0022] According to the present invention, a third image is generated by superimposing a second image, which is created by applying an abstraction process to a first image, with skeletal information. In this third image, the external details of the subject's face and body are obscured by the abstraction process. Therefore, the subject's privacy can be protected. Furthermore, if a change occurs within the monitored area, for example, due to the movement of furniture within the monitored area, the third image, generated based on the first image acquired after the change, will show the situation within the monitored area after the change. This prevents the third image from being inaccurate. In addition, the skeletal information included in the third image allows for the determination of the subject's position and posture. Therefore, according to the present invention, it is possible to accurately grasp the situation within the monitored area (for example, the positional relationship between the subject and furniture). In other words, according to the present invention, it is possible to accurately grasp the situation within a monitored area while protecting the privacy of people present in that area. Also, According to this configuration, it is possible to display an appropriate one of the first image and the third image on the display unit according to the gender of the monitor. Thereby, for example, when the monitored area is a room in an elderly care facility and the monitor is a staff member of the elderly care facility, a system can be realized in which only staff members of the same gender as the resident can view the first image.

[0023] Another feature of the monitoring system according to the present invention is that it comprises: an imaging unit that images a target area; a skeleton information generation unit that generates skeleton information indicating the skeleton of a target person present in the target area based on a first image which is an image acquired by the imaging unit; an abstraction unit that generates a second image by performing an abstraction process on the first image to such an extent that the individual of the target person cannot be identified; an image generation unit that generates a third image which is an image in which the second image and the skeleton information are superimposed; and a display unit capable of displaying the third image. The display unit can display the first image and includes an image determination unit that determines which of the first image and the third image is to be displayed on the display unit. According to the determination by the image determination unit, one of the first image and the third image is displayed on the display unit. The image determination unit determines that the third image is to be displayed on the display unit during a preset time period, and determines that the first image is to be displayed on the display unit outside of that time period. It is 。

[0024] According to the present invention, a third image is generated by superimposing a second image, which is created by applying an abstraction process to a first image, with skeletal information. In this third image, the external details of the subject's face and body are obscured by the abstraction process. Therefore, the subject's privacy can be protected. Furthermore, if a change occurs within the monitored area, for example, due to the movement of furniture within the monitored area, the third image, generated based on the first image acquired after the change, will show the situation within the monitored area after the change. This prevents the third image from being inaccurate. In addition, the skeletal information included in the third image allows for the determination of the subject's position and posture. Therefore, according to the present invention, it is possible to accurately grasp the situation within the monitored area (for example, the positional relationship between the subject and furniture). In other words, according to the present invention, it is possible to accurately grasp the situation within a monitored area while protecting the privacy of people present in that area. Also, According to this configuration, a system can be realized in which the third image is displayed on the display unit during a time period when consideration for privacy is required, and the first image is displayed on the display unit during other time periods.

[0025] Another feature of the monitoring system according to the present invention is that it comprises: an imaging unit that images a target area; a skeleton information generation unit that generates skeleton information indicating the skeleton of a target person present in the target area based on a first image which is an image acquired by the imaging unit; an abstraction unit that generates a second image by performing an abstraction process on the first image to such an extent that the individual of the target person cannot be identified; an image generation unit that generates a third image which is an image in which the second image and the skeleton information are superimposed; and a display unit capable of displaying the third image. The display unit is capable of displaying the first image and includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination by the image determination unit, either the first image or the third image is displayed on the display unit and an identification unit is included to identify the subject. The image determination unit determines which of the first image and the third image is displayed on the display unit based on the identification result by the identification unit. It is .

[0026] According to the present invention, a third image is generated by superimposing a second image, which is created by applying an abstraction process to a first image, with skeletal information. In this third image, the external details of the subject's face and body are obscured by the abstraction process. Therefore, the subject's privacy can be protected. Furthermore, if a change occurs within the monitored area, for example, due to the movement of furniture within the monitored area, the third image, generated based on the first image acquired after the change, will show the situation within the monitored area after the change. This prevents the third image from being inaccurate. In addition, the skeletal information included in the third image allows for the determination of the subject's position and posture. Therefore, according to the present invention, it is possible to accurately grasp the situation within the monitored area (for example, the positional relationship between the subject and furniture). In other words, according to the present invention, it is possible to accurately grasp the situation within a monitored area while protecting the privacy of people present in that area. Also, This configuration allows the display to show either the first or third image, depending on who the subject is. For example, if the subject is someone whose privacy needs to be considered, the third image will be displayed on the display, and if the subject is someone whose privacy does not need to be considered, the first image will be displayed on the display. More specifically, if the area to be monitored is a room in a nursing home and the monitor is a staff member of the nursing home, the system can be designed so that if the person in the monitored area is a resident, the third image will be displayed on the display, and if the person in the monitored area is a staff member, the first image will be displayed on the display.

[0027] Furthermore, in the present invention, it is preferable that the abstraction unit is configured to generate the second image by applying abstraction processing to only a part of the first image.

[0028] This configuration allows for the abstraction of only the parts of the captured image that require consideration of privacy. This makes it easier to balance the protection of the subject's privacy with a detailed understanding of their condition.

[0029] Furthermore, in the present invention, it is preferable that the abstraction unit determines the portion of the first image to be abstracted based on the skeletal information.

[0030] This configuration allows for the abstraction process to be performed on specific parts of the image based on skeletal information. For example, it enables the abstraction of only the parts of the captured image that require privacy considerations, such as the subject's body and clothing. In other words, this configuration makes it easier to balance the protection of the subject's privacy with a detailed understanding of their condition.

[0031] Another feature of the monitoring system according to the present invention is that it comprises: an imaging unit that images a target area; a skeleton information generation unit that generates skeleton information indicating the skeleton of a target person present in the target area based on a first image which is an image acquired by the imaging unit; an abstraction unit that generates a second image by performing an abstraction process on the first image to such an extent that the individual of the target person cannot be identified; an image generation unit that generates a third image which is an image in which the second image and the skeleton information are superimposed; and a display unit capable of displaying the third image. The system includes a body position determination unit that determines the body position of the subject based on the skeletal information, and a display determination unit that determines the display manner of the skeletal information in the third image based on the determination result by the body position determination unit. It is .

[0032] According to the present invention, a third image is generated by superimposing a second image, which is created by applying an abstraction process to a first image, with skeletal information. In this third image, the external details of the subject's face and body are obscured by the abstraction process. Therefore, the subject's privacy can be protected. Furthermore, if a change occurs within the monitored area, for example, due to the movement of furniture within the monitored area, the third image, generated based on the first image acquired after the change, will show the situation within the monitored area after the change. This prevents the third image from being inaccurate. In addition, the skeletal information included in the third image allows for the determination of the subject's position and posture. Therefore, according to the present invention, it is possible to accurately grasp the situation within the monitored area (for example, the positional relationship between the subject and furniture). In other words, according to the present invention, it is possible to accurately grasp the situation within a monitored area while protecting the privacy of people present in that area. Also, This configuration allows for a system where the display of skeletal information changes according to the subject's body position. This makes it easy for the observer to understand the subject's body position. [Brief explanation of the drawing]

[0033] [Figure 1] This diagram shows an overview of the monitoring system. [Figure 2] This is a block diagram of the monitoring system. [Figure 3] This is a diagram showing an example of the first image. [Figure 4] This is a diagram showing an example of the second image. [Figure 5] This figure shows an example of skeletal information. [Figure 6] This is a diagram showing an example of the third image. [Figure 7] This is a flowchart of the image decision flow. [Modes for carrying out the invention]

[0034] Embodiments for carrying out the present invention will be described with reference to the drawings.

[0035] [Overall configuration of the monitoring system] As shown in Figure 1, the monitoring system A in this embodiment comprises a monitoring device 1 and a display unit 2. The monitoring device 1 and the display unit 2 are connected to each other via a predetermined network.

[0036] The monitoring device 1 is installed in the area to be monitored B. The monitoring device 1 monitors the subject D, who is a person present in the area to be monitored B. An image showing the situation inside the area to be monitored B is transmitted from the monitoring device 1 to the display unit 2. The user E of the display unit 2 (corresponding to the "monitoring person" in this invention) can understand the situation inside the area to be monitored B, even while outside the area to be monitored B, by viewing the image displayed on the display unit 2.

[0037] In this embodiment, the monitored area B is a resident's room in the elderly care facility, user E is a staff member of the elderly care facility, and display unit 2 is an information terminal (smartphone, tablet, etc.) carried by the staff member. Also, in the example shown in Figure 1, subject D is a resident of the elderly care facility, and user E, who is a staff member, is located in a room other than the resident's room (for example, a waiting room). Note that subject D is not limited to a resident, but may also be a staff member or a family member of a resident.

[0038] The monitoring device 1 included in monitoring system A may be one device or multiple devices. In other words, monitoring system A may monitor one subject D or multiple subject D devices.

[0039] Multiple monitoring devices 1 may be installed within the elderly care facility. For example, one monitoring device 1 may be installed in each resident's room of the elderly care facility, or it may be installed in areas other than the resident's rooms (for example, corridors).

[0040] User E may be one person or multiple people. The display unit 2 included in monitoring system A may be one person or multiple people. Multiple users E may each carry one display unit 2. The display unit 2 may be a computer or display installed in a room for user E (for example, a waiting room).

[0041] [Generating the third image] As shown in Figure 2, the monitoring device 1 has an imaging unit 3 and a control unit 4. The imaging unit 3 captures the monitoring target area B (see Figure 1). In other words, the monitoring system A includes an imaging unit 3 that captures the monitoring target area B.

[0042] In this embodiment, the imaging unit 3 is configured as an infrared camera. However, the present invention is not limited thereto, and the imaging unit 3 may be any device other than an infrared camera, as long as it is capable of imaging the monitored area B. The imaging unit 3 is preferably a ToF (Time of Flight) camera.

[0043] Figure 3 shows an example of a first image 11, which is an image acquired by the imaging unit 3. Preferably, the first image 11 is a depth image, which includes depth information. The first image 11 shows the subject D, a bed, curtains, a wall, etc.

[0044] The first image 11 is sent from the imaging unit 3 to the control unit 4. As shown in Figure 2, the control unit 4 has an abstraction unit 41. The abstraction unit 41 generates the second image 12 shown in Figure 4 by performing an abstraction process on the first image 11. That is, the monitoring system A is equipped with an abstraction unit 41 that generates the second image 12 by performing an abstraction process on the first image 11.

[0045] In this embodiment, the abstraction process is specifically a mosaic process. However, the present invention is not limited thereto, and the abstraction process may be any blurring process other than a mosaic process, or a silhouette process.

[0046] Furthermore, it is preferable that the abstraction process is such that it becomes impossible to identify the individual subject D, but that the positions of objects (e.g., bed, curtains, walls) in the monitored area B can still be determined.

[0047] As shown in Figure 2, the control unit 4 has a skeletal information generation unit 42. The skeletal information generation unit 42 generates skeletal information 10 as shown in Figure 5 based on the first image 11. The skeletal information 10 is information indicating the skeleton of subject D. That is, the monitoring system A is equipped with a skeletal information generation unit 42 that generates skeletal information 10 indicating the skeleton of subject D, a person who is present in the monitoring area B, based on the first image 11, which is an image captured by the imaging unit 3.

[0048] In this embodiment, the skeletal information 10 is specifically a skeletal model. The skeletal model is generated by analyzing the first image 11 to recognize the positions of the joints of subject D in three dimensions and connecting each joint with a straight line.

[0049] As shown in Figure 2, the control unit 4 has an image generation unit 43. The image generation unit 43 generates a third image 13, as shown in Figure 6, based on the second image 12 and the skeletal information 10. The third image 13 is an image in which the second image 12 and the skeletal information 10 are superimposed. In other words, the monitoring system A is equipped with an image generation unit 43 that generates a third image 13, which is an image in which the second image 12 and the skeletal information 10 are superimposed.

[0050] In the examples shown in Figures 3 and 4, the second image 12 is generated by applying an abstraction process to the entire first image 11. However, the present invention is not limited thereto. The abstraction unit 41 may be configured to generate the second image 12 by applying an abstraction process to only a part of the first image 11. Furthermore, the abstraction unit 41 may determine which part of the first image 11 to apply the abstraction process to based on the skeletal information 10. For example, the abstraction unit 41 may abstract only the part of the first image 11 that corresponds to the part of the subject D's body or clothing that requires consideration of privacy (e.g., the face) based on the skeletal information 10.

[0051] Furthermore, for example, the abstraction unit 41 may be configured to generate the second image 12 by applying abstraction processing only to the region of the first image 11 that has been pre-set by the subject D or user E.

[0052] As shown in Figure 2, the image generation unit 43 has a display determination unit 44. The control unit 4 also has a body position determination unit 45.

[0053] The body position determination unit 45 is configured to determine the body position of subject D based on the skeletal information 10. The display determination unit 44 is configured to determine the display mode of the skeletal information 10 in the third image 13 based on the determination result by the body position determination unit 45. Although not particularly limited, the display mode may be, for example, the color in which the skeletal information 10 is displayed.

[0054] For example, if it is determined that subject D is standing or lying down in bed, the display determination unit 44 may decide to display the skeletal information 10 in blue. Also, if it is determined that subject D is sitting, the display determination unit 44 may decide to display the skeletal information 10 in yellow. Also, if it is determined that subject D is lying down on the floor, the display determination unit 44 may decide to display the skeletal information 10 in red.

[0055] Thus, monitoring system A includes a body position determination unit 45 that determines the body position of subject D based on skeletal information 10. Monitoring system A also includes a display determination unit 44 that determines the display manner of the skeletal information 10 in the third image 13 based on the determination result by the body position determination unit 45.

[0056] [Determining the display image] As shown in Figure 2, the monitoring device 1 has a transmitting unit 5. The control unit 4 is configured to transmit the third image 13 to the display unit 2 via the transmitting unit 5. The display unit 2 can display the received third image 13. In other words, the monitoring system A has a display unit 2 that can display the third image 13.

[0057] Furthermore, the control unit 4 is configured to transmit the first image 11 to the display unit 2 via the transmission unit 5. The display unit 2 can display the received first image 11.

[0058] Furthermore, the first image 11 and the third image 13 may be still images or videos.

[0059] As shown in Figure 2, the control unit 4 has an image determination unit 46. The image determination unit 46 determines which of the first image 11 and the third image 13 will be displayed on the display unit 2, based on information about the subject D or information about the user E of the display unit 2.

[0060] In other words, the monitoring system A includes an image determination unit 46 that determines which of the first image 11 and the third image 13 will be displayed on the display unit 2, based on information about the subject D or information about the user E of the display unit 2.

[0061] Furthermore, the image determination unit 46 may determine which of the first image 11 and the third image 13 will be displayed on the display unit 2 based solely on information regarding subject D. Alternatively, the image determination unit 46 may determine which of the first image 11 and the third image 13 will be displayed on the display unit 2 based solely on information regarding user E of the display unit 2. Alternatively, the image determination unit 46 may determine which of the first image 11 and the third image 13 will be displayed on the display unit 2 based on information regarding subject D and information regarding user E of the display unit 2.

[0062] When the image determination unit 46 determines that the first image 11 should be displayed on the display unit 2, the control unit 4 transmits the first image 11 to the display unit 2. In this case, the display unit 2 displays the first image 11.

[0063] When the image determination unit 46 determines that the third image 13 should be displayed on the display unit 2, the control unit 4 transmits the third image 13 to the display unit 2. In this case, the display unit 2 displays the third image 13.

[0064] In other words, according to the determination made by the image determination unit 46, either the first image 11 or the third image 13 is displayed on the display unit 2.

[0065] The image determination unit 46 is configured to determine which of the first image 11 and the third image 13 will be displayed on the display unit 2, according to the image determination flow shown in Figure 7.

[0066] When this image determination flow is initiated, the process in step S01 is executed first. In step S01, it is determined whether or not subject D is an occupant of the room that is the monitored area B.

[0067] In this determination, first, the identification unit 47 (see Figure 2) of the control unit 4 identifies the subject D. That is, the monitoring system A is equipped with an identification unit 47 that identifies the subject D. The identification unit 47 may be configured to identify the subject D based on the first image 11, for example. The method for identifying the subject D is not particularly limited, but the subject D may be identified by analyzing the first image 11 using a machine learning-trained neural network. The subject D may be identified by applying facial recognition processing to the face image captured in the first image 11, or by performing authentication processing such as card authentication or facial recognition when entering the room which is the monitored area B, and identifying the subject D based on the result of said authentication processing.

[0068] Then, based on the identification result by the identification unit 47, if it is determined that subject D is not a resident (for example, if subject D is identified as an employee) ("No" in step S01 of Figure 7), the process proceeds to step S02.

[0069] In step S02, the image determination unit 46 determines that the first image 11 will be displayed on the display unit 2. After that, this image determination flow ends for the time being.

[0070] In step S01, if it is determined that subject D is a resident based on the identification result by the identification unit 47 ("Yes" in step S01 of Figure 7), the process proceeds to step S03.

[0071] In other words, the image determination unit 46 determines, based on the identification result by the identification unit 47, which of the first image 11 and the third image 13 will be displayed on the display unit 2.

[0072] Furthermore, in the explanations for steps S03 through S08 below, subject D will be assumed to be a resident.

[0073] In step S03, the danger determination unit 48 (see Figure 2) of the control unit 4 determines whether or not subject D is in a dangerous state. The danger determination unit 48 is configured to determine whether or not subject D is in a dangerous state by analyzing the first image 11, which is an image captured by the imaging unit 3. In other words, the monitoring system A includes a danger determination unit 48 that determines whether or not subject D is in a dangerous state. The danger determination unit 48 may be configured to determine that subject D is in a dangerous state if, for example, subject D falls. Note that the technology for detecting dangerous states such as falls based on captured images is publicly known, so a description will be omitted.

[0074] If it is determined that subject D is in a dangerous state (Yes in step S03 in Figure 7), the process proceeds to step S02. That is, if it is determined that subject D is in a dangerous state, the image determination unit 46 decides that the first image 11 will be displayed on the display unit 2.

[0075] If subject D is not determined to be in a dangerous condition (No in step S03 of Figure 7), the process proceeds to step S04.

[0076] Here, the monitoring device 1 shown in Figure 2 has a storage device 6. The storage device 6 has a first storage unit 61, a second storage unit 62, and a third storage unit 63. The first storage unit 61 stores intention information, which is information indicating the intention of the subject D regarding the content displayed by the display unit 2. In other words, the monitoring system A is equipped with a first storage unit 61 that stores intention information, which is information indicating the intention of the subject D regarding the content displayed by the display unit 2.

[0077] Furthermore, the information regarding the person's wishes may be information entered by user E based on the person's (subject D's) declaration of intent at the time of moving in.

[0078] The second storage unit 62 stores authorization information, which is information indicating user E's authority regarding the content displayed by the display unit 2. In other words, the monitoring system A is equipped with a second storage unit 62 that stores authorization information, which is information indicating user E's authority regarding the content displayed by the display unit 2.

[0079] The third memory unit 63 stores a time period that has been pre-set by user E. This time period may be, for example, a time when user E wipes subject D's body or a time when subject D needs to use the toilet, or any other time period related to subject D's care. In this case, the pre-set time period is information related to subject D and also information related to user E.

[0080] The memory device 6 sends the information stored in the first memory unit 61, the second memory unit 62, and the third memory unit 63 to the control unit 4.

[0081] In step S04 shown in Figure 7, the image determination unit 46 determines whether subject D wishes to see the third image 13 based on the intention information stored in the first storage unit 61. If subject D wishes to see the third image 13 (Yes in step S04 in Figure 7), the process proceeds to step S05.

[0082] In step S05, the image determination unit 46 determines that the third image 13 will be displayed on the display unit 2. After that, this image determination flow ends.

[0083] If subject D does not wish to see the third image 13 (indicated as "No" in step S04 of Figure 7), that is, if subject D wishes to see the first image 11, the process proceeds to step S06.

[0084] In other words, the image determination unit 46 determines which of the first image 11 and the third image 13 will be displayed on the display unit 2, based on the intention information stored in the first storage unit 61.

[0085] In step S06, the image determination unit 46 determines whether user E has the authority to view the first image 11 based on the authorization information stored in the second storage unit 62. At this time, the image determination unit 46 may, for example, obtain the employee number of user E and determine whether user E has the authority to view the first image 11 based on the authorization information corresponding to the employee number.

[0086] If user E does not have permission to view the first image 11 (in step S06 of Figure 7, the answer is "No"), the process proceeds to step S05. If user E does have permission to view the first image 11 (in step S06 of Figure 7, the answer is "Yes"), the process proceeds to step S07.

[0087] In other words, the image determination unit 46 determines which of the first image 11 and the third image 13 will be displayed on the display unit 2, based on the authorization information stored in the second storage unit 62.

[0088] In step S07, the image determination unit 46 determines whether user E is of the same sex as subject D. At this time, the image determination unit 46 may, for example, obtain the employee number of user E, obtain gender information corresponding to the employee number, and compare the gender information with the gender of subject D to determine whether user E is of the same sex as subject D.

[0089] If user E is not of the same gender as subject D (No in step S07 of Figure 7), the process proceeds to step S05. If user E is of the same gender as subject D (Yes in step S07 of Figure 7), the process proceeds to step S08.

[0090] In other words, the image determination unit 46 determines, based on the gender of user E, which of the first image 11 and the third image 13 will be displayed on the display unit 2.

[0091] In step S08, the image determination unit 46 determines whether the current time falls within the time period stored in the third storage unit 63. If the current time falls within the time period ("Yes" in step S08 of Figure 7), the process proceeds to step S05. If the current time does not fall within the time period ("No" in step S08 of Figure 7), the process proceeds to step S02. In other words, the image determination unit 46 determines that the third image 13 will be displayed on the display unit 2 during the pre-set time period, and that the first image 11 will be displayed on the display unit 2 outside of that time period.

[0092] Furthermore, as is clear from the above explanation, if subject D is not determined to be in a dangerous state (No in step S03 of Figure 7), under predetermined conditions, the image determination unit 46 decides to display the third image 13 on the display unit 2. In other words, if subject D is not determined to be in a dangerous state, the image determination unit 46 decides to display the third image 13 on the display unit 2.

[0093] Furthermore, the control unit 4, the abstraction unit 41 and other functional units included in the control unit 4, and the transmission unit 5 may be physical devices such as a microcomputer, or they may be functional units in software. For example, the program corresponding to each functional unit included in the control unit 4 may be stored in ROM or non-volatile memory (not shown), and the process corresponding to each functional unit may be executed by loading and executing these programs into the CPU.

[0094] According to the configuration described above, the second image 12, which is generated by applying an abstraction process to the first image 11, and the skeletal information 10 are superimposed to generate the third image 13. In this third image 13, the external details of subject D's face and body are obscured by the abstraction process. Therefore, the privacy of subject D can be protected.

[0095] Furthermore, if a change occurs within the monitored area B, for example, due to the movement of furniture within the monitored area B, the third image 13, generated based on the first image 11 acquired after the change, will show the situation within the monitored area B after the change. This prevents the third image 13 from being inaccurate. In addition, the skeletal information 10 included in the third image 13 allows for the determination of the position and posture of subject D. Therefore, with the configuration described above, it becomes possible to accurately grasp the situation within the monitored area B (for example, the positional relationship between subject D and furniture).

[0096] In other words, with the configuration described above, it becomes possible to accurately understand the situation within monitored area B while protecting the privacy of people present in monitored area B.

[0097] [Other Embodiments] (1) In the above embodiment, the monitoring method may be configured to perform the actions performed by each component in one or more steps. For example, the monitoring method may be configured to include an imaging step corresponding to the imaging unit 3, a skeletal information generation step corresponding to the skeletal information generation unit 42, an abstraction step corresponding to the abstraction unit 41, an image generation step corresponding to the image generation unit 43, and a display step corresponding to the display unit 2.

[0098] (2) The monitoring program may be configured to cause a computer to implement the functions of each component in the above embodiment. For example, the monitoring program may be configured to cause a computer to implement an imaging function corresponding to the imaging unit 3, a skeletal information generation function corresponding to the skeletal information generation unit 42, an abstraction function corresponding to the abstraction unit 41, an image generation function corresponding to the image generation unit 43, and a display function corresponding to the display unit 2.

[0099] (3) Some or all of the control unit 4, transmission unit 5, and storage device 6 may be located outside the monitoring device 1 (for example, on a personal computer used by user E).

[0100] (4) In the image determination flow shown in Figure 7, one or more of steps S01, S03, S04, and S06 to S08 may be omitted, and the order of the steps may be changed.

[0101] (5) The monitoring system A may be configured such that the display unit 2 cannot display the first image 11. The control unit 4 may also be configured not to transmit the first image 11 to the display unit 2 under any circumstances.

[0102] (6) In step S01 of the image determination flow shown in Figure 7, the system may be configured to determine "No" if subject D is identified as a person other than a resident, and to determine "Yes" if subject D cannot be identified, or if subject D is identified as a resident.

[0103] Furthermore, the configurations disclosed in the above-described embodiments (including other embodiments, the same applies hereinafter) can be applied in combination with configurations disclosed in other embodiments, as long as no inconsistencies arise. In addition, the embodiments disclosed herein are illustrative, and the embodiments of the present invention are not limited thereto, and can be modified as appropriate without departing from the object of the present invention. [Industrial applicability]

[0104] This invention can be used not only in elderly care facilities but also in various other facilities such as medical facilities and public facilities, and can even be used outdoors. [Explanation of Symbols]

[0105] 1: Monitoring device 2: Display section 3: Imaging Unit 10: Skeletal Information 11: First image 12: Second image 13: Third image 41: Abstraction part 42: Skeletal Information Generation Unit 43: Image generation unit 44: Display determination section 45: Body position determination section 46: Image determination unit 47: Identification unit 48: Hazard Assessment Department 61: 1st memory section 62:Second memory section A: Monitoring system B: Monitoring area D: Target audience E: User (Observer)

Claims

1. An imaging unit that captures images of the area to be monitored, A skeleton information generation unit generates skeleton information indicating the skeleton of a person, which is a subject, located in the monitored area, based on a first image, which is an image captured by the imaging unit. An abstraction unit generates a second image by applying an abstraction process to the first image to such an extent that the individual subject cannot be identified, An image generation unit that generates a third image which is an image obtained by superimposing the second image and the skeletal information, The system comprises a display unit capable of displaying the third image, The display unit is capable of displaying the first image, The system includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination made by the image determination unit, either the first image or the third image is displayed on the display unit. The system includes a second storage unit that stores authority information, which is information indicating the authority of the monitor, who is the person monitoring the subject by looking at the display unit, regarding the content displayed by the display unit. The image determination unit is a monitoring system that determines which of the first image and the third image will be displayed on the display unit, based on the authorization information stored in the second storage unit.

2. A first storage unit that stores intention information which is information indicating the intention of the subject regarding the content displayed by the display unit, The monitoring system according to claim 1, wherein the image determination unit determines which of the first image and the third image is to be displayed on the display unit based on the intention information stored in the first storage unit.

3. A danger determination unit that determines whether the subject is in a dangerous state, If the subject is determined to be in a dangerous state, the image determination unit decides to display the first image on the display unit. The monitoring system according to claim 1 or 2, wherein if the subject is not determined to be in a dangerous state, the image determination unit determines that the third image is displayed on the display unit.

4. An imaging unit for imaging the area to be monitored, A skeleton information generation unit generates skeleton information indicating the skeleton of a person, which is a subject, located in the monitored area, based on a first image, which is an image captured by the imaging unit. An abstraction unit generates a second image by applying an abstraction process to the first image to such an extent that the individual subject cannot be identified, An image generation unit that generates a third image which is an image obtained by superimposing the second image and the skeletal information, The system comprises a display unit capable of displaying the third image, The display unit is capable of displaying the first image, The system includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination made by the image determination unit, either the first image or the third image is displayed on the display unit. The image determination unit is a monitoring system that determines which of the first image and the third image will be displayed on the display unit based on the gender of the monitor, who is the person monitoring the subject, by looking at the display unit.

5. An imaging unit for imaging a target area, A skeleton information generation unit generates skeleton information indicating the skeleton of a person, which is a subject, located in the monitored area, based on a first image, which is an image captured by the imaging unit. An abstraction unit generates a second image by applying an abstraction process to the first image to such an extent that the individual subject cannot be identified, An image generation unit that generates a third image which is an image obtained by superimposing the second image and the skeletal information, The system comprises a display unit capable of displaying the third image, The display unit is capable of displaying the first image, The system includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination made by the image determination unit, either the first image or the third image is displayed on the display unit. The image determination unit determines that the third image will be displayed on the display unit during a predetermined time period, and determines that the first image will be displayed on the display unit outside of that time period.

6. An imaging unit for imaging a target area, A skeleton information generation unit generates skeleton information indicating the skeleton of a person, which is a subject, located in the monitored area, based on a first image, which is an image captured by the imaging unit. An abstraction unit generates a second image by applying an abstraction process to the first image to such an extent that the individual subject cannot be identified, An image generation unit that generates a third image which is an image obtained by superimposing the second image and the skeletal information, The system comprises a display unit capable of displaying the third image, The display unit is capable of displaying the first image, The system includes an image determination unit that determines which of the first image and the third image is displayed on the display unit. In accordance with the determination made by the image determination unit, either the first image or the third image is displayed on the display unit. The system includes an identification unit for identifying the aforementioned target person, The image determination unit is a monitoring system that determines which of the first image and the third image will be displayed on the display unit based on the identification result by the identification unit.

7. The monitoring system according to any one of claims 1 to 6, wherein the abstraction unit is configured to generate the second image by applying abstraction processing to only a part of the first image.

8. The monitoring system according to claim 7, wherein the abstraction unit determines the portion of the first image to be abstracted based on the skeletal information.

9. An imaging unit for imaging the area to be monitored, A skeleton information generation unit generates skeleton information indicating the skeleton of a person, which is a subject, located in the monitored area, based on a first image, which is an image captured by the imaging unit. An abstraction unit generates a second image by applying an abstraction process to the first image to such an extent that the individual subject cannot be identified, An image generation unit that generates a third image which is an image obtained by superimposing the second image and the skeletal information, The system comprises a display unit capable of displaying the third image, A body position determination unit that determines the body position of the subject based on the skeletal information, A monitoring system comprising: a display determination unit that determines the display mode of the skeletal information in the third image based on the determination result by the body position determination unit.

Images