Monitoring device and monitoring method

Abstract

To provide a monitoring device and a monitoring method, capable of gripping a situation of a monitored person immediately at the time when starting or restarting a monitoring by a monitoring person while achieving both of gripping of a situation of a person to be monitored and a privacy protection.SOLUTION: A monitoring device comprises: a monitoring camera 1; and a home server 2, and monitors a monitoring space 4 where a person to be monitored 3 monitors exists. The monitoring camera 1 photographs a monitoring space where the person to be monitored exists. The home server 2 comprises: a person detection part that detects a person from a photographing image acquired by the monitoring camera 1, and divides the person to be detected into a face part and a physical body portion; a processing drawing part that performs a processing and drawing processing against the physical portion excluding the face part divided by the person detection part; an image composite part 203 that creates a composite image of an image to which the processing drawing processing is made and the photographing image; and a network part that transmits the composite image to a display terminal 8 of a monitoring person 7.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a monitoring device and a monitoring method for monitoring a person using a surveillance camera, and to a monitoring technology that achieves both understanding of the situation of a person being monitored (a person being monitored) and protecting their privacy. [Background technology]

[0002] In surveillance devices, in order to protect the privacy of the person being watched, abstraction such as mosaic processing or frosted glass processing is performed on the entire image captured by the surveillance camera or on body parts including the person's face, but this poses the problem of making it difficult to grasp the situation of the person being watched.

[0003] In this regard, Patent Document 1 discloses a surveillance system that is equipped with a sensor that detects human movement, and when the sensor detects human movement, blurs the captured image data before outputting it, and when no human movement is detected for a predetermined period of time, outputs the captured image data. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2018-18456 Summary of the Invention [Problem to be solved by the invention]

[0005] In the method of Patent Document 1, even if privacy protection is taken into consideration, a blurring process is applied when a person is moving, making it difficult to accurately grasp the person's situation. In particular, because the blurring process is applied to the person's face as well, it becomes difficult to determine whether the person is the person being watched. There is no need to hide the face of the person being watched; rather, it is possible to know the person's health condition, etc. by looking at their face. Furthermore, in Patent Document 1, image output is controlled after a sensor detects whether the person is moving, so it takes a certain amount of time until the image is output, and there is an issue that when the person watching (watcher) tries to start watching, the situation of the person being watched cannot be instantly grasped.

[0006] The present invention has been made in consideration of the above points, and its purpose is to provide a monitoring device and a monitoring method that can simultaneously grasp the situation of the person being watched while protecting their privacy, and that can instantly grasp the situation of the person being watched when the watcher tries to start or resume watching. [Means for solving the problem]

[0007] In order to solve the above problems, the monitoring device of the present invention comprises a surveillance camera that captures images of the monitoring space in which the person being monitored is located, a person detection unit that detects people from the captured image acquired by the surveillance camera and divides the detected people into facial and body parts, a processing and drawing unit that performs processing and drawing processing on the body parts excluding the facial parts divided by the person detection unit, an image synthesis unit that creates a composite image of the processed and drawn image and the captured image, and a network unit that transmits the composite image to the display terminal of the person monitoring.

[0008] The monitoring method of the present invention includes an image acquisition step of acquiring a photographed image of a monitoring space in which a person being monitored is present; a person detection step of detecting a person from the photographed image and dividing the detected person into a face and a body part; a processing and drawing step of performing a processing and drawing process on the body parts excluding the face divided in the person detection step; an image synthesis step of creating a composite image of the processed and drawn image and the photographed image; and a transmission step of transmitting the composite image to a display terminal of the monitor. [Effects of the Invention]

[0009] According to the present invention, it is possible to grasp the situation of the person being watched while protecting their privacy, and it is possible for the watcher to instantly grasp the situation of the person being watched when the watcher starts or attempts to resume watching. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing the overall configuration of a monitoring system according to a first embodiment. [Figure 2A] FIG. 1 is a block diagram showing an example of the configuration of a surveillance camera. [Figure 2B] FIG. 1 is a block diagram showing an example of the configuration of a surveillance camera (with a distance measurement function). [Figure 3] FIG. 2 is a block diagram showing an example of the configuration of a home server. [Figure 4A] 10A and 10B are diagrams showing an example of processing and drawing processing (coloring processing) performed by a processing and drawing unit; [Figure 4B] 10A and 10B are diagrams showing an example of processing and drawing processing (shading processing) performed by a processing and drawing unit; [Figure 5] FIG. 10 is a flowchart showing a monitoring process. [Figure 6] FIG. 10 is a diagram showing the overall configuration of a monitoring system according to a second embodiment. [Figure 7] FIG. 2 is a block diagram showing an example of the configuration of a home server. [Figure 8] FIG. 10 is a diagram showing an example of a monitoring image displayed on a display terminal. [Figure 9] FIG. 10 is a flowchart showing a monitoring process. [Figure 10] FIG. 11 is a block diagram showing an example of the configuration of a home server according to a third embodiment. [Figure 11] FIG. 10 is a diagram showing an example of a monitoring image displayed on a display terminal. [Figure 12] FIG. 10 is a flowchart showing a monitoring process. [Figure 13A] FIG. 10 is a block diagram showing an example of the configuration of a server-integrated surveillance camera according to a fourth embodiment. [Figure 13B]FIG. 1 is a block diagram showing an example of the configuration of a server-integrated surveillance camera (with a ranging function). [Figure 14] FIG. 10 is a diagram showing the overall configuration of a monitoring system according to a fifth embodiment. [Figure 15A] FIG. 10 is a diagram showing an example of a displayed monitoring image (monitoring camera 101). [Figure 15B] FIG. 10 is a diagram showing an example of a displayed monitoring image (monitoring camera 102). [Figure 16] FIG. 10 is a flowchart showing a monitoring process. [Figure 17] FIG. 13 is a diagram showing the overall configuration of a monitoring system according to a sixth embodiment. [Figure 18] FIG. 2 is a block diagram showing an example of the configuration of a server. [Figure 19] FIG. 10 is a flowchart showing a monitoring process. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. [Example]

[0012] In the first embodiment, the basic configuration and basic operation (person detection, processing and drawing) of the monitoring device will be described.

[0013] FIG. 1 is a diagram showing the overall configuration of a monitoring system according to a first embodiment. The monitoring system includes a monitoring camera 1 and a home server 2 as monitoring devices, and monitors a monitoring space 4 in which a monitored person 3 is present. The home server 2 is connected to a display terminal 8 carried by a monitor 7 via a network 5 and an access point 6. The monitoring camera 1 captures images of the monitoring space 4. Image data captured by the monitoring camera 1 is transmitted to the home server 2 via communication signals 1a and 2a. The home server 2 performs processing and drawing on the image data, which will be described later, and transmits the processed image to the display terminal 8 via the network 5 and the access point 6 via communication signals 6a and 8a. The processed image is displayed on the display terminal 8, and the monitor 7 can visually view the image to understand the situation of the monitored person 3.

[0014] 2A and 2B are block diagrams showing two configuration examples of the surveillance camera 1. FIG. 2A is configured to have only a photographing function, and includes a lens unit 10, an image sensor 11, a signal processing unit 12, a network unit 13, and a controller 14. The light of the subject focused by the lens unit 10 is formed on the image sensor 11, converted into image data by the signal processing unit 12, and transmitted from the network unit 13 to the home server 2 as a communication signal 1a. The controller 14 controls all of these operations.

[0015] The surveillance camera 1 shown in FIG. 2B has a distance measurement function in addition to the imaging function of FIG. 2A. To this end, it also includes an irradiation unit 16, a light receiving unit 17, an infrared light generating unit 18, and a TOF (Time of Flight) detection unit 19. The infrared light generated by the infrared light generating unit 18 is emitted from the irradiation unit 16 to the outside (monitored space 4). The light receiving unit 17 receives the reflected light of the emitted infrared light reflected by the target (monitored person 3). The TOF detection unit 19 measures the distance from the surveillance camera to the target based on the time difference between the emitted light and the reflected light, thereby obtaining distance measurement data. Note that the distance measurement function can also be achieved by other methods than TOF, such as irradiating infrared light with a specific two-dimensional pattern and calculating the distance from the distortion of the two-dimensional pattern of the received infrared light, or by using laser light instead of infrared light.

[0016] FIG. 3 is a block diagram showing an example configuration of the home server 2. The home server 2 comprises a network unit 21, a CPU 22, a RAM 23, and a storage unit 24. The storage unit 24 has a basic program 24a and a monitoring program 24b as processing programs and data. The processing programs and data are expanded in the RAM 23 and executed by the CPU 22 to perform various controls of the monitoring operation. The storage unit 24 may be configured by a semiconductor such as a flash ROM or by combining it with a hard disk, and there is no limitation on the configuration method. The network unit 21 receives captured image data from the monitoring camera 1 and transmits image data for display to the display terminal 8.

[0017] The storage unit 24 stores a monitoring program 24b, which the CPU 22 executes to realize functional units such as a person detection unit 201, a processing and rendering unit 202, and an image synthesis unit 203. The person detection unit 201 detects a person (the person being watched) from an image captured by the camera and performs processing to divide the detected person area into body parts such as the face and torso. The processing and rendering unit 202 also performs color processing on each body part of the person and shading processing according to the distance to each part of the person to protect the privacy of the person being watched. The image synthesis unit 203 synthesizes the captured image with an image that has been processed and rendered, and the synthesized image is transmitted as a monitoring image to the display terminal 8 of the watcher 7 via the network unit 21. As described above, the home server 2 of this embodiment has the function of performing image processing on the image captured by the camera and providing the image to the watcher, and can be called a "management device."

[0018] 4A and 4B are diagrams showing an example of the processing and drawing process performed by the processing and drawing unit 202. FIG. 4A shows an example of coloring a person's body parts. When the person detection unit 201 detects a person 30 from a captured image, it classifies the person into body parts such as a face 31, torso 32, hands 33, and feet 34. The processing and drawing unit 202 then assigns different colors to the parts 32, 33, and 34 other than the face 31 (here, the color differences are represented by patterns).

[0019] Fig. 4B shows an example of shading processing applied to body parts according to distance measurement data when the surveillance camera 1 has the configuration of Fig. 2B (with distance measurement function). Shading processing is performed on parts other than the face 31, pixel by pixel or on a multiple-pixel basis, brightening pixels that are closer and darkening pixels that are farther away. Shading processing is performed after and in combination with the coloring processing applied in Fig. 4A, but shading processing alone may be performed without coloring processing.

[0020] The surveillance image that has undergone the above-mentioned processing and rendering process is combined with the captured image and sent to the display terminal 8 for display. As a result, the watcher 7 cannot see the details of the body of the person 30, but the movement of the body is easy to understand. In particular, the shading process makes it possible to express the positional relationship between the front and back with shading, making it easier to grasp the movement.

[0021] The processing and drawing process of this embodiment is not performed on the face of the person 30. Therefore, the situation can be understood without any problems based on the facial expression. In addition, since each body part is drawn, it has the effect of making it easier to understand the behavior of the person.

[0022] If there are multiple people (people being watched), different colors may be assigned to each person being watched, making it easier to distinguish between them. Also, depending on the direction of the person's face, the face of the person being watched may not be detected in the image captured by the surveillance camera, and only the head may be detected. Even in this case, it is best to prioritize protecting the privacy of the person being watched and perform the above processing.

[0023] 5 is a flowchart of the watch-over monitoring process. The following process is realized by the CPU 22 of the home server 2 executing the watch-over monitoring program 24b.

[0024] Processing begins at S100, and at S101 the user logs in to the monitoring program. By logging in, the home server 2 and the surveillance camera 1 are linked. At S102, captured image data is received from the surveillance camera 1 via the network unit 21. When the surveillance camera 1 shown in FIG. 2B is used, ranging data is received along with the captured image data.

[0025] In S103, the person detection unit 201 detects people from the captured image data. Furthermore, in S104, the person detection unit 201 classifies the area of ​​the detected person into body parts such as face, torso, hands, and feet. In S106, it is determined whether the body part classification has been completed for all detected people. If there are any unprocessed people, the process returns to S104 and is repeated.

[0026] In S107, the processing and rendering unit 202 assigns different color data to the body parts. Furthermore, in S108, if distance measurement data is present along with the image data, shading data is assigned according to the distance measurement data. In S109, processing and rendering processing of the image data is performed according to the assigned color and shading data. In S110, it is determined whether processing and rendering processing has been completed for all body parts. Furthermore, in S111, it is determined whether processing and rendering processing has been completed for all people. If there are unprocessed body parts or people, the process returns to S107 and is repeated.

[0027] In S112, the processed and rendered image data is combined with the captured image data. In S113, the combined image data is transmitted to the display terminal 8 via the network 5 by the network unit 21.

[0028] In S114, if an instruction to end the monitoring process is given by logging off or the like, the process ends in S115. If an instruction to end the process is not given, the process returns to S102 and continues the monitoring process.

[0029] As described above, the processing and drawing process of this embodiment is not performed on the face of a person, but on body parts other than the face. In the case of monitoring, for example, when watching over elderly parents at home or when watching over children and nursery teachers at a nursery school, the person being watched over is a person that the watcher has previously intended, and displaying the face image of the person being watched over as is often not a privacy issue.

[0030] On the other hand, the purpose of performing the processing and drawing process in this embodiment is to hide the details of the body parts of the person being watched, so that the person being watched over will not be seen in casual clothing, such as when changing clothes or after a bath. It also prevents the watcher from seeing the privacy of the person being watched over, especially elderly parents, more than necessary. It also prevents image data showing the entire body from being leaked from the display terminal. This allows the person being watched over to be monitored without having to worry about their lifestyle or activity style for privacy reasons. Furthermore, the processing and drawing process of this embodiment allows the same style of processed image to be displayed regardless of the movements of the person being watched over.

[0031] As described above, according to the first embodiment, it is possible to grasp the situation of the person being watched from facial expressions and body part movements, while also achieving privacy protection that does not obscure the lifestyle of the person being watched. Furthermore, the same style of watching image is provided regardless of the situation of the person being watched, which has the effect of enabling the watching person to instantly grasp the watching situation even when the watching person starts or tries to resume watching. [Example]

[0032] In the second embodiment, a case will be described in which a plurality of people exist in the watching space, and each person is judged as to whether or not they are being watched, and each drawing process is performed.

[0033] Fig. 6 is a diagram showing the overall configuration of a monitoring system according to Example 2. An example is shown in which multiple people 3a to 3c are present in a monitoring space 4. Elements having the same functions as those in the monitoring device shown in Example 1 (Fig. 1) are given the same reference numerals, and redundant explanations will be omitted.

[0034] Fig. 7 is a block diagram showing an example of the configuration of the home server 2 in Example 2. Compared with the configuration of the home server 2 in Example 1 (Fig. 3), the storage unit 24 includes face information registration data 24c in addition to a basic program 24a and a monitoring program 24b. The face information registration data 24c is pre-registered face information of the person being watched over, who is the target of watching over.

[0035] The person detection unit 201 compares the facial information of the detected person with the registered data of the facial information registration data 24c to determine whether the target person is registered. If the target person is a registered person (a person being watched), the processing and drawing unit 202 performs processing and drawing processing, and if the target person is an unregistered person (a person other than a person being watched), the processing and drawing processing is not performed.

[0036] 8 is a diagram showing an example of a monitoring image 80 displayed on the display terminal 8. Here, it is assumed that a plurality of persons 3a to 3c are detected, of which persons 3b and 3c are persons being watched over whose face information is registered in the face information registration data 24c, and person 3a is not a person being watched over whose face information is not registered in the face information registration data 24c.

[0037] The processing and drawing unit 202 performs coloring processing on body parts other than the face for the registered persons 3b and 3c, but does not perform processing and drawing processing on body parts for the unregistered person 3a.

[0038] Fig. 9 is a diagram showing a flowchart of the monitoring process. Steps that perform the same processes as those shown in the first embodiment (Fig. 5) are given the same reference numerals, and duplicated explanations will be omitted. In the flowchart of Fig. 9, S105a and S105b are added.

[0039] In S105a, the person detection unit 201 compares the facial information of the detected person with the registered data of the facial information registration data 24c to determine whether the target person is registered. In S105b, the process branches depending on whether the target person is a person being watched (registered), and if the target person is a person being watched (Yes in S105b), processing and drawing processing from S107 onwards is performed. If the target person is not a person being watched (No in S105b), processing and drawing processing is not performed and the process proceeds to S111, where processing for the next person is carried out.

[0040] According to the above process flow, the processing and drawing process is performed only on people registered as watched over, thereby protecting the privacy of the watched over. For people other than the watched over, the captured image is displayed as is without the processing and drawing process, allowing the user to grasp the detailed situation.

[0041] As described above, Example 2 has the same effect as Example 1, and also has the effect that when a person other than the person being watched is present in the watched space, the person watching over can clearly and quickly grasp the situation as an unintended person. [Example]

[0042] In the third embodiment, a case where a suspicious person is detected in a monitored space and an alarm is issued will be described. The overall configuration of the monitoring system is the same as that of the second embodiment (FIG. 6).

[0043] Fig. 10 is a block diagram showing an example of the configuration of the home server 2 in Example 3. Compared with the configuration of the home server 2 in Example 2 (Fig. 7), the storage unit 24 includes suspicious person pattern registration data 24d and an image data storage unit 24e in addition to the basic program 24a, the monitoring program 24b, and the face information registration data 24c.

[0044] The suspicious person pattern registration data 24d is data that registers appearance situations in which a person intentionally hides their face as a suspicious person pattern. For example, a suspicious person pattern corresponds to a situation in which a person is wearing a full-face helmet, a ski mask, a mask and sunglasses, etc. The image data storage unit 24e stores image data (before processing and rendering) captured by a surveillance camera.

[0045] The person detection unit 201 compares the detected person with the suspicious person pattern registration data 24d to determine whether or not it corresponds to a suspicious person pattern. If it is determined that the person corresponds to a suspicious person pattern, alarm information is created and notified to the display terminal 8. In this case, the processing and drawing unit 202 does not perform processing and drawing processing on the captured image, but transmits the captured image data (before processing and drawing processing) from the surveillance camera stored in the image data storage unit 24e to the display terminal 8.

[0046] 11 is a diagram showing an example of a surveillance image 80 displayed on the display terminal 8. Here, multiple persons 3a to 3c are detected, of which persons 3b and 3c are persons being watched whose facial information is registered in the facial information registration data 24c. On the other hand, it is assumed that the facial information of person 3a is not registered in the facial information registration data 24c, and that person 3a matches a suspicious person pattern 39 (e.g., wearing a helmet) registered in the suspicious person pattern registration data 24d.

[0047] When the person detection unit 201 detects a person 3a that matches the suspicious person pattern among the people, it creates alarm information 89 such as "There is a suspicious person," includes it in the surveillance image 80, and sends it to the display terminal 8 to be displayed. Alternatively, the alarm information may be transmitted to the display terminal 8 by voice. In this case, the processing and drawing unit 202 does not perform processing and drawing processing on the captured image not only for the person 3a that has been determined to be a suspicious person, but also for the registered people 3b and 3c. This is because the presence of a suspicious person indicates an abnormal situation, and it is necessary to inform the watcher 7 in detail of the status of each detected person.

[0048] Fig. 12 is a diagram showing a flowchart of the monitoring process. Steps that perform the same processes as those shown in the second embodiment (Fig. 9) are given the same reference numerals, and duplicated explanations will be omitted. In the flowchart of Fig. 12, abnormality processing steps S123 to S125 have been added.

[0049] In S120, the person detection unit 201 compares the person with the suspicious person pattern registration data 24d, and if a person that fits the suspicious person pattern is detected, the process proceeds to abnormality processing from S123 onwards. Also, in S121, if the person does not fit the suspicious person pattern but the face information does not match the registered data in the face information registration data 24c, the person is considered to be an unintended person other than the person being watched, and the process proceeds to abnormality processing from S123 onwards.

[0050] Furthermore, in S122, even if a higher-ranking system authority (e.g., a watcher) issues an instruction to stop the processing and drawing process, the process proceeds to S123 and subsequent abnormality processing. This is the case when the watcher requests a detailed image.

[0051] In S123, alarm information 89 is created and sent to the display terminal 2. Taking into consideration the case where the watcher 7 is not looking at the display terminal 8, alarm information 89 may be sent by voice, email, etc. In S124, the captured image is saved in the image data saving unit 24e. This allows the image data to be analyzed when an abnormality occurs and used to identify suspicious individuals, etc. In S125, the captured image data saved in the image data saving unit 24e is sent to the display terminal 8 as is, rather than image data that has been processed and drawn.

[0052] The conditions for performing the above-described abnormality processing (S123 to S125) are not limited to the above cases. In cases where the number of people being watched is limited, such as elderly people living alone, the abnormality processing may be set to be performed immediately when more people than the number of face information registered in the image data are detected. For example, this applies to a situation where face information for two people, 3b and 3c, is registered in the face information registration data 24c, and three people, 3a to 3c, are detected, as shown in Figure 11.

[0053] Furthermore, the conditions for executing abnormality processing can be set appropriately by a person with higher system authority (a watcher), thereby realizing a system that is easy for users to use.

[0054] As described above, according to the third embodiment, an abnormal situation such as the presence of a suspicious person in the monitored space can be quickly detected and notified to the monitor. [Example]

[0055] In the first to third embodiments, the surveillance camera 1 and the home server 2 are described as separate entities, but in the fourth embodiment, a configuration in which the two are integrated will be described.

[0056] Figures 13A and 13B are block diagrams showing two configuration examples of a server-integrated surveillance camera. Figure 13A shows a case where the surveillance camera 1 of Figure 2A and the functions of the home server 2 of Example 2 (Figure 7) are integrated. Figure 13B shows a case where the surveillance camera 1 (with a distance measurement function) of Figure 2B and the functions of the home server 2 of Example 2 (Figure 7) are integrated.

[0057] In the configuration of these server-integrated surveillance cameras 1b, the home server 2 is referred to as the controller 14a, and the storage unit 24 has added thereto a camera control program 24f that controls the surveillance camera 1. The network unit 21 also sends composite image data of the processed and rendered image to the external network 5, but this may be sent directly to the network, or via an access point placed in the monitoring space 4. It goes without saying that the controller 14a in the server-integrated surveillance camera 1b may have the configuration of other embodiments (Figs. 3 and 10).

[0058] According to the fourth embodiment, the surveillance camera 1 and the home server 2 are integrated, which has the effect of reducing the overall size of the surveillance device and facilitating the installation of the device. [Example]

[0059] In the fifth embodiment, a case will be described in which, when a plurality of surveillance images are displayed by installing a plurality of surveillance cameras, a common processing and rendering process is performed on the same person.

[0060] FIG. 14 is a diagram showing the overall configuration of a monitoring system according to a fifth embodiment. Multiple (here, two) monitoring cameras 101, 102 are installed in a monitoring space 4. Each monitoring camera 101, 102 has the same configuration as the monitoring camera 1 shown in FIG. 2A or 2B, and is connected to a common home server 2. Here, the home server 2 shown in the second embodiment (FIG. 7) is applied. The home server 2 performs processing and rendering on the images captured by the monitoring cameras 101, 102. Then, upon request from the watcher 7, it switches between the images captured by the monitoring cameras 101, 102 and transmits them to the display terminal 8. By using multiple monitoring cameras, even if the monitoring space 4 is large, it is possible to monitor the entire space by dividing the area.

[0061] 15A and 15B are diagrams showing examples of surveillance images displayed on display terminal 8. FIG. 15A shows surveillance image 81 from surveillance camera 101, and FIG. 15B shows surveillance image 82 from surveillance camera 102. Because the surveillance areas of surveillance cameras 101 and 102 overlap a portion of monitoring space 4, the same person may appear in both surveillance images 81 and 82. In this example, person 3b is included in both surveillance images 81 and 82. In such a case, editing and rendering unit 202 applies the same editing and rendering process to the two surveillance images for person 3b, applying the same coloring and shading process. On the other hand, different editing and rendering processes are applied to the other persons 3a and 3c. As a result, when switching between surveillance cameras 101 and 102, the same editing and rendering process is applied to the same person, so the watcher can immediately tell that they are the same person, and can easily recognize them as a single person even if the monitoring space is divided.

[0062] 16 is a flowchart of the monitoring process, in which the same steps as those shown in the second embodiment (FIG. 9) are denoted by the same reference numerals.

[0063] In the flowchart of FIG. 16, a step S130 is added, and the person detection process of S102 to S106 is repeated for each of the multiple surveillance cameras.

[0064] In S131, the person detection unit 201 identifies the same person appearing in multiple surveillance cameras. In S132, selection information for the surveillance camera from which the surveillance image should be transmitted is received from the display terminal 8. From S107 onwards, processing and rendering is performed on the images captured by the selected surveillance camera, but the same processing and rendering is performed on people identified as the same person in S131. Note that, as in the second embodiment, processing and rendering is not performed on people who are not registered in the face data matching in S105. In S113, the surveillance image from the selected surveillance camera is transmitted to the display terminal 8.

[0065] As described above, according to Example 5, in a surveillance monitoring device equipped with multiple surveillance cameras, by monitoring while switching between the surveillance cameras, it is possible to perceive the divided surveillance space as a continuous space and easily identify the same person being watched across multiple surveillance images. [Example]

[0066] In the sixth embodiment, a configuration will be described in which a server, which is a management device of a monitoring system, is arranged on a network.

[0067] Fig. 17 is a diagram showing the overall configuration of a monitoring system according to Example 6. The monitoring system of this example is configured to realize the functions of the home server 2 in Example 1 (Fig. 1) or Example 2 (Fig. 6) by a server 9 placed on a network 5. This configuration is suitable for providing monitoring as a commercial service via a network.

[0068] The new configuration adds a server 9, which is a management device placed on the network 5, and an access point 60, which relays images captured by the surveillance camera 1. The access point 60 transmits image data captured by the surveillance camera 1 to the server 9 via a communication signal 60a and the network 5. The server 9 processes and renders the images captured by the surveillance camera 1, and transmits them to the display terminal 8 via the access point 6.

[0069] FIG. 18 is a block diagram showing an example of the configuration of the server 9 in the sixth embodiment. The basic configuration is the same as that of the second embodiment (FIG. 7), and includes a network unit 91, a CPU 92, a RAM 93, and a storage unit 94. The storage unit 94 includes, as processing programs and data, a basic program 94a, a user management program 94b, a monitoring service program 94c, and face information registration data 94d. Here, the user management program 94b manages users to whom the monitoring service is provided. The monitoring space 4 in FIG. 17 is a space for a specific user to whom the service is provided. The face information registration data 94d contains pre-registered face data of the person being monitored for each user. The processing programs and data are expanded in the RAM 93 and executed by the CPU 92. The storage unit 94 is not limited to a configuration that may be formed from a semiconductor such as a flash ROM or a combination with a hard disk.

[0070] 19 is a diagram showing a flowchart of the watch-over monitoring process. The following process is executed by the watch-over monitoring service program 94c. The same reference numerals are given to steps that perform the same processes as those shown in the second embodiment (FIG. 9).

[0071] The process starts at S100, and the user logs in to the monitoring service at S140. The login associates the server 9 with the user's monitoring camera 1.

[0072] In S102, image data is received from the surveillance camera. The subsequent steps are the same as those in the processing flow of FIG.

[0073] As described above, the sixth embodiment has the effect of being applicable to a commercial monitoring service for multiple users.

[0074] In the above embodiments, the communication signals 1a, 2a, 6a, 8a, etc. are illustrated as wireless signals, but they may be wired signals. Furthermore, even if an image compression / expansion technique such as H.264 is used to reduce the amount of data in the communication section, this does not impair the object and effect of the present invention.

[0075] The embodiments described above are not limited to these, and it is possible to replace part of the configuration of one embodiment with that of another embodiment. It is also possible to add the configuration of one embodiment to the configuration of another embodiment. These all fall within the scope of the present invention, and the numerical values, messages, etc. appearing in the text and figures are merely examples, and the use of different ones does not impair the effects of the present invention.

[0076] Furthermore, some or all of the functional blocks described in the embodiments may be implemented in hardware, for example, by designing them as integrated circuits. Alternatively, they may be implemented in software by a microprocessor unit, CPU, or the like interpreting and executing an operating program. Furthermore, the scope of software implementation is not limited, and both hardware and software may be used. [Explanation of symbols]

[0077] 1, 1b, 101, 102: surveillance camera, 2: home server, 3, 3a to 3c, 30: person (monitored person), 4: monitoring space, 5: network, 6, 60: access point, 7: monitor, 8: display terminal, 9: server, 21: network unit, 22: CPU, 23: RAM, 24, 94: storage unit, 24b: monitoring program, 94c: monitoring service program, 24c, 94d: face information registration data, 24d: suspicious person pattern registration data, 24e: image data storage unit, 24f: camera control program, 31: head, 32 to 34: body parts, 39: suspicious person pattern, 80, 81, 82: surveillance image, 201: person detection unit, 202: processing and drawing unit, 203: image synthesis unit.

Claims

1. A surveillance camera captures the space, a person detection unit that detects a person from the captured image acquired by the surveillance camera and classifies the detected person into a face part and a body part; a processing and rendering unit that performs processing and rendering on the body parts excluding the face part that have been separated by the person detection unit; an image synthesis unit that creates a synthesized image of the image that has been subjected to the processing and drawing process and the captured image; a network unit that transmits the composite image to a display terminal; Equipped with The face information registration data includes face information of a person to be monitored, the person detection unit compares face information of the plurality of detected persons with the face information registration data, and determines whether the plurality of detected persons are registered; A monitoring device characterized in that, if the result of the judgment is that all of the multiple detected people are registered, the processing and drawing unit performs processing and drawing processing on all of the multiple people, and if there is even one person among the multiple detected people who is not registered, the processing and drawing processing on all of the multiple people is not performed by the processing and drawing unit.

2. 2. The monitoring device according to claim 1, A monitoring device characterized by performing, as a monitoring abnormality processing, a process of creating alarm information and sending it to the display terminal, or a process of sending images captured by the monitoring camera to the display terminal without performing any processing or drawing processing on them.

3. 3. The monitoring device according to claim 2, having suspicious person pattern registration data in which the appearance of a suspicious person is registered as a suspicious person pattern; The monitoring device is characterized in that the person detection unit compares the detected person with the suspicious person pattern registration data, and when it is determined that the detected person corresponds to a suspicious person pattern, executes the abnormality processing.

4. 3. The monitoring device according to claim 2, A monitoring device characterized in that the person detection unit executes the abnormality processing when it determines that the detected person is an unintended person other than a person to be monitored, or when the number of people detected by the person detection unit is greater than the number of people registered as people to be monitored.

5. 2. The monitoring device according to claim 1, A configuration in which a plurality of the surveillance cameras are installed to acquire a plurality of captured images, A monitoring device characterized in that when the person detection unit determines that the same person is included in multiple captured images, the editing and drawing unit performs a common editing and drawing process for the same person across the multiple captured images.

6. a network unit that receives images captured by a surveillance camera that captures images of a space; Facial information registration data that registers facial information of people who are the targets of surveillance; a person detection unit that detects a person from the captured image acquired by the surveillance camera and classifies the detected person into a face part and a body part; a processing and rendering unit that performs processing and rendering on the body parts excluding the face part that have been separated by the person detection unit; an image synthesis unit that creates a synthesized image of the image that has been subjected to the processing and drawing process and the photographed image, The composite image is transmitted to a display terminal via the network unit, the person detection unit compares face information of the plurality of detected persons with the face information registration data, and determines whether the plurality of detected persons are registered; A monitoring device characterized in that, if the result of the judgment is that all of the multiple detected people are registered, the processing and drawing unit performs processing and drawing processing on all of the multiple people, and if there is even one person among the multiple detected people who is not registered, the processing and drawing processing on all of the multiple people is not performed by the processing and drawing unit.

7. 7. The monitoring device according to claim 1, The monitoring device is characterized in that the processing and drawing unit performs coloring processing on body parts excluding the face.

8. 8. The monitoring device according to claim 1, The surveillance camera further has a function of measuring the distance to a subject to be photographed, The monitoring device is characterized in that the processing and rendering unit performs shading processing on body parts excluding the face in accordance with distance measurement data acquired by the monitoring camera.

9. an image acquisition step of acquiring a photographed image of the space; a person detection step of detecting a person from the photographed image and classifying the detected person into a face part and a body part; a processing and drawing step of performing processing and drawing on the body parts excluding the face part classified in the person detection step; an image synthesis step of creating a synthesized image of the image that has been subjected to the processing and drawing process and the photographed image; a transmitting step of transmitting the composite image to a display terminal; Equipped with A step of registering face information of a person to be monitored in advance in face information registration data, In the person detection step, face information of the plurality of detected persons is compared with the face information registration data to determine whether the plurality of detected persons are registered; This monitoring method is characterized in that, if the result of the judgment is that all of the multiple detected people are registered, the processing and drawing process is performed for all of the multiple people using the processing and drawing step, and if there is even one person among the multiple detected people who is not registered, the processing and drawing process is not performed for all of the multiple people using the processing and drawing step.

10. 10. The monitoring method according to claim 9, A step of registering the appearance of a suspicious person in advance as a suspicious person pattern in suspicious person pattern registration data; an abnormality processing step of creating alarm information and transmitting it to the display terminal, or of transmitting the captured image to the display terminal without processing or rendering the captured image, The monitoring method is characterized in that in the person detection step, the detected person is compared with the suspicious person pattern registration data, and if it is determined that the person corresponds to a suspicious person pattern, the abnormality processing step is executed.

11. 10. The monitoring method according to claim 9, In the image acquisition step, a plurality of photographed images of the space are acquired, A monitoring method characterized in that, if the person detection step determines that the same person is included in multiple captured images, the editing and drawing step performs a common editing and drawing process for the same person across the multiple captured images.

12. 12. The monitoring method according to claim 9, further comprising: A surveillance method characterized in that the processing and drawing step performs either or both of coloring processing on body parts excluding the face and shading processing on body parts according to distance measurement data to the subject.

Images