Monitoring device, monitoring program and monitoring method

Abstract

To monitor multiple people in real time so as to see whether they are behaving according to a predetermined rule in a space of a monitoring target.SOLUTION: The present invention relates to a monitoring device that monitors a space which serves as a monitoring target. Then, a monitoring device according to the present invention is characterized by comprising: individual behavior evaluating means for evaluating a behavior of each of a plurality of target persons shown in a monitoring image provided by a camera imaging a space which is a monitoring target, and for obtaining an individual behavior evaluation result; and comprehensive behavior evaluation means for determining whether or not a detection target behavior which is a detection target has been performed by the target person in the space, by comprehensively considering the individual behavior evaluation result obtained for the plurality of target persons by the individual behavior evaluating means.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a monitoring device, a monitoring program, and a monitoring method, and can be applied to, for example, monitoring of operators at ATMs (Automatic Teller Machines) in financial institutions and the like. [Background technology]

[0002] When transferring cash from an ATM installed at a financial institution, etc., access may be provided by opening a door at the rear of the ATM from a room located at the rear of the ATM (hereinafter referred to as the "ATM machine room").

[0003] Normally, the ATM machine room is locked by security guards or employees (attendants) of the financial institution when it is necessary to operate the ATM machine room. In addition, although there are usually surveillance cameras in the ATM machine room, they only record the images and do not systematically detect suspicious people or manage the number of people entering the room.

[0004] Conventionally, there are techniques disclosed in Patent Documents 1 to 3 and Non-Patent Document 1 as techniques for detecting suspicious individuals in a monitored room and managing the number of people who have entered the room.

[0005] Patent Document 1 describes the detection of suspicious behavior by video analysis using an AI model for detecting specific behavior. Patent Document 2 describes the detection of suspicious behavior based on whether the behavior of people in a monitored area is a previously registered behavior. Patent Document 3 describes the measurement of the number of people passing through a gate by image processing, and the control of not opening the door if the number does not match the number of authenticated people. Furthermore, Non-Patent Document 1 describes a door lock that requires authentication by multiple people to open it. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] International Publication WO 2022 / 059123 [Patent Document 2] Japanese Patent Application Publication No. 06-119564 [Patent Document 3] JP 2008-59179 A [Non-patent literature]

[0007] [Non-Patent Document 1] Enspiration Co., Ltd., "Require authentication by multiple people to unlock," [Online], INTERNET, [Retrieved August 17, 2023],<URL:http: / / lavish-lock.com / various / case9.html> Summary of the Invention [Problem to be solved by the invention]

[0008] Generally, in an ATM room, multiple workers (e.g., security guards and bank employees) perform tasks such as loading cash into the ATM, but there is a risk of fraud or accidents occurring due to the actions of workers who enter the room legitimately. For example, if workers who enter the room legitimately do not follow the prescribed rules when loading cash into the ATM, there is a high risk of fraud by those workers.

[0009] However, when the above-mentioned conventional technologies (for example, Patent Documents 1 to 3 and Non-Patent Document 1) are used, while it is possible to deter unauthorized intruders and detect individuals behaving suspiciously, it is not possible to monitor in real time whether multiple operators who have legitimately entered an ATM machine room are operating in accordance with predetermined rules.

[0010] In view of the above problems, there is a demand for a monitoring device, a monitoring program, and a monitoring method that can monitor in real time whether multiple people are behaving in accordance with predetermined rules in a monitored space (e.g., an ATM machine room). [Means for solving the problem]

[0011] The first surveillance device of the present invention is characterized in having an individual behavior evaluation means for evaluating the behavior of each of a number of subjects captured in surveillance footage supplied from a camera that captures the space to be monitored to obtain individual behavior evaluation results, and an overall behavior evaluation means for determining whether or not a subject has engaged in a detectable behavior within the space by taking into consideration, in a comprehensive manner, the individual behavior evaluation results obtained for the multiple subjects by the individual behavior evaluation means.

[0012] The second surveillance program of the present invention is characterized in that it causes a computer to function as an individual behavior evaluation means for evaluating the behavior of each of a number of subjects captured in surveillance footage supplied from a camera that captures the space to be monitored to obtain individual behavior evaluation results, and as a comprehensive behavior evaluation means for determining whether or not a subject has engaged in a detectable behavior within the space by taking into consideration, in a comprehensive manner, the individual behavior evaluation results obtained by the individual behavior evaluation means for a number of the subjects.

[0013] The third invention is a monitoring method performed by a monitoring device, the monitoring device comprising an individual behavior evaluation means and a comprehensive behavior evaluation means, the individual behavior evaluation means evaluating the behavior of each of a plurality of subjects captured in a surveillance video provided by a camera capturing an image of the space to be monitored to obtain an individual behavior evaluation result, and the comprehensive behavior evaluation means determining whether or not a detectable behavior to be detected has been performed by a subject within the space, taking into consideration in a comprehensive manner the individual behavior evaluation results obtained by the individual behavior evaluation means for the plurality of subjects.

[0014] In the first aspect of the present invention, when the comprehensive behavior evaluation means determines that the target behavior has been performed, it may notify an external device that the target behavior has been performed.

[0015] In addition, in the first present invention, the individual behavior evaluation means may be configured to extract an area in which each of the subjects is captured from the surveillance video to generate a subject image, and to obtain the individual behavior evaluation result for each of the subjects by evaluating each of the subject images.

[0016] Furthermore, in the first aspect of the present invention, the individual behavior evaluation means may obtain the individual behavior evaluation result by inputting the subject video to a classifier in which a pre-trained learning model is set.

[0017] In addition, in the first invention, the system may further include a learning processing means for performing a learning process based on a training subject image in which an area showing a human is cut out from a training image, and the individual behavior evaluation result that is the correct answer corresponding to the training subject image, to obtain the learning model. Effect of the Invention

[0018] According to the present invention, it is possible to monitor in real time whether a plurality of people in a monitored space are behaving in accordance with predetermined rules. [Brief description of the drawings]

[0019] [Figure 1] FIG. 2 is a block diagram showing a connection configuration of each device related to the embodiment (including a functional configuration of a monitoring device according to the embodiment). [Diagram 2] 1 is a diagram (image seen from above) illustrating an example of the layout relationship of devices in an ATM machine room that is a target of monitoring by a monitoring device of an embodiment. [Diagram 3] FIG. 2 is an image diagram showing an example of work rules for an operator in an ATM machine room according to the embodiment. [Figure 4] 11 is a diagram illustrating an example of a process in which a detection processing unit according to an embodiment extracts a target person video from surveillance video data. FIG. [Diagram 5] 11 is a diagram showing a process of individual behavior evaluation by a detection processing unit according to the embodiment. FIG. [Figure 6] FIG. 13 is a diagram showing a list of individual behaviors (individual behavior evaluation labels) classified by a classifier according to the embodiment. [Figure 7] 4 is a diagram showing an example of the configuration of detection definition data stored in a detection definition data storage unit according to the embodiment; FIG. [Figure 8] 10A to 10C are diagrams illustrating an example of a learning process performed by a learning processing unit according to the embodiment. [Figure 9] 11 is a flowchart illustrating an example of a learning process performed by a detection processing unit according to the embodiment. [Figure 10] 11 is a flowchart illustrating an example of a target behavior detection process based on surveillance video data by a detection processing unit according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] (A) Main embodiment Hereinafter, an embodiment of a monitoring device, a monitoring program, and a monitoring method according to the present invention will be described in detail with reference to the drawings.

[0021] (A-1) Configuration of the embodiment FIG. 1 is a block diagram showing the connection configuration of each device related to this embodiment (including the functional configuration of a monitoring device 10 according to the embodiment).

[0022] The monitoring device 10 is a device that monitors for the presence or absence of abnormalities, etc. in the ATM machine room, which is the space to be monitored, based on data obtained from cameras, etc. installed in the ATM machine room, and notifies (reports) any abnormalities, etc. detected through predetermined processing.

[0023] In this embodiment, a notification from the monitoring device 10 is provided to a monitoring terminal 20 arranged in a monitoring center. The monitoring terminal 20 is a terminal used by an operator OP. As the monitoring terminal 20, various information processing terminals (computers) such as a PC and a tablet PC can be applied.

[0024] The monitoring device 10 and the monitoring terminal 20 in this embodiment are connected to a network N, and are capable of connecting to each other via the network N.

[0025] FIG. 2 is a diagram (image seen from above) showing an example of the layout of devices in an ATM machine room that is the target of monitoring by the monitoring device 10 of this embodiment.

[0026] As shown in FIG. 2, three ATMs 70-1 to 70-3 are arranged in the ATM machine room. The ATMs 70-1 to 70-3 are provided with a front panel 71 for dealing with customers and a rear panel 72 that can be accessed by workers for various operations (e.g., loading cash, maintenance, inspection, etc.). The rear panels 72 of all the ATMs 70 are accessible only from the ATM machine room. In other words, workers need to enter the ATM machine room to access the rear panels 72 of the ATMs 70-1 to 70-3 to perform operations. As shown in FIG. 2, a door D is arranged at the only entrance to the ATM machine room, and the door D is provided with a door lock device 60 that can be electrically locked / unlocked.

[0027] In Fig. 2, two cameras 30-2 and 30-3 are arranged in the ATM machine room. A speaker 40 and a microphone 50 are also arranged in the ATM machine room. Furthermore, a camera 30-1 is arranged outside the ATM machine room to monitor people entering and exiting door D. Hereinafter, the position where this camera 30-1 is arranged will also be referred to as "in front of the ATM machine room." Note that there is no limitation on the positions and number of cameras 30 arranged in the ATM machine room, which is the space to be monitored.

[0028] As shown in FIG. 1, the cameras 30-1 to 30-3, the speaker 40, the microphone 50, and the door lock device 60 are connected to a network N.

[0029] Each camera 30 is capable of transmitting captured video data (hereinafter referred to as "monitoring video data") to the monitoring device 10 and the monitoring terminal 20 via a network. As the camera 30, cameras connectable to various networks can be used.

[0030] The speaker 40 and the microphone 50 are capable of audio communication (for example, telephone communication) with the monitoring terminal 20 via the network N.

[0031] In this embodiment, it is assumed that there are predetermined work rules (for example, work rules determined by a financial institution or the like that owns the ATM 70) for the work performed by an operator in the ATM machine room. In other words, it is assumed that the policy is such that even an operator who has entered the ATM machine room legitimately is not permitted to take any action that violates the work rules (hereinafter, referred to as "illegal action").

[0032] FIG. 3 is a conceptual diagram showing an example of work rules for workers in an ATM machine room.

[0033] Figure 3 shows an example of a surveillance video in which two workers, W1 and W2, are present in an ATM machine room. Figure 3(a) shows the two workers, W1 and W2, behaving in accordance with the work rules, and Figure 3(b) shows the two workers, W1 and W2, behaving in violation of the work rules (irregular behavior).

[0034] Here, it is assumed that there is a work rule that two workers must enter the ATM machine room at the same time, one of them must perform work, and the other must watch (gazing at) the other worker's work. In this case, in the surveillance video of FIG. 3(a), the first worker W1 is performing work on the ATM 70, and the second worker W2 is watching the first worker W1's work (directing his / her gaze at the first worker W1), so it can be said that the two workers W1 and W2 are acting in accordance with the work rule. On the other hand, in the surveillance video of FIG. 3(b), the first worker W1 is performing work on the ATM 70, but the second worker W2 is walking away without watching the first worker W1's work, so the two workers W1 and W2 are acting in violation of the work rule.

[0035] The monitoring device 10 monitors the behavior of people present in the ATM machine room, which is the space to be monitored, based on the monitoring video data, detects the behavior to be detected (hereinafter referred to as "detection target behavior") as abnormal, and notifies the monitoring terminal 20 of the detected abnormality. The detection target behavior also includes behavior that violates the work rules as shown in Fig. 3(b).

[0036] In addition, in the ATM machine room monitored by the monitoring device 10, as described above, there is a work rule that two workers must enter and leave the room together. In this embodiment, in order to make the workers comply with the work rule, the door lock device 60 is configured not to unlock the door unless the two workers are together in front of the door lock device 60 (in front of the ATM machine room). For example, the door lock device 60 may be configured not to unlock the door unless it is recognized that the two workers are together by an authentication process for the two workers (for example, authentication by biometric authentication or image sensing of the video from the camera 30). Specifically, for example, when the worker enters the ATM machine room, the door lock device 60 may sense the two workers based on the surveillance video acquired from the camera 30-1 in front of the ATM machine room and only unlock the door if the authentication is successful. Also, for example, when the worker leaves the ATM machine room, the door lock device 60 may sense the two workers based on the surveillance video acquired from the cameras 30-2 and 30-3 in the ATM machine room and only unlock the door if the authentication is successful. The authentication process and unlocking control process in the door lock device 60 may be performed within the door lock device 60, or may be performed by the monitoring device 10 or another computer on another network N (e.g., a computer on the cloud).

[0037] Next, the internal configuration of the monitoring device 10 will be described.

[0038] The monitoring device 10 includes a data acquisition unit 11, a detection unit 12, and a notification unit 13.

[0039] The monitoring device 10 may be configured entirely as hardware, or may be configured partially or entirely as software (configured by a computer program). For example, the monitoring device 10 may be configured by installing a program (a monitoring program according to an embodiment) in a computer having a memory and a processor.

[0040] Furthermore, the physical location at which the monitoring device 10 is placed is not limited, and it may be placed in the ATM machine room or at any connection location on the network N (e.g., in a data center or on the cloud).

[0041] The data acquisition unit 11 has a function of acquiring data (monitoring video data) supplied from the camera 30 and supplying it to the detection unit 12. In this embodiment, it is assumed that the monitoring video data of the cameras 30-2 and 30-3 arranged in the ATM machine room is supplied to the monitoring device 10.

[0042] The detection unit 12 detects the behavior of a person (e.g., an operator or an entrant; hereinafter, simply referred to as a "subject") whose behavior is to be detected in a space to be monitored (inside an ATM machine room in this embodiment) based on the surveillance video data supplied from the data acquisition unit 11, and monitors whether or not the subject has engaged in a detectable behavior. When the detection unit 12 detects a detectable behavior of the subject, it notifies the notification unit 13. The detection unit 12 includes a determiner 122a for evaluating (classifying) the behavior of the subject shown in the surveillance video data. The detection unit 12 sets a learning model related to the detectable behavior in the determiner 122a, and evaluates (classifies) each behavior (behavior) based on the surveillance video data.

[0043] The notification unit 13 performs a process of notifying (reporting) the target behavior (abnormality) detected by the detection unit 12 through a predetermined process. In this embodiment, the notification unit 13 performs a process of notifying the monitoring terminal 20 that the target behavior has been detected for a target person in the ATM machine room (hereinafter, this notification will be referred to as an "abnormality notification"). When an abnormality notification is received, the monitoring terminal 20 performs a process of outputting the abnormality notification to the operator OP. The method of outputting the abnormality notification to the operator OP in the monitoring terminal 20 is not limited, and various methods (for example, output from a display or audio output from a speaker, etc.) are supported.

[0044] Next, the internal configuration of the detection unit 12 will be described.

[0045] The detection unit 12 includes a learning processing unit 121 , a detection processing unit 122 , and a detection definition data storage unit 123 .

[0046] The detection processing unit 122 performs a process (hereinafter referred to as a "detection process of target behavior") of detecting a target behavior by a target person in the ATM machine room based on data (monitoring video data) supplied from the data acquisition unit 11. Specifically, the detection processing unit 122 cuts out a video of each of the multiple targets from the monitoring video data, and uses the determiner 122a in which a learning model is set for the cut-out video of each target person (hereinafter referred to as a "target person video") to individually evaluate (hereinafter referred to as an "individual behavior evaluation") the behavior (movement) of each target person, and further comprehensively evaluates (hereinafter referred to as an "overall behavior evaluation") the results of the individual evaluations of each target person (hereinafter referred to as an "individual behavior evaluation label") to detect the presence or absence of a target behavior. The detection processing unit 122 performs the overall behavior evaluation according to data (hereinafter referred to as "detection definition data") registered in the detection definition data storage unit 123.

[0047] Note that the process of extracting the subject image of each subject from the surveillance video data can be similar to the image recognition process that detects various types of humans (extracts the area in which humans are shown), so a detailed explanation will be omitted. Also, for the determiner 122a and the learning model used for the determiner 122a, various AI (Artificial Intelligence) platforms capable of evaluating (classifying) images can be applied.

[0048] The detection processing unit 122 performs the above-mentioned target behavior detection process for each camera 30 (each piece of monitoring video data). For example, the detection processing unit 122 may perform the target behavior detection process each time a certain period of monitoring video data for each camera 30 is accumulated. In other words, the detection processing unit 122 may perform the target behavior detection process for a certain period of monitoring video data as one processing unit.

[0049] FIG. 4 is a diagram showing an example of a process in which the detection processing unit 122 extracts a target person video from the surveillance video data.

[0050] For example, in the case where the surveillance video data includes an image as shown in Fig. 4, the detection processing unit 122 cuts out an image GW1 of a bounding box of an area in which a first worker (subject) W1 is shown and an image GW2 of a bounding box of an area in which a second worker (subject) W2 is shown as subject images. The detection processing unit 122 cuts out and acquires each frame of the surveillance video data that is the subject of the target behavior detection process as a subject image.

[0051] Next, the individual behavior evaluation performed by the detection processing unit 122 will be described.

[0052] FIG. 5 is a diagram showing the process of individual behavior evaluation by the detection processing unit 122. As shown in FIG.

[0053] As shown in FIG. 5, the detection processing unit 122 inputs a subject video to a determiner 122a in which a learning model LM is set, and obtains an identifier (hereinafter referred to as a "label ID") of a label (hereinafter referred to as an "individual behavior evaluation label") indicating the result of classifying the behavioral content of the subject shown in the subject video.

[0054] FIG. 6 is a diagram showing a list of individual behaviors (individual behavior evaluation labels) classified by the classifier 122a.

[0055] In FIG. 6, individual action names are shown for each label ID.

[0056] In the list in Figure 6, the individual action name for label ID "1" is "Working" (for example, the subject is working on something such as loading cash into ATM 70), the individual action name for label ID "2" is "Looking at another subject" (for example, looking at another subject who is working on ATM 70), the individual action name for label ID "3" is "Gesture of checking camera position", the individual action name for label ID "4" is "Gesture of touching cash", the individual action name for label ID "5" is "Gesture of putting cash in pocket", ...

[0057] For example, if the subject video includes an image such as image GW1 in Fig. 4, the determiner 122a will output the label ID "1" (working) as the determination result. Also, for example, if the subject video includes an image such as image GW2 in Fig. 4, the determiner 122a will output the label ID "2" (looking at the other subject) as the determination result.

[0058] In addition, when the input subject video cannot be classified into any individual behavior evaluation label (label ID) (for example, when the reliability of the classification result is equal to or lower than a predetermined value), the determiner 122a may output a determination result indicating that classification is impossible (for example, outputting 0 as the label ID).

[0059] Next, the overall behavior evaluation performed by the detection processing unit 122 will be described.

[0060] As described above, the detection processing unit 122 attempts to detect the target behavior in the ATM machine room by acquiring the individual behavior evaluations of the two subjects and subjecting the individual behavior evaluations of the two subjects to an overall behavior evaluation in accordance with the detection definition data.

[0061] FIG. 7 is a diagram showing an example of the configuration of detection definition data stored in the detection definition data storage unit 123. As shown in FIG.

[0062] As shown in Fig. 7, the detection definition data registers a "detection ID" (identifier of the behavior to be detected), a "name of the behavior to be detected," and a "behavior detection method" for each behavior to be detected. In other words, in the detection definition data shown in Fig. 7, one detection target behavior is defined in one line.

[0063] The "behavior detection method" item is data describing a method for comprehensively evaluating the individual behavioral evaluations of each subject and detecting the target behavior corresponding to the target behavior name (i.e., a comprehensive evaluation method). In FIG. 7, for ease of explanation, an outline of the behavior detection method is described in text (natural language), but in reality, a program, script, etc. corresponding to the outline (text) as described in FIG. 7 is registered in the behavior detection method. The detection processing unit 122 can detect behavior corresponding to the target behavior name by performing processing according to the program, script, etc. set in the behavior detection method.

[0064] 7, for the management number "1," the name of the detection target behavior is [gesture of checking camera position], and the behavior detection method is [detection that at least one of the subjects performed the behavior of "gesture of checking camera position."] Therefore, when the individual behavior evaluation of at least one of the first subject or the second subject is "gesture of checking camera position," the detection processing unit 122 detects the detection target behavior of detection ID: 1 (gesture of checking camera position).

[0065] 7, for example, for the management number "2," the name of the detection target behavior is [gesture of touching cash], and the behavior detection method is [detection of at least one of the subjects performing the behavior of 'gesture of touching cash']. Therefore, when the individual behavior evaluation of at least one of the first subject or the second subject is "gesture of touching cash," the detection processing unit 122 detects the detection target behavior of detection ID: 2 (gesture of touching cash).

[0066] 7, for example, for the management number "3," the detection target behavior name is [gesture of putting cash in pocket], and the behavior detection method is [detection of at least one of the subjects performing the behavior of 'gesture of putting cash in pocket.'] Therefore, when the individual behavior evaluation of at least one of the first subject or the second subject is the "gesture of putting cash in pocket," the detection processing unit 122 detects the detection target behavior of detection ID: 3 (gesture of putting cash in pocket).

[0067] 7, for example, for the management number "4," the name of the behavior to be detected is [one of the two people is not looking at the target], and the behavior detection method is [detection of a state in which at least one of the targets is not performing the behavior of "looking at the other target" for a certain period of time or more]. Therefore, if the individual behavior evaluations of both the first target and the second target are not "looking at the other target," the detection processing unit 122 will detect the behavior to be detected with detection ID: 4 (one of the two people is not looking at the target).

[0068] 7, for example, for the management number "5," the name of the detection target behavior is "detection of at least one of the subjects continuing the behavior of 'working' for a certain period of time or more." Therefore, when the individual behavior evaluation of either the first subject or the second subject continues to be "working for a certain period of time or more" for a certain period of time or more, the detection processing unit 122 detects the detection target behavior of detection ID: 5 (working for a certain period of time or more).

[0069] Next, the learning process performed by the learning processing unit 121 will be described.

[0070] FIG. 8 is a diagram showing an example of the learning process performed by the learning processing unit 121. As shown in FIG.

[0071] As shown in FIG. 8, the learning processing unit 121 cuts out the subject video of each subject that appears in the learning video data by the same process as the detection processing unit 122, inputs the subject video together with the label ID (correct label) of the corresponding individual behavior evaluation label to the learning device 121a for learning, and generates a learning model LM. The method of assigning the correct label to each subject video is not limited. For example, the learning processing unit 121 may accept the user's manual assignment of the correct label to each subject video cut out from the learning video data. In addition, when only one subject appears in the learning video data, the learning video data itself may be assigned a correct label and the learning processing unit 121 may accept the input. Furthermore, the learning processing unit 121 may be configured to receive an input of a pair of the subject video (already cut out video) and the correct label from the beginning, and input it directly to the learning device 121a for learning (in this case, pre-processing is not performed in the learning processing unit 121).

[0072] The learning video data may be, for example, surveillance video data when an incident actually occurs in an ATM machine room (when an abnormality occurs) (hereinafter referred to as "actual video data when an abnormality occurs") or surveillance video data during normal operation (when the behavior to be detected does not occur). In this embodiment, in addition to actual video of the ATM machine room, a sample video may be separately created (for example, created by shooting a video of an actor performing the role of a target person) and used as the learning video data. In other words, the learning video data may be actual surveillance video data or video data shot for learning purposes.

[0073] It should be noted that the same AI platform as that of the determiner 122a can be applied to the learning processing unit 121 and the learning device 121a.

[0074] (A-2) Operation of the embodiment Next, the operation of the monitoring device 10 of this embodiment (the procedure of the monitoring method according to the embodiment) will be described.

[0075] First, the learning process performed by the learning processing unit 121 will be described.

[0076] FIG. 9 is a flowchart showing an example of the learning process performed by the detection processing unit 122.

[0077] Here, it is assumed that a plurality of pieces of learning video data are input to the learning processing unit 121, and one of the input learning video data is acquired by the learning processing unit 121 (S101).

[0078] Next, the learning processing unit 121 performs pre-processing on the acquired learning video data (S102). Specifically, the learning processing unit 121 performs processing to cut out a subject image of each subject appearing in the acquired learning video data, and further to acquire a correct label (label ID of an individual behavior evaluation label) corresponding to each cut out subject image (S103).

[0079] Next, the learning processing unit 121 supplies each of the acquired pairs of the subject video and the correct label to the learning device 121a to cause it to execute learning processing (S103).

[0080] Next, the learning processing unit 121 checks whether or not there is any unprocessed learning video data among the input learning video data (S104), and if unprocessed learning video data remains, it returns to the above-mentioned step S101 and proceeds to processing the next learning video data, otherwise it supplies the learning model LM generated by the learning process to the detection processing unit 122 (sets it in the determiner 122a) (S105) and ends the processing.

[0081] Next, a detection target behavior detection process based on monitoring video data by the detection processing unit 122 will be described.

[0082] FIG. 10 is a flowchart showing an example of a target behavior detection process performed by the detection processing unit 122 based on surveillance video data.

[0083] The flowchart in Fig. 10 shows an example of a case where a target behavior detection process is performed on one processing unit of monitoring video data. That is, the detection processing unit 122 performs the process of the flowchart in Fig. 10 (target behavior detection process) every time one processing unit of monitoring video data is supplied.

[0084] First, it is assumed that one processing unit of surveillance video data is acquired by the detection processing unit 122 (S201).

[0085] Next, the detection processing unit 122 performs a pre-processing to cut out the target person images of each target person included in the acquired surveillance video data (S202).

[0086] Next, the detection processing unit 122 checks whether or not the extraction of the target image in step S202 was successful (whether or not the target was captured in the surveillance video data) (S203), and if the extraction of the target image was successful, proceeds to step S204 described below, and if the extraction of the target image was unsuccessful, terminates the processing.

[0087] When the extraction of the subject video from the surveillance video data is successful, the detection processing unit 122 inputs each subject video to the determiner 122a and acquires the result of the individual behavior evaluation (the label ID of the individual behavior evaluation label) (S204).

[0088] Next, the detection processing unit 122 performs a comprehensive behavior evaluation on the results of the acquired individual behavior evaluations (S205). Specifically, the detection processing unit 122 performs a comprehensive behavior evaluation on the individual behavior evaluations of the two subjects according to the detection definition data, thereby attempting to detect the detection target behavior in the ATM machine room.

[0089] Next, the detection processing unit 122 checks whether or not one or more target detection behaviors have been detected as a result of the overall behavior evaluation (S206), and if one or more target detection behaviors have been detected, supplies the detected target detection behaviors (detection IDs) to the notification unit 13 (S207), and if not, terminates the processing.

[0090] As described above, the detection processing unit 122 performs the detection target behavior detection process.

[0091] When information (detection ID) of the detected detection target behavior is supplied from the detection processing unit 122 to the notification unit 13, the notification unit 13 notifies the monitoring terminal 20 that the detection target behavior has occurred (anomaly has occurred).

[0092] This allows the operator OP using the monitoring terminal 20 to be aware of the occurrence of an abnormality in the ATM machine room. This allows the operator OP to check the situation in the ATM machine room from the images of the cameras 30-2 and 30-3, or to talk to a person (worker) in the ATM machine room by phone via the speaker 40 / microphone 50 to check the situation.

[0093] (A-3) Effects of the embodiment According to this embodiment, the following effects can be obtained.

[0094] In the monitoring device 10 (detection processing unit 122) of this embodiment, the behavior of each subject is evaluated by individual behavior evaluation, and a comprehensive behavior evaluation is performed on the results of the individual behavior evaluations of multiple subjects, and the detection target behavior (suspicious behavior that violates work rules) is detected based on the result. For example, simply performing individual behavior evaluation cannot detect the behavior shown in Fig. 3(b) as a detection target, but the monitoring device 10 (detection processing unit 122) of this embodiment performs a comprehensive behavior evaluation, so it can also detect patterns of combinations of behaviors of multiple people.

[0095] (B) Other embodiments The present invention is not limited to the above-described embodiments, and may include modified embodiments such as those exemplified below.

[0096] (B-1) In the above embodiment, the detection processing unit 122 performs the target behavior detection process based only on the monitoring video data from the camera 30, but the target behavior detection process may be performed based on other media besides video. For example, the detection processing unit 122 may perform the target behavior detection process based on audio data (hereinafter referred to as "recorded data") supplied from a microphone 50 that records audio in the ATM machine room. For example, the detection processing unit 122 may be configured to hold a learning model that has previously learned sounds generated during normal work (hereinafter referred to as "work sounds"), input the recorded data into the learning model and make a judgment to detect abnormal sounds that do not occur during normal work, and when an abnormal sound is detected, notify the monitoring terminal 20 that the target behavior has been detected.

[0097] (B-2) In the above embodiment, the monitoring device 10 performs target behavior detection processing on surveillance video data supplied from the camera 30 installed in the ATM machine room. However, it goes without saying that the video data that is the subject of the target behavior detection processing is not limited to that of the ATM machine room, and may be video data of any space where target behavior detection processing is required.

[0098] (B-3) In the monitoring device 10 of the above embodiment, the learning processing unit 121 is included, but when learning processing is not performed (when a pre-prepared learning model is used), the configuration may exclude the learning processing unit 121.

[0099] (B-4) In the monitoring device 10 of the above embodiment, processing is performed under the assumption that the work rule requires that the number of subjects entering the ATM machine room is always two, but it may be adapted to handle work rules with three or more subjects. In that case, it is necessary to describe in the detection definition data in the detection definition data storage unit 123 a behavior detection method that combines individual behavior evaluations of three or more subjects to perform a comprehensive behavior evaluation.

[0100] (B-5) The monitoring device 10 in the above embodiment includes the notification unit 13 and notifies an external device (the monitoring terminal 20 in the above embodiment) of the occurrence of a behavior to be detected (occurrence of an abnormality), but the configuration may be such that the notification unit 13 is excluded. In this case, the monitoring device 10 may, for example, simply record (for example, record in any data recording medium) the detection result of the detection processing unit 122. [Explanation of symbols]

[0101] 10...monitoring device, 11...data acquisition unit, 12...detection unit, 13...notification unit, 20...monitoring terminal, 30, 30-1, 30-2, 30-3...camera, 40...speaker, 50...microphone, 60...door lock device, 71...front panel, 72...rear panel, 121...learning processing unit, 121a...learning device, 122...detection processing unit, 122a...determinator, 123...detection definition data storage unit, LM...learning model, N...network, OP...operator, W1, W2...subject (worker)

Claims

1. an individual behavior evaluation means for evaluating the behavior of each of a plurality of subjects captured in a surveillance video provided by a camera capturing an image of a space to be monitored, and obtaining an individual behavior evaluation result; a comprehensive behavior evaluation means for determining whether or not a detection target behavior that is a detection target has been performed by the subject in the space by comprehensively considering the individual behavior evaluation results for the plurality of subjects obtained by the individual behavior evaluation means; A monitoring device comprising:

2. The monitoring device according to claim 1, characterized in that, when the comprehensive behavior evaluation means determines that the target behavior has been performed, an external device is notified that the target behavior has been performed.

3. The monitoring device described in claim 1, characterized in that the individual behavior evaluation means extracts an area in which each of the subjects is visible from the surveillance video to generate a subject image, and evaluates each of the subject images to obtain the individual behavior evaluation result for each of the subjects.

4. The monitoring device according to claim 3, characterized in that the individual behavior evaluation means obtains the individual behavior evaluation result by inputting the subject's video to a classifier in which a pre-trained learning model is set.

5. The monitoring device according to claim 4, further comprising a learning processing means for performing a learning process based on a training subject image in which an area showing a human is cut out from a training image and the individual behavior evaluation result that is the correct answer corresponding to the training subject image, to obtain the learning model.

6. Computer, an individual behavior evaluation means for evaluating the behavior of each of a plurality of subjects captured in a surveillance video provided by a camera capturing an image of a space to be monitored, and obtaining an individual behavior evaluation result; a comprehensive behavior evaluation means for determining whether or not a detection target behavior that is a detection target has been performed by the subject in the space by comprehensively considering the individual behavior evaluation results for the plurality of subjects obtained by the individual behavior evaluation means; A monitoring program characterized by functioning as a

7. A monitoring method performed by a monitoring device, The monitoring device includes an individual behavior evaluation means and a comprehensive behavior evaluation means, the individual behavior evaluation means evaluates the behavior of each of a plurality of subjects captured in a surveillance video provided from a camera capturing an image of a space to be monitored, to obtain an individual behavior evaluation result; The comprehensive behavior evaluation means determines whether or not a detection target behavior to be detected has been performed by the subject in the space, taking into consideration the individual behavior evaluation results for the plurality of subjects obtained by the individual behavior evaluation means in a comprehensive manner. A monitoring method comprising:

Images