Processing device, processing method, and program

Abstract

A processing device (10) according to the present invention comprises: an action analysis unit (11) that detects that a target person shown in an image obtained by imaging a person within a facility is performing any of a plurality of detection-target actions; a determination unit (12) that determines a notification destination on the basis of at least one from among the type of a detected detection-target action, a location at which the detected detection-target action is performed, the length of time for which the detected detection-target action has been performed, a time at which the detected detection-target action is performed, and the confidence of a detected result; and a notification unit (13) that sends, to the determined notification destination, a detection notification of the detection-target action.

Description

[Technical Field]

[0001] The present invention relates to a processing device, a processing method, and a recording medium. [Background technology]

[0002] Techniques related to the present invention are disclosed in Patent Documents 1 to 3 and Non-Patent Document 1.

[0003] Patent Document 1 discloses a technology that analyzes an image of an ATM (Automatic Teller Machine) user, and if it is determined that the user is or is likely to be a victim of a bank transfer fraud, notifies a monitoring center of that fact. It also discloses that the notification recipient is selected depending on the likelihood of victimization.

[0004] Patent Document 2 discloses a technology that analyzes an image of an ATM user and issues an alarm if it is detected that the user is in a talking position.

[0005] Patent Document 3 discloses a technology that calculates the feature values ​​of each of multiple key points of a human body contained in an image, searches for images containing human bodies with similar postures or movements based on the calculated feature values, and classifies images with similar postures or movements together.

[0006] Non-Patent Document 1 discloses a technique related to human skeleton estimation. [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Patent Publication No. 2010-238204 [Patent Document 2] Patent Publication No. 2010-176531 [Patent Document 3] International Publication No. 2021 / 084677 [Non-patent literature]

[0008] [Non-Patent Document 1] Zhe Cao, Tomas Simon, Shih-En Wei, Yaser Sheikh, "Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields", The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, P. 7291-7299 Summary of the Invention [Problem to be solved by the invention]

[0009] Image analysis can detect people engaged in various activities, such as talking on a mobile phone, using a wheelchair, or using a white cane. By notifying the appropriate contact point when a person engaging in a specific activity is detected, incidents can be prevented and the quality of services can be improved.

[0010] The method disclosed in Patent Document 1 for selecting notification recipients according to the likelihood of harm cannot be used in cases other than those likely to be harmed. In other words, there is a problem that the usage scenarios are limited. Patent Documents 2 and 3 and Non-Patent Document 1 do not disclose the problem of notifying appropriate information to appropriate notification recipients or the means for achieving this.

[0011] In view of the above-mentioned problems, one example of the object of the present invention is to provide a processing device, a processing method, and a recording medium that solve the problem of notifying an appropriate notification destination upon detection of a person performing a specified behavior. [Means for solving the problem]

[0012] According to one aspect of the present invention, a behavior analysis means for detecting whether a target person captured in an image of a person in the facility is performing one of a plurality of detection target behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; A processing device is provided having:

[0013] According to one aspect of the present invention, The computer Detects whether a target person in an image of a person in a facility is performing one of a plurality of detection target actions, determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the certainty of the detection result; notifying the determined notification destination of the detection of the target behavior; A processing method is provided.

[0014] According to one aspect of the present invention, Computer, behavior analysis means for detecting whether a target person captured in an image of a person in the facility is performing one of a plurality of detection target behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; A recording medium is provided on which a program for causing the device to function as the above is recorded. [Effects of the Invention]

[0015] According to one aspect of the present invention, a processing device, a processing method, and a recording medium are provided that solve the problem of notifying an appropriate notification destination in response to the detection of a person performing a predetermined behavior. [Brief explanation of the drawings]

[0016] The above and other objects, features and advantages are described below. Suitable This will become more apparent from the following embodiments and the accompanying drawings.

[0017] [Figure 1] FIG. 2 is a diagram illustrating an example of a functional block diagram of a processing device. [Figure 2] FIG. 1 is a diagram illustrating an example of a functional block diagram of a processing system. [Figure 3] FIG. 1 is a diagram showing a specific example of a functional block diagram of a processing system. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of a processing device. [Figure 5] FIG. 10 is a diagram for explaining the processing of a behavior analysis unit. [Figure 6] FIG. 2 is a diagram schematically illustrating an example of information processed by a processing device. [Figure 7] 10 is a flowchart illustrating an example of a processing flow of the processing device. [Figure 8] FIG. 2 is a diagram schematically illustrating an example of information processed by a processing device. [Figure 9] 10 is a flowchart showing an example of a processing flow of the processing device. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and their description will be omitted where appropriate.

[0019] First Embodiment 1 is a functional block diagram showing an overview of a processing device 10 according to the first embodiment. The processing device 10 includes a behavior analysis unit 11, a determination unit 12, and a notification unit 13.

[0020] The behavior analysis unit 11 detects whether a target person captured in an image of a person in a facility is engaging in one of a plurality of target behaviors. The determination unit 12 determines a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the confidence level of the detection result. The notification unit 13 notifies the determined notification destination of the detection of the target behavior.

[0021] The processing device 10 having such a configuration solves the problem of notifying an appropriate notification destination in response to the detection of a person performing a predetermined action.

[0022] <Second embodiment> "overview" The processing device 10 of the second embodiment is a more specific version of the processing device 10 of the first embodiment.

[0023] "Overall view of the processing system including the processing device 10" An example of a functional block diagram of the processing system is shown in Fig. 2. As shown in Fig. 2, the processing system includes a processing device 10, a camera 30, and a notification destination terminal 40.

[0024] The camera 30 is installed within a facility. The facility may be a bank, post office, supermarket, convenience store, department store, amusement park, building, station, airport, or other facility, and the details are not particularly limited. Although multiple cameras 30 are shown in FIG. 2, one camera 30 may be installed within the facility, or multiple cameras 30 may be installed within the facility. The camera 30 may be installed, for example, at the entrance / exit of the facility, in a location within the facility where a specific piece of equipment is installed, in an area leading to a staircase, or the like. Note that the installation locations of the camera 30 illustrated here are merely examples and are not limited to these. The camera 30 may capture moving images or may capture still images at predetermined timings.

[0025] The processing device 10 analyzes the image generated by the camera 30 and detects whether the target person in the image is performing one of a plurality of predefined detection target behaviors. Next, the processing device 10 determines a notification destination of the detection result based on at least one of the type of the detected detection target behavior, the location where the detected detection target behavior is being performed, the length of time the detected detection target behavior is being performed, the time when the detected detection target behavior is being performed, and the confidence level of the detection result. The processing device 10 then notifies the determined notification destination of the detection result. One of a plurality of notification destination terminals 40 is determined as the notification destination. Details of the processing by the processing device 10 will be described later.

[0026] The notification destination terminal 40 is a terminal to which the detection result is notified. The notification destination terminal 40 may be, but is not limited to, a smartphone, a tablet terminal, a mobile phone, a personal computer, a dedicated terminal, digital signage, etc. The processing device 10 transmits the detection result to the notification destination terminal 40 of the determined notification destination. The notification destination terminal 40 outputs predetermined information in response to the notification from the processing device 10.

[0027] Next, a specific example of the processing system of the second embodiment will be described with reference to Fig. 3. Note that the specific example shown in Fig. 3 is merely an example, and the configuration of the processing system is not limited to this.

[0028] The facility in question is a facility where an ATM (Automatic Teller Machine) is installed, and examples thereof include, but are not limited to, banks, post offices, convenience stores, supermarkets, department stores, etc.

[0029] A camera 30 is installed at each ATM and photographs users of each ATM. FIG. 3 shows a first ATM camera 30-1 and a second ATM camera 30-2. The first ATM camera 30-1 photographs users of the first ATM. The second ATM camera 30-2 photographs users of the second ATM. Although FIG. 3 shows two ATM cameras 30, the number is not limited to this.

[0030] Camera 30 may also be installed at a position to capture images of people in other locations within the facility, such as, but not limited to, an entrance camera 30-3 that captures images of people near the entrance to the facility, or a digital signage camera 30-4 that captures images of people near digital signage installed in the facility.

[0031] As shown in Figure 3, the notification destination terminal 40 can include at least one of a facility worker terminal 40-1, a security terminal 40-2, a processing device manager terminal 40-3, a first ATM terminal 40-4, a second ATM terminal 40-5, a digital signage 40-6, and a processing device provider terminal 40-7.

[0032] The facility worker terminal 40-1 is a terminal used by facility workers (workers who serve visitors, etc.). The facility workers are notified of the detection results of the processing device 10, i.e., that a person performing a predetermined detection target behavior has been detected, via the facility worker terminal 40-1. The facility worker terminal 40-1 is a smartphone, tablet terminal, mobile phone, personal computer, dedicated terminal, etc. The detection results may be transmitted from the processing device 10 to the facility worker terminal 40-1. Alternatively, the detection results may be transmitted from the processing device 10 to a facility server, and then transmitted from the facility server to the facility worker terminal 40-1. Alternatively, the detection results of the processing device 10 may be transmitted to the facility worker terminal 40-1 via another route.

[0033] The security terminal 40-2 is a terminal used by security guards and personnel of a security organization (such as a security company). The security guards and personnel of a security organization (such as a security company) are notified of the detection results of the processing device 10, i.e., that a person performing a predetermined detection target behavior has been detected, via the security terminal 40-2. The security terminal 40-2 is a smartphone, tablet terminal, mobile phone, personal computer, dedicated terminal, or the like. The detection results may be transmitted from the processing device 10 to the security terminal 40-2. Alternatively, the detection results may be transmitted from the processing device 10 to a server of the security organization (such as a security company), and then transmitted from the server of the security organization (such as a security company) to the security terminal 40-2. The detection results of the processing device 10 may also be transmitted to the security terminal 40-2 via another route.

[0034] The processing device administrator terminal 40-3 is a terminal used by an administrator (such as a facility staff member) who manages (maintains, repairs, etc.) the processing device 10. The administrator who manages the processing device 10 is notified of the detection result of the processing device 10, i.e., that a person performing a predetermined detection target behavior has been detected, via the processing device administrator terminal 40-3. The processing device administrator terminal 40-3 is a smartphone, tablet terminal, mobile phone, personal computer, dedicated terminal, etc. The detection result may be transmitted from the processing device 10 to the processing device administrator terminal 40-3. Alternatively, the detection result may be transmitted from the processing device 10 to a server of an organization to which the administrator who manages the processing device 10 belongs, and the detection result may be transmitted from the server to the processing device administrator terminal 40-3. Alternatively, the detection result of the processing device 10 may be transmitted to the processing device administrator terminal 40-3 via another route.

[0035] The first ATM terminal 40-4 and the second ATM terminal 40-5 are terminals that send notifications to ATM users. A notification destination terminal 40 is provided for each ATM. The first ATM terminal 40-4 and the second ATM terminal 40-5 may be the ATM itself or an output device installed near the ATM. Examples of the output device include a speaker, a display, and a warning lamp. The detection results may be transmitted from the processing device 10 to the first ATM terminal 40-4 and the second ATM terminal 40-5, respectively. Alternatively, the detection results may be transmitted from the processing device 10 to a facility server, and then transmitted from the facility server to the first ATM terminal 40-4 and the second ATM terminal 40-5, respectively. Alternatively, the detection results of the processing device 10 may be transmitted to the first ATM terminal 40-4 and the second ATM terminal 40-5, respectively, via another route.

[0036] The digital signage 40-6 is a terminal that presents notifications and other information to visitors to the facility. The digital signage 40-6 is installed at any location within the facility. The detection results may be transmitted from the processing device 10 to the digital signage 40-6. Alternatively, the detection results may be transmitted from the processing device 10 to a server at the facility, and then transmitted from the server at the facility to the digital signage 40-6. Alternatively, the detection results of the processing device 10 may be transmitted to the digital signage 40-6 via another route.

[0037] The processing device provider terminal 40-7 is a terminal used by a person in charge of the provider (the manufacturer or seller of the processing device 10) who provided the processing device 10. The person in charge of the provider who provided the processing device 10 is notified of the detection result of the processing device 10, i.e., that a person performing a predetermined detection target behavior has been detected, via the processing device provider terminal 40-7. The processing device provider terminal 40-7 is a smartphone, tablet terminal, mobile phone, personal computer, dedicated terminal, etc. The detection result may be transmitted from the processing device 10 to the processing device provider terminal 40-7. Alternatively, the detection result may be transmitted from the processing device 10 to a server of the provider who provided the processing device 10, and then transmitted from the server to the processing device provider terminal 40-7. The detection result of the processing device 10 may also be transmitted to the processing device provider terminal 40-7 via another route.

[0038] "Hardware Configuration" Next, an example of the hardware configuration of the processing device 10 will be described. Each functional unit of the processing device 10 is realized by any combination of hardware and software, centered around a CPU (Central Processing Unit) of any computer, memory, programs loaded into the memory, a storage unit such as a hard disk that stores the programs (this can store programs that are pre-loaded when the device is shipped, as well as programs downloaded from recording media such as CDs (Compact Discs) or servers on the Internet), and a network connection interface. Those skilled in the art will understand that there are many variations in the realization methods and devices.

[0039] FIG. 4 is a block diagram illustrating an example of the hardware configuration of a processing device 10. As shown in FIG. 4, the processing device 10 has a processor 1A, a memory 2A, an input / output interface 3A, a peripheral circuit 4A, and a bus 5A. The peripheral circuit 4A includes various modules. The processing device 10 does not necessarily have to have the peripheral circuit 4A. Note that the processing device 10 may be composed of multiple devices that are physically and / or logically separated. In this case, each of the multiple devices may have the above hardware configuration.

[0040] The bus 5A is a data transmission path for the processor 1A, memory 2A, peripheral circuit 4A, and input / output interface 3A to transmit and receive data among them. The processor 1A is an arithmetic processing device such as a CPU or a GPU (Graphics Processing Unit). The memory 2A is a memory such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The input / output interface 3A includes an interface for acquiring information from an input device, an external device, an external server, an external sensor, a camera, etc., and an interface for outputting information to an output device, an external device, an external server, etc. Examples of input devices include a keyboard, a mouse, a microphone, physical buttons, a touch panel, etc. Examples of output devices include a display, a speaker, a printer, a mailer, etc. The processor 1A can issue commands to each module and perform calculations based on the results of those calculations.

[0041] "Function Configuration" Next, the functional configuration of the processing device 10 according to the second embodiment will be described in detail. An example of a functional block diagram of the processing device 10 according to the second embodiment is shown in FIG. 1. As shown in the figure, the processing device 10 includes a behavior analysis unit 11, a determination unit 12, and a notification unit 13.

[0042] The behavior analysis unit 11 detects whether a person (hereinafter referred to as a "target person") appearing in an image generated by the camera 30, i.e., an image of a person inside the facility, is engaging in one of multiple detection target behaviors.

[0043] "Detection target behavior" is behavior that is desirable to detect for purposes such as preventing incidents and improving the quality of services provided by a facility. For example, detection target behavior may include at least one of talking on a mobile phone, operating an ATM while talking on a mobile phone, moving using a wheelchair, moving using a white cane, and behavior showing interest in predetermined materials placed within the facility (pamphlets, catalogs, advertisements, flyers, etc.). Behaviors showing interest in predetermined materials placed within the facility include behavior of reaching for the predetermined material, behavior of picking up the predetermined material, behavior of looking at the predetermined material, behavior of looking at the predetermined material for a predetermined period of time, etc.

[0044] By detecting mobile phone calls and ATM operation while talking on a mobile phone, crimes such as bank transfer fraud can be prevented. By detecting movement using a wheelchair or a white cane and providing assistance to such visitors, the quality of services can be improved. Furthermore, by detecting behavior showing interest in certain materials placed within the facility and providing such visitors with appropriate information, the quality of services can be improved.

[0045] By analyzing the image, it is possible to detect whether a target person is performing a target behavior. The image analysis is performed by a pre-prepared image analysis system 20. As shown in FIG. 5, a behavior analysis unit 11 inputs an image to the image analysis system 20. Then, the behavior analysis unit 11 obtains the analysis result of the image from the image analysis system 20. The image analysis system 20 may be a part of the processing device 10, or may be an external device that is physically and / or logically independent from the processing device 10.

[0046] Here, we will explain the image analysis system 20. The image analysis system 20 has at least one of a face recognition function, a human shape recognition function, a posture recognition function, a movement recognition function, an appearance attribute recognition function, an image gradient feature detection function, an image color feature detection function, an object recognition function, a character recognition function, and a gaze detection function.

[0047] The face recognition function extracts facial features of a person. It may also compare and calculate the similarity between facial features (to determine whether they are the same person, etc.). It may also compare the extracted facial features with those of multiple users pre-registered in a database to identify which user is the person in the image.

[0048] The human recognition function extracts a person's physical features (e.g., overall features such as whether they are fat or thin, height, clothing, etc.). It may also compare and calculate the similarity between the physical features (e.g., to determine whether they belong to the same person). It may also compare the extracted physical features with the physical features of multiple users pre-registered in a database to identify the person in the image.

[0049] The posture recognition function and the movement recognition function detect the joint points of a person and connect the joint points to construct a stick figure model. Then, based on the stick figure model, the person is detected, the height of the person is estimated, the posture is identified, and the movement is identified based on changes in posture. For example, postures and actions of talking on the phone, postures and actions of operating an ATM, postures and actions of moving around using a wheelchair, postures and actions of moving around using a white cane, postures and actions of reaching for a document, and postures and actions of picking up a document may be identified. Furthermore, similarities between posture features and movement features may be compared and calculated (e.g., determining whether the postures or movements are the same). The estimated height may also be compared with the heights of multiple users pre-registered in a database to identify which user is the person appearing in the image. The posture recognition function and the movement recognition function may be realized by the technology disclosed in Patent Document 3 and Non-Patent Document 1.

[0050] The appearance attribute recognition function recognizes appearance attributes associated with a person (for example, clothing color, shoe color, hairstyle, whether a hat or tie is worn, etc., for a total of more than 100 types of appearance attributes). Furthermore, the function may compare and calculate the similarity of the recognized appearance attributes (it is possible to determine whether they are the same attribute). The recognized appearance attributes may also be compared with the appearance attributes of multiple users pre-registered in a database to identify which user the person in the image is.

[0051] Image gradient feature detection functions include SIFT, SURF, RIFF, ORB, BRISK, CARD, HOG, etc. According to these functions, the gradient features of each frame image are detected.

[0052] The image color feature detection function generates data indicating the color features of the image, such as a color histogram, and detects the color features of each frame image.

[0053] The object recognition function is realized using an engine such as YOLO (which can extract general objects (such as tools and equipment used in sports and other performances) and people). By using the object recognition function, various objects can be detected from images. For example, wheelchairs, white canes, mobile phones, specified documents, etc. may be detected.

[0054] The character recognition function recognizes numbers, letters, etc. that appear in an image.

[0055] The gaze detection function detects the gaze direction of a person in an image. Based on the detected gaze direction and the detected position of a specific document in the image, it can be determined that the person is looking at that document.

[0056] The behavior analysis unit 11 detects that the target person is performing one of a plurality of detection target behaviors based on the analysis results received from the image analysis system 20 described above.

[0057] Returning to FIG. 1 , the determination unit 12 determines the notification destination based on at least one of the type of detected target behavior, the location where the detected target behavior is occurring, the duration of the detected target behavior, the time when the detected target behavior is occurring, and the certainty of the detection result. For example, the determination unit 12 may determine the notification destination based on at least two of the type of detected target behavior, the location where the detected target behavior is occurring, the duration of the detected target behavior, the time when the detected target behavior is occurring, and the certainty of the detection result. Alternatively, the determination unit 12 may determine the notification destination based on at least three of the type of detected target behavior, the location where the detected target behavior is occurring, the duration of the detected target behavior, the time when the detected target behavior is occurring, and the certainty of the detection result. Alternatively, the determination unit 12 may determine the notification destination based on all of the type of detected target behavior, the location where the detected target behavior is occurring, the duration of the detected target behavior, the time when the detected target behavior is occurring, and the certainty of the detection result.

[0058] A plurality of notification destinations are defined in advance. Then, the determination unit 12 determines a notification destination from among the plurality of notification destinations. For example, the plurality of predefined notification destinations may include at least one of a facility worker terminal 40-1, a security terminal 40-2, a processing device manager terminal 40-3, and a visitor terminal that is installed in the facility and outputs information to people within the facility. The visitor terminals include at least one of a first ATM terminal 40-4, a second ATM terminal 40-5, and a digital signage 40-6.

[0059] 6, first correspondence information that associates the detection result with the notification destination is generated in advance and stored in the processing device 10. The determination unit 12 determines the notification destination according to the detection result based on the detection result by the behavior analysis unit 11 and the first correspondence information.

[0060] The detection results by the behavior analysis unit 11 include at least one of the type of detected behavior to be detected, the location where the detected behavior to be detected is being performed, the length of time the detected behavior to be detected is being performed, the time when the detected behavior to be detected is being performed, and the confidence level of the detection result.

[0061] The "type of detected behavior to be detected" indicates one of a plurality of predefined behaviors to be detected.

[0062] The "location where the detected target behavior is occurring" indicates the location within the facility where the person performing the detected target behavior is located. For example, the installation location of the camera 30 that generated the image in which the target behavior is detected may be indicated as the location where the detected target behavior is occurring.

[0063] The "length of time that a person performing the detected target behavior continues to perform the target behavior" refers to the length of time that the person performing the detected target behavior continues to perform the target behavior. This length of time may be the length of time that a person continues to perform a specific target behavior (e.g., operating an ATM while talking on a mobile phone) in an image captured by a single camera. Alternatively, the length of time that a person continues to perform a specific target behavior (e.g., talking on a mobile phone) within a facility may be calculated based on the analysis results of images captured by multiple cameras installed within the facility. The latter example is realized, for example, by the following process. First, the behavior analysis unit 11 identifies a person appearing in multiple images captured by multiple cameras using features of the person's appearance (e.g., facial information). Then, when a person detected to be performing a predetermined behavior to be detected in the first image (e.g., talking on a mobile phone) frames out of the first image while still performing the predetermined behavior to be detected and then frames in the second image while still performing the predetermined behavior to be detected, the behavior analysis unit 11 may calculate the length of time by assuming that the predetermined behavior to be detected was continued from the time the person frames out of the first image to the time the person frames in the second image. Note that, when the condition "the time from the time the person frames out of the first image to the time the person frames in the second image is less than a predetermined time" is satisfied, the behavior analysis unit 11 may calculate the length of time by assuming that the predetermined behavior to be detected was continued from the time the person frames out of the first image to the time the person frames in the second image.

[0064] The "time at which the detected target behavior is being performed" indicates the time at which the person performing the detected target behavior is performing the target behavior.

[0065] The "certainty of the detection result" indicates the degree of certainty that the detected target behavior is being performed. For example, the certainty of the posture and movement detected by the posture recognition function and the movement recognition function may be used as the certainty of the detection result. Alternatively, the certainty of the detection result may be the result of integrating, using a predetermined method (e.g., averaging, weighted averaging, etc.), the certainty of the posture and movement detected by the posture recognition function and the certainty of the object (e.g., mobile phone, wheelchair, white cane, document) used together when performing each target behavior detected by the object recognition function.

[0066] Here, a specific example of the first correspondence information that associates the detection result with the notification destination will be described. In this specific example, the notification destination includes a facility worker terminal 40-1, a security terminal 40-2, a processing device manager terminal 40-3, a first ATM terminal 40-4, a second ATM terminal 40-5, and a digital signage 40-6.

[0067] "Determining who to notify based on the type of behavior detected" For example, the first correspondence information may stipulate that if the detected target behavior is either "making a call using a mobile phone" or "operating an ATM while making a call using a mobile phone," at least one of the facility's worker terminal 40-1, security terminal 40-2, and notification destination terminal 40 for that ATM (first ATM terminal 40-4 or second ATM terminal 40-5) will be determined as the notification destination.

[0068] By designating the facility's worker terminal 40-1 or security terminal 40-2 as the notification destination, facility workers, security guards, etc. can grasp the situation (the situation in which the above-mentioned target behavior was detected). Then, the facility workers, security guards, etc. can rush to the scene and speak to the person engaging in the behavior, thereby preventing an incident from occurring. In addition, by designating the notification destination terminal 40 for the ATM used by the person engaging in the target behavior as the notification destination, a warning or a call for attention can be issued to the person engaging in the target behavior.

[0069] Furthermore, the first response information may stipulate that when the detected target behavior is either "movement using a wheelchair" or "movement using a white cane," the facility worker terminal 40-1 is determined as the notification destination. By designating the facility worker terminal 40-1 as the notification destination, the facility worker can understand the situation (the situation in which the target behavior was detected). Then, the facility worker rushes to the scene and helps the person performing the behavior, thereby improving the quality of service, etc.

[0070] In addition, the first correspondence information may specify that if the detected behavior to be detected is "behavior showing interest in a predetermined material placed in a facility," the digital signage 40-6 near the person performing that behavior is determined as the notification destination. detection The digital signage 40-6 installed closest to the camera 30 that generated the image in which the target behavior was detected may be determined as the notification destination. Based on pre-registered information indicating the installation location of the camera 30 and information indicating the installation location of the digital signage 40-6, it is possible to identify the digital signage 40-6 installed closest to a specific camera 30. By designating such a digital signage 40-6 as the notification destination, it is possible to provide appropriate information related to a specific material to a person who is interested in the specific material.

[0071] Furthermore, the first correspondence information may stipulate that, regardless of the type of the detected target behavior, at least one of the processing device manager terminal 40-3 and the processing device provider terminal 40-7 is to be determined as the notification destination. By designating the processing device manager terminal 40-3 and the processing device provider terminal 40-7 as the notification destination for all data, it becomes possible to appropriately review the determination results of the processing device 10, determine the validity of the algorithm of the processing device 10, and determine whether correction or maintenance is required.

[0072] "Determine who to notify based on the location of the detected behavior" For example, the first response information may specify that if the detected target behavior is either "talking on a mobile phone" or "operating an ATM while talking on a mobile phone," and the location where the target behavior is detected is away from the ATM (i.e., a location different from the location where the person is located when operating the ATM), then the facility worker terminal 40-1 is to be determined as the notification destination. By designating the facility worker terminal 40-1 as the notification destination, the facility worker can understand the situation (the situation in which the target behavior is detected). Then, the facility worker can rush to the scene and speak to the person performing the behavior, thereby preventing an incident from occurring.

[0073] Furthermore, the first correspondence information may stipulate that if the detected behavior to be detected is either "talking on a mobile phone" or "operating an ATM while talking on a mobile phone" and the location where the behavior to be detected is near an ATM (i.e., the location where the person is located when operating the ATM), the notification destination terminal 40 for that ATM (first ATM terminal 40-4, second ATM terminal 40-5, etc.) is determined as the notification destination. By designating the notification destination terminal 40 for the ATM used by the person performing the behavior to be detected as the notification destination, a warning or a call to attention can be given to the person performing the behavior to be detected.

[0074] In this way, in situations where it is possible to directly warn or warn the person performing the behavior to be detected (when the person is near the notification destination terminal 40), a direct warning or warning is given to the person, and in situations where it is not possible to directly warn or warn the person performing the behavior to be detected (when there is no notification destination terminal 40 nearby), a notification is given to a facility worker, who can then prompt the worker to take action against the person.

[0075] "Determine who to notify based on the length of time the detected behavior is occurring" For example, the first correspondence information may stipulate that when the length of time during which an ATM is operated while talking on a mobile phone is less than a threshold, the notification destination terminal 40 for that ATM (first ATM terminal 40-4, second ATM terminal 40-5, etc.) is determined as the notification destination.The first correspondence information may stipulate that when the length of time during which an ATM is operated while talking on a mobile phone is equal to or greater than a threshold, the facility worker terminal 40-1 is determined as the notification destination.

[0076] "Determining notification recipients based on the time of day when the detected behavior is occurring" For example, the first correspondence information may stipulate that when a target behavior is detected during business hours of the facility, the worker terminal 40-1 of the facility is determined as the notification destination. As another example, when a target behavior is detected during business hours of the facility, the notification destination is determined by one of the methods described above in "Determining the notification destination based on the type of detected target behavior" or "Determining the notification destination based on the location where the detected target behavior is being performed."

[0077] In addition, the first response information may stipulate that if the target behavior is detected outside the facility's business hours, at least one of the processing device manager terminal 40-3 and the processing device provider terminal 40-7 will be determined as the notification destination.

[0078] By notifying the appropriate notification destination according to the situation during business hours of the detection result, it is possible to prevent incidents from occurring and improve the quality of service. Furthermore, since there are no visitors in the facility outside of business hours, the behavior to be detected will not be detected. If the behavior to be detected is detected despite this, there is a possibility that a malfunction or the like has occurred in the processing device 10. Therefore, by notifying the administrator or provider of the processing device 10 of the detection of the behavior to be detected outside of business hours, it is possible to realize early detection of malfunctions or the like in the processing device 10.

[0079] "Determining who to notify based on the confidence level of detection results" For example, the first response information may stipulate that if the detected target behavior is either "making a call using a mobile phone" or "operating an ATM while making a call using a mobile phone" and the certainty is high (above the first reference level), the security terminal 40-2 is determined to be the notification destination.

[0080] In addition, the first correspondence information may stipulate that if the detected behavior to be detected is either "a call using a mobile phone" or "a behavior of operating an ATM while making a call using a mobile phone" and the certainty level is medium (if it is equal to or greater than the second reference level and less than the first reference level), at least one of the facility's worker terminal 40-1 and the notification destination terminal 40 for that ATM (first ATM terminal 40-4, second ATM terminal 40-5, etc.) is determined to be the notification destination.

[0081] Furthermore, the first correspondence information may stipulate that, regardless of the confidence level, at least one of the processing device manager terminal 40-3 and the processing device provider terminal 40-7 is to be determined as the notification destination. By designating the processing device manager terminal 40-3 and the processing device provider terminal 40-7 as the notification destinations for all data, it becomes possible to appropriately review the determination results of the processing device 10, determine the validity of the algorithm of the processing device 10, and determine whether correction or maintenance is necessary.

[0082] 1, the notification unit 13 notifies the determined notification destination of the detection of the detection target behavior. The notification destination terminal 40 of the notification destination outputs predetermined information in response to the notification from the notification unit 13.

[0083] Here, we will explain a specific example of information output by the notification destination terminal 40. In this specific example, the notification destinations include a facility worker terminal 40-1, a security terminal 40-2, a processing device manager terminal 40-3, a first ATM terminal 40-4, a second ATM terminal 40-5, and a digital signage 40-6.

[0084] For example, the facility's worker terminal 40-1, security terminal 40-2, and processing device manager terminal 40-3 output information indicating that a person performing a detection target behavior has been detected. In addition, the facility's worker terminal 40-1, security terminal 40-2, and processing device manager terminal 40-3 may further output information indicating at least one of the type of the detected detection target behavior, the location where the detected detection target behavior is being performed, the length of time the detected detection target behavior is being performed, the time the detected detection target behavior is being performed, and the certainty of the detection result.

[0085] Furthermore, the first ATM terminal 40-4 and the second ATM terminal 40-5 output predetermined warning information. For example, the first ATM terminal 40-4 and the second ATM terminal 40-5 may output a warning sound through a speaker, or may output audio information that calls attention to incidents such as bank transfer fraud. Furthermore, the first ATM terminal 40-4 and the second ATM terminal 40-5 may turn on a warning lamp, or may output information that calls attention to incidents such as bank transfer fraud through a display.

[0086] Furthermore, the digital signage 40-6 provides appropriate information related to a predetermined material to a person who is interested in the predetermined material. For example, the digital signage 40-6 can output advertisements, guidance, information related to the predetermined material, etc.

[0087] Furthermore, the processing device manager terminal 40-3 and the processing device provider terminal 40-7 output new data, i.e., information indicating that new detection results have been accumulated. Then, the processing device manager terminal 40-3 and the processing device provider terminal 40-7 store the detection results by the processing device 10. That is, the processing device manager terminal 40-3 and the processing device provider terminal 40-7 store a history of the detection results by the processing device 10. The detection result history includes at least one of the type of detected behavior to be detected, the location where the detected behavior to be detected is occurring, the length of time the detected behavior to be detected is occurring, the time when the detected behavior to be detected is occurring, and the certainty of the detection result.

[0088] Next, an example of the processing flow of the processing device 10 will be described with reference to the flowchart of FIG.

[0089] When the processing device 10 acquires an image generated by the camera 30 (S10), it analyzes the image and detects whether the target person in the image is performing one of a plurality of detection target actions (S11).

[0090] If the target behavior is detected (Yes in S12), the processing device 10 1 The notification destination is determined based on at least one of the detection results, i.e., the type of the detected target behavior, the location where the detected target behavior is occurring, the length of time the detected target behavior is occurring, the time the detected target behavior is occurring, and the certainty of the detection result (S13).

[0091] Then, the processing device 10 notifies the notification destination determined in S13 of the detection of the detection target behavior (S14). The notification destination terminal 40 of the notification destination that has been notified of the detection of the detection target behavior outputs predetermined information in response to the notification.

[0092] "Action and effect" The processing device 10 of the second embodiment can determine a notification destination to notify of the detection of a target behavior based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time the detected target behavior is being performed, and the confidence level of the detection result. The processing device 10 having such a configuration solves the problem of notifying an appropriate notification destination in response to the detection of a person performing a predetermined behavior.

[0093] Furthermore, the processing device 10 of the second embodiment can notify a predetermined notification destination in response to the detection of a call using a mobile phone or an action of operating an ATM while talking on a mobile phone, thereby realizing prevention of incidents.

[0094] Furthermore, the processing device 10 of the second embodiment can notify a predetermined notification destination in response to the detection of movement using a wheelchair or a white cane, thereby helping such visitors and improving the quality of service.

[0095] Furthermore, the processing device 10 of the second embodiment can notify a predetermined notification destination in response to the detection of a behavior indicating an interest in a predetermined material placed in a facility. As a result, by providing appropriate information related to the predetermined material to a person who is interested in the predetermined material, improvement in the quality of service can be realized.

[0096] Furthermore, according to the processing device 10 of the second embodiment, it is possible to determine an appropriate notification destination according to the detection results from among notification destination terminals 40 such as the facility's worker terminal 40-1, security terminal 40-2, processing device manager terminal 40-3, first ATM terminal 40-4, second ATM terminal 40-5, digital signage 40-6, processing device provider terminal 40-7, etc. By determining an appropriate notification destination from such a variety of notification destination terminals 40, it is possible to effectively prevent incidents from occurring and improve the quality of services provided by the facility.

[0097] <Third embodiment> The processing device 10 of the third embodiment has a function of acquiring operation information indicating the operation content of the ATM, and determining the notification destination of the detection of the target behavior based on the operation information, as will be described in detail below.

[0098] The facility in the third embodiment is a facility where an ATM is installed. The images processed by the behavior analysis unit 11 include images of ATM users (such as images captured by the first ATM camera 30-1 and the second ATM camera 30-2). No. The image includes an image taken by ATM camera 30-2 of ATM No. 2.

[0099] In addition to the processes described in the first and second embodiments, the behavior analysis unit 11 also performs a process of acquiring operation information indicating the operation details of the ATM. The operation information includes at least one of the transaction details (transfer, withdrawal, deposit, entry, etc.) and the transaction amount. The behavior analysis unit 11 acquires such operation information from the ATM or from a bank system or post office system linked to the ATM.

[0100] The determination unit 12 determines a notification destination of the detection of the detection target behavior based on the operation information. As in the second embodiment, the determination unit 12 determines a notification destination based on first correspondence information generated in advance. A specific example of the first correspondence information will be described below.

[0101] For example, the first response information may stipulate that if the detected behavior to be detected is "the behavior of operating an ATM while talking on a mobile phone," the transaction content is a transfer, and the transaction amount (transfer amount) is equal to or greater than a predetermined amount, then the facility's worker terminal 40-1 and security terminal 40-2 are determined to be the notification destinations. In cases where the transaction amount is relatively large and the importance is high, designating not only the facility's worker terminal 40-1 but also security terminal 40-2 as the notification destinations enables security guards and security organizations (such as security companies) to respond quickly, which is expected to prevent incidents from occurring and resolve them quickly.

[0102] Furthermore, the first response information may stipulate that if the detected behavior to be detected is "behavior of operating an ATM while talking on a mobile phone," the transaction is a transfer, and the transaction amount (transfer amount) is less than a predetermined amount, the facility worker terminal 40-1 is determined to be the notification destination. In cases where the transaction amount is relatively small and the importance is low, the excessive workload of security guards and security organizations (security companies, etc.) can be reduced by refraining from notifying the security terminal 40-2.

[0103] Furthermore, in the first response information, the detected target behavior is "operating an ATM while talking on a mobile phone," but if the transaction content is something other than a transfer, it may be specified that the facility's worker terminal 40-1 and security terminal 40-2 will not be determined as the notification destination. If the transaction content is not a transfer, it is highly likely that it is not a bank transfer fraud. By refraining from sending unnecessary notifications to the facility's worker terminal 40-1 and security terminal 40-2, the excessive workload of facility workers, security guards, and security organizations (security companies, etc.) can be reduced.

[0104] Other configurations of the processing apparatus 10 of the third embodiment are similar to those of the processing apparatus 10 of the first and second embodiments.

[0105] The processing device 10 of the third embodiment achieves the same effects as the processing device 10 of the first and second embodiments. Furthermore, the processing device 10 of the third embodiment can determine the notification destination of the detection of the target behavior based on the operation content of the ATM. Such a processing device 10 can suppress unnecessary notifications to the facility's worker terminal 40-1 and security terminal 40-2. As a result, it is possible to reduce the excessive workload of facility workers, security guards, and security organizations (security companies, etc.).

[0106] <Fourth embodiment> The processing device 10 of the fourth embodiment has a function of determining additional notification content to be notified in addition to the detection of the target behavior, based on the detection result of the target behavior, as will be described in detail below.

[0107] The determination unit 12 determines additional notification content to be notified in addition to "detection of target behavior" based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the duration of the detected target behavior, the time when the detected target behavior is being performed, and the confidence level of the detection result.The notification unit 13 then notifies the determined notification destination of the "detection of target behavior" and the determined "additional notification content."The notification destination terminal 40 can output the notified additional notification content and notify the user.

[0108] The additional notification content includes at least one of the following: an image of the person performing the detected target behavior; information indicating the basis for determining that the person is performing the target behavior; the age of the person whose target behavior was detected; the gender of the person whose target behavior was detected; external characteristics of the person whose target behavior was detected; whether the person whose target behavior was detected was accompanied by anyone; information indicating past transactions of the person whose target behavior was detected; and the current location of the person whose target behavior was detected.

[0109] The "image of the person performing the detected target behavior" is an image of the person performing the target behavior. By including such an image in the additional notification content, the person receiving the notification can easily identify the person performing the target behavior. Furthermore, by including such an image in the additional notification content, the person receiving the notification can confirm whether the target behavior is actually being performed.

[0110] The "information indicating the basis for determining that the target behavior is being performed" is the posture and movement determination results obtained by the image analysis system 20. Specific examples of information included in the determination results will be described below.

[0111] First, the image analysis system 20 can determine posture and movement using the technology disclosed in Patent Document 3. In this case, the image analysis system 20 calculates the similarity between a template image showing each of a plurality of postures and movements registered in advance and the posture and movement of the target person appearing in the image, and searches for a template image whose similarity to the posture and movement of the target person appearing in the image satisfies a predetermined condition (e.g., similarity to a template image that is a positive example is at least a predetermined level, similarity to a template image that is a negative example is at most a predetermined level, etc.). The system then determines that the posture and movement shown by the template image whose similarity satisfies the predetermined condition is the posture and movement performed by the target person.

[0112] In this example, the posture and movement determination results by the image analysis system 20 include information indicating the type of posture and movement determined to be performed by the target person, a template image whose similarity satisfies a predetermined condition, and the similarity with the template image, etc. Note that if there are multiple template images whose similarity satisfies a predetermined condition, the determination results may include the multiple template images and the similarity with each of them.

[0113] By including such information in the content of the additional notification, the person receiving the notification can confirm whether the behavior to be detected is actually occurring (validity of the detection result).

[0114] The "age of the person whose behavior is being detected," "gender of the person whose behavior is being detected," and "appearance characteristics of the person whose behavior is being detected" can be identified by analyzing the image generated by the camera 30 using well-known techniques. The appearance characteristics of the person whose behavior is being detected are the characteristics of their clothing and belongings.

[0115] By including such information in the content of the additional notification, the person receiving the notification can easily identify the person who is performing the behavior to be detected.

[0116] "Whether or not the person whose behavior is being detected has a companion" can be determined, for example, by analyzing the image generated by camera 30 and determining whether or not there is a person pushing a wheelchair (a condition for a companion) or whether or not there is a person moving together while touching the body of the person whose behavior is being detected (another condition for a companion).

[0117] By including such information in the content of the additional notification, the person receiving the notification can determine whether or not they need help with the person performing the behavior to be detected.

[0118] "Information indicating the past transaction details of the person whose behavior to be detected has been detected" is acquired from a database of a bank or post office, etc. If the person whose behavior to be detected has been identified (if an individual has been identified) by face recognition based on the operation details of the ATM or images generated by the camera 30, the past (e.g., most recent) transaction details of that person stored in the database of the bank or post office, etc., can be extracted and used as additional notification content.

[0119] By including such information in the additional notification, the person receiving the notification can be provided with information to determine whether or not the person engaging in the behavior to be detected has fallen victim to a bank transfer scam.

[0120] The "current location of the person whose behavior is being detected" can be determined by tracking the person in the image generated by the camera 30 or by using facial recognition technology to search for the person in the image generated by the camera 30.

[0121] By including such information in the content of the additional notification, the person receiving the notification can easily ascertain the current location of the person performing the behavior to be detected.

[0122] The determination unit 12 can determine additional notification content for each determined notification destination. Second correspondence information that associates the detection result, the notification destination, and the additional notification content as shown in Fig. 8 is generated in advance and stored in the processing device 10. The determination unit 12 determines additional notification content for each detection result based on the detection result by the behavior analysis unit 11, the determined notification destination, and the second correspondence information.

[0123] Here, a specific example of the second correspondence information will be described. Note that the example here is merely an example and is not limited to this. In this specific example, notification destinations include a facility worker terminal 40-1, a security terminal 40-2, a processing device manager terminal 40-3, a first ATM terminal 40-4, a second ATM terminal 40-5, and a digital signage 40-6.

[0124] In the second correspondence information, when the detected behavior to be detected is "behavior of operating an ATM while talking on a mobile phone" and the determined notification destination is the first ATM terminal 40-4 or the second ATM terminal 40-5, it may be specified that additional notification content is not included. In this case, for example, only the content described in the second and third embodiments is notified to the first ATM terminal 40-4 or the second ATM terminal 40-5.

[0125] In addition, the second response information may stipulate that if the detected target behavior is "the behavior of operating an ATM while talking on a mobile phone" and the determined notification destination includes at least one of the facility's worker terminal 40-1 and security terminal 40-2, an image of the person performing the detected target behavior is determined as the additional notification content.

[0126] In addition, the second correspondence information may stipulate that if the detected target behavior is "the behavior of operating an ATM while talking on a mobile phone" and the determined notification destination includes at least one of the processing device manager terminal 40-3 and the processing device provider terminal 40-7, an image of the detected target behavior being performed and information indicating the basis for determining that the target behavior is being performed are determined as additional notification content.

[0127] In addition, the second correspondence information may stipulate that if the detected target behavior is "behavior showing interest in specified materials placed within the facility" and the determined notification destination is digital signage 40-6, the external characteristics of the person whose target behavior was detected will be determined as the additional notification content.

[0128] In addition, the second correspondence information may stipulate that if the detected target behavior is "behavior showing interest in specified materials placed within the facility" and the determined notification destination is the facility's worker terminal 40-1, an image of the person performing the detected target behavior will be determined as the additional notification content.

[0129] Furthermore, the second correspondence information may stipulate that if the detected target behavior is "behavior showing interest in specified materials placed within the facility" and the determined notification destination includes at least one of the processing device manager terminal 40-3 and the processing device provider terminal 40-7, an image of the detected target behavior being performed and information indicating the basis for determining that the target behavior is being performed will be determined as the additional notification content.

[0130] Next, an example of the processing flow of the processing device 10 will be described with reference to the flowchart of FIG.

[0131] When the processing device 10 acquires an image generated by the camera 30 (S20), it analyzes the image and detects whether the target person in the image is performing one of a plurality of detection target actions (S21).

[0132] If the target behavior is detected (Yes in S22), the processing device 10 performs the process of S23. 1 The notification destination is determined based on at least one of the detection results, i.e., the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the confidence level of the detection result. 1 The content of the additional notification is determined based on at least one of the detection results, i.e., the type of detected behavior to be detected, the location where the detected behavior to be detected is occurring, the length of time the detected behavior to be detected is occurring, the time the detected behavior to be detected is occurring, and the certainty of the detection result, and the determined notification destination.

[0133] Then, the processing device 10 notifies the notification destination determined in S23 of the detection of the target behavior and the additional notification content determined in S23 (S24). The notification destination terminal 40 of the notification destination that has been notified of the detection of the target behavior outputs predetermined information in response to the notification.

[0134] Other configurations of the processing apparatus 10 of the fourth embodiment are similar to those of the processing apparatus 10 of the first to third embodiments.

[0135] The processing device 10 of the fourth embodiment achieves the same effects as the processing devices 10 of the first to third embodiments. Furthermore, the processing device 10 of the fourth embodiment can notify appropriate additional notification content according to the detection result and the notification destination. As a result, it is possible to notify the notification destination of appropriate information while suppressing the inconvenience of excessive information being notified.

[0136] <Fifth embodiment> In the fifth embodiment, variations of various processes performed by the processing system will be described. The processing system can adopt one of the variations described below. Also, the processing system can adopt a combination of multiple variations described below.

[0137] -Photography by the first ATM camera 30-1 and the second ATM camera 30-2- The first ATM camera 30-1 and the second ATM camera 30-2 may continuously capture moving images during a predetermined time period (e.g., while the facility is open, while the ATM is in operation, etc.). Alternatively, the first ATM camera 30-1 and the second ATM camera 30-2 may capture still images in response to the detection of a predetermined trigger. Alternatively, the first ATM camera 30-1 and the second ATM camera 30-2 may capture moving images for a predetermined period of time in response to the detection of a predetermined trigger.

[0138] The predetermined trigger is the detection of a predetermined operation on the ATM. The predetermined operation may be inserting a card into the ATM, inserting a passbook into the ATM, or making a predetermined input via an input device (touch panel, operation buttons, etc.) of the ATM. The predetermined input may be, for example, an input to start a transfer procedure.

[0139] -Photographed by Camera 30- The camera 30 may capture a still image at predetermined intervals (e.g., every minute). If a person in a predetermined posture (e.g., a posture while talking, a posture while sitting in a wheelchair, or a posture while using a white cane) is detected in the still image, the camera 30 may capture a moving image from that position.

[0140] -Image selection by behavior analysis unit 11- First, the camera 30 captures a moving image. Then, the behavior analysis unit 11 may perform processing to detect the target behavior, with all frame images included in the moving image as processing targets.

[0141] Alternatively, the behavior analysis unit 11 may select some of the frame images included in the video, and perform the process of detecting the target behavior by processing only the selected frame images.

[0142] As a method for selecting some of the frame images, for example, the behavior analysis unit 11 may select frame images captured at the timing when the "predetermined trigger" described above in "Photographing by the first ATM camera 30-1 and the second ATM camera 30-2" is detected. Alternatively, the behavior analysis unit 11 may select frame images for a predetermined time from the frame images captured at the timing when the above-mentioned predetermined trigger is detected.

[0143] Alternatively, the behavior analysis unit 11 may select frame images at predetermined time intervals (for example, every predetermined number of frame images). If a person in a predetermined posture (for example, a posture while talking on the phone, a posture while sitting in a wheelchair, or a posture while using a white cane) is detected in the frame images selected in this manner, the behavior analysis unit 11 may select consecutive frame images for a predetermined period of time from that position.

[0144] -Data to be registered in Image Analysis System 20- First, the image analysis system 20 can determine posture and movement using the technology disclosed in Patent Document 3. In this case, the image analysis system 20 calculates the similarity between a template image showing each of a plurality of postures and movements registered in advance and the posture and movement of the target person appearing in the image, and searches for a template image whose similarity to the posture and movement of the target person appearing in the image satisfies a predetermined condition. The system then determines that the posture and movement shown by the template image whose similarity satisfies the predetermined condition is the posture and movement being performed by the target person. Variations of this processing by the image analysis system 20 will be described below.

[0145] The template image may be a positive example, a negative example, or both. A positive example is a posture or movement when performing a predetermined behavior to be detected. A negative example is a posture or movement that is not the posture or movement when performing the predetermined behavior to be detected, but is similar to the posture or movement when performing the predetermined behavior to be detected. For example, if the predetermined behavior to be detected is "talking on a mobile phone," a positive example would be a talking posture, and a standing posture, a head scratching posture, etc. would be negative examples.

[0146] In addition to the registered posture, the weight of each body part used in matching may also be registered. Matching using the weight of each body part is disclosed in Patent Document 3.

[0147] In addition to the registered posture, essential requirements for matching with a template image may be registered. Images that do not satisfy the essential requirements will not be matched with the template image. Essential requirements may be defined for each posture or movement. For example, an essential requirement for matching with a template image showing a talking posture may be defined as "the target person's hands are visible." If the target person's hands are not visible, it is impossible to determine whether or not the target person is talking, and therefore matching with the template image is meaningless. By matching only images that satisfy the essential requirements with the template image, unnecessary matching can be avoided and the burden on the computer can be reduced.

[0148] -First posture and movement determination process by image analysis system 20- First, both positive and negative examples are registered for each posture and movement of a detection target. Multiple positive and negative examples are registered in association with one posture or movement of a detection target.

[0149] When the image to be analyzed matches at least one positive example, the image analysis system 20 may determine that the subject of the image to be analyzed is performing the posture or movement indicated by the matched positive example.

[0150] Additionally, if the image to be analyzed matches at least one negative example, the image analysis system 20 may determine that the subject in the image to be analyzed does not have a posture or movement similar to that of the matched negative example.

[0151] Alternatively, when the image to be analyzed matches both positive and negative examples, the image analysis system 20 may determine the number of matches based on each. For example, if the number of matches with positive examples is greater than the number of matches with negative examples, the image analysis system 20 may determine that the subject in the image to be analyzed is performing the pose or movement shown in the matched positive example. On the other hand, if the number of matches with negative examples is greater than the number of matches with positive examples, the image analysis system 20 may determine that the subject in the image to be analyzed is not performing the pose or movement of the detection target that is similar to the pose or movement of the matched negative example.

[0152] Alternatively, the image analysis system 20 may make a judgment based on the similarity between the image to be analyzed and the template image when the image to be analyzed matches both the positive and negative examples of the first posture (or first movement).

[0153] For example, if the template image having the highest similarity to the image to be analyzed is a positive example, the image analysis system 20 may determine that the subject in the image to be analyzed is in a first posture (or a first movement). On the other hand, if the template image having the highest similarity to the image to be analyzed is a negative example, the image analysis system 20 may determine that the subject in the image to be analyzed is not in a first posture (or a first movement).

[0154] As another example, the image analysis system 20 may make a judgment based on the magnitude relationship between the "statistical value (average value, maximum value, minimum value, mode, median, etc.) of the similarity of the positive examples that matched the image to be analyzed among the positive examples of the first posture (or first movement)" and the "statistical value of the similarity of the negative examples that matched the image to be analyzed among the negative examples of the first posture (or first movement)."

[0155] For example, if the statistics of the positive examples are greater, the image analysis system 20 may determine that the subject in the image being analyzed is performing the pose or movement shown in the matched positive examples. Conversely, if the statistics of the negative examples are greater, the image analysis system 20 may determine that the subject in the image being analyzed is not performing the pose or movement of the detected subject that is similar to the pose or movement of the matched negative examples.

[0156] As a variation of this example, the "statistical value of similarity of positive examples that match the image to be analyzed among the positive examples of the first posture (or the first movement)" may be replaced with the "statistical value of similarity of all positive examples of the first posture (or the first movement)." And the "statistical value of similarity of negative examples that match the image to be analyzed among the negative examples of the first posture (or the first movement)" may be replaced with the "statistical value of similarity of all negative examples of the first posture (or the first movement)."

[0157] Additionally, the image analysis system 20 may use other techniques such as machine learning to determine the posture and movement of the target person.

[0158] -Second posture and movement determination process by image analysis system 20- The image analysis system 20 determines the posture and movement of a target person based on multiple images. The multiple images may be multiple still images generated by capturing multiple images, or multiple frame images generated by capturing a moving image.

[0159] Then, if the image analysis system 20 determines that a person in an image is performing a predetermined posture or movement at least once in multiple determinations based on multiple images, it may determine that the person in that image is performing that posture or movement.

[0160] Additionally, the image analysis system 20 may determine that a target person in an image is performing a specified posture or movement if, in multiple judgments based on multiple images, the target person is determined to be performing that posture or movement at a predetermined rate or more (i.e., at least N times out of M times).

[0161] Additionally, the image analysis system 20 may determine that a target person in an image has taken a specific posture or movement if it determines that the target person has taken that posture or movement a predetermined number of times or more consecutively based on multiple judgments made on multiple images.

[0162] Additionally, the image analysis system 20 may weight the judgment results of each of the multiple images before making the judgment.

[0163] -Third posture and movement determination process by image analysis system 20- When multiple people appear in the images generated by the first ATM camera 30-1 and the second ATM camera 30-2, the image analysis system 20 can identify the person who appears largest in the image as the target person and determine their posture and movement. Alternatively, the image analysis system 20 may track people in the image using their faces and postures, and identify the person who appears continuously as the target person. The image analysis system 20 may also work in conjunction with the ATM to identify the timing from the start to the end of a transaction. 。 Then, the image analysis system 20 may identify the person who appears in the photograph for the longest period from the start of the transaction to the end of the transaction as the target person.

[0164] Furthermore, the image analysis system 20 may determine whether an ATM is being operated by peeking based on the number of people other than the target person, the direction of their face, the size of their face, and the like.

[0165] -Information output by Digital Signage 40-6- The digital signage 40-6 may output information related to the material in which the target person is interested, and information determined based on various attributes of the target person. of analysis To do Alternatively, an individual may be identified from a card or passbook inserted into an ATM, and information (such as an address) linked to that individual and registered in advance in a bank system or post office system may be used as information indicating the attributes of the target person.

[0166] In addition, the behavior history of a target person identified by tracking the target person in an image generated by the camera 30 may be used as information indicating the target person's attributes. Note that when tracking a target person, multiple cameras 30 can be linked together.

[0167] -Using the detection results- The processing device 10 may calculate the tendency of each target behavior to appear based on the detection results for each of the multiple target behaviors. For example, the processing device 10 may calculate the situation for each case by dividing the cases by time of day, day of the week, store, etc. and performing statistical processing. Based on the calculation results, the facility can take measures such as increasing the number of workers in certain cases or playing audio warnings about bank transfer fraud.

[0168] -Output by the notification destination terminal 40- If the additional notification content includes an image of the person performing the detected target behavior and the image is a moving image, the notification destination terminal 40 may output, as the additional notification content, a moving image from a timing a predetermined time before the timing when the target behavior was detected.

[0169] The other configurations of the processing system of the fifth embodiment are similar to the configurations of the processing systems of the first to fourth embodiments.

[0170] The processing system of the fifth embodiment achieves the same effects as the processing systems of the first to fourth embodiments. Furthermore, the processing system of the fifth embodiment can employ various variations, which is preferable because it increases the degree of freedom in design.

[0171] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations may be adopted. The configurations of the above-described embodiments may be combined with each other, or some of the configurations may be replaced with other configurations. Furthermore, various modifications may be made to the configurations of the above-described embodiments without departing from the spirit of the invention. Furthermore, the configurations and processes disclosed in the above-described embodiments and modified examples may be combined with each other.

[0172] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0173] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. 1. A behavior analysis means for detecting whether a target person in an image of a person in a facility is performing one of a plurality of target detection behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; A processing device having: 2. The processing device described in 1, wherein the determination means determines the notification destination based on at least two of the type of the detected target behavior, the location where the detected target behavior is occurring, the length of time the detected target behavior is occurring, the time the detected target behavior is occurring, and the certainty of the detection result. 3. The processing device described in 1, wherein the determination means determines the notification destination based on at least three of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the certainty of the detection result. 4. The processing device described in 1, wherein the determination means determines the notification destination based on the type of the detected target behavior, the location where the detected target behavior is occurring, the length of time the detected target behavior is occurring, the time when the detected target behavior is occurring, and the certainty of the detection result. 5. The facility is equipped with an ATM (Automatic Teller Machine), The image includes an image of an ATM user, The behavior analysis means acquires operation information indicating the operation content of the ATM, 5. The processing device according to any one of 1 to 4, wherein the determining means determines a notification destination of the detection of the detection target behavior further based on the operation information. 6. The processing device according to 5, wherein the operation information includes at least one of the transaction details and the transaction amount. 7. A processing device described in any one of 1 to 6, wherein the behavior to be detected includes at least one of talking on a mobile phone, operating an ATM while talking on a mobile phone, moving around using a wheelchair, moving around using a white cane, and showing interest in specified materials placed within the facility. 8. A processing device described in any one of 1 to 7, wherein the notification destination includes at least one of a worker terminal at the facility, a security terminal, an administrator terminal that manages the processing device, a visitor terminal that is installed at the facility and outputs information to people within the facility, and a provider terminal that provided the processing device. 9. The determination means determines the content of an additional notification to be notified in addition to the detection of the target behavior based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the certainty of the detection result; 9. The processing device according to any one of 1 to 8, wherein the notification means notifies the determined notification destination of the determined additional notification content. 10. The processing device described in 9, wherein the additional notification content includes at least one of the image of the person performing the detected target behavior, information indicating the basis for determining that the person is performing the target behavior, the age of the person performing the target behavior, the gender of the person performing the target behavior, the external characteristics of the person performing the target behavior, whether the person performing the target behavior is accompanied by anyone, information indicating past transactions of the person performing the target behavior, and the current location of the person performing the target behavior. 11. The processing device according to 9 or 10, wherein the determining means determines the additional notification content for each notification destination. 12. The computer Detects whether a target person in an image of a person in a facility is performing one of a plurality of detection target actions, determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the certainty of the detection result; notifying the determined notification destination of the detection of the target behavior; Processing method. 13. Computer, behavior analysis means for detecting whether a target person captured in an image of a person in the facility is performing one of a plurality of detection target behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; A recording medium on which a program that functions as a [Explanation of symbols]

[0174] 10 Processing equipment 11 Behavior Analysis Department 12 Decision Section 13 Notification Department 20 Image analysis system 30 Camera 40 Notification destination terminal 1A processor 2A Memory 3A input / output I / F 4A peripheral circuit 5A Bus

Claims

1. a behavior analysis means for detecting whether a target person captured in an image of a person in the facility is performing one of a plurality of detection target behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; and The determining means If the length of time during which the detected target behavior is being performed is less than a threshold, the device operated by the person during whom the target behavior is detected is determined to be the notification destination; a processing device that determines a worker terminal of the facility as the notification destination when the length of time during which the detected target behavior is being performed is equal to or greater than the threshold value;

2. The plurality of behaviors to be detected include at least one first behavior related to preventing incidents and at least one second behavior related to improving the quality of services provided by the facility; The determining means When the first behavior is detected, determining at least one of a worker terminal, a security terminal, and a terminal operated by the person who detected the first behavior as the notification destination; The processing device according to claim 1 , wherein when the second behavior is detected, a worker terminal of the facility is determined as the notification destination.

3. The facility is equipped with an ATM (Automatic Teller Machine), The image includes an image of an ATM user, The behavior analysis means acquires operation information indicating the operation content of the ATM, The processing device according to claim 1 , wherein the determining unit determines a notification destination of the detection of the target behavior based further on the operation information.

4. 4. A processing device according to claim 1, wherein the behavior to be detected includes at least one of the following: talking on a mobile phone, operating an ATM while talking on a mobile phone, moving using a wheelchair, moving using a white cane, and showing interest in specified materials placed within the facility.

5. A processing device described in any one of claims 1 to 4, wherein the notification destinations include at least one of a worker terminal at the facility, a security terminal, an administrator terminal that manages the processing device, a visitor terminal installed at the facility that outputs information to people within the facility, and a provider terminal that provided the processing device.

6. the determining means determines additional notification content to be notified in addition to the detection of the detection target behavior based on at least one of the type of the detected detection target behavior, the location where the detected detection target behavior is being performed, the length of time the detected detection target behavior is being performed, the time when the detected detection target behavior is being performed, and the certainty of the detection result; The processing device according to claim 1 , wherein the notification unit notifies the determined notification destination of the determined additional notification content.

7. The processing device described in claim 6, wherein the additional notification content includes at least one of the image of the person performing the detected target behavior, information indicating the basis for determining that the person is performing the target behavior, the age of the person whose target behavior was detected, the gender of the person whose target behavior was detected, the external characteristics of the person whose target behavior was detected, whether the person whose target behavior was detected is accompanied by anyone, information indicating past transaction details of the person whose target behavior was detected, and the current location of the person whose target behavior was detected.

8. The processing device according to claim 6 or 7, wherein the determining unit determines the additional notification content for each of the notification destinations.

9. The computer Detects whether a target person in an image of a person in a facility is performing one of a plurality of detection target actions, determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; notifying the determined notification destination of the detection of the target behavior; In the process of determining the notification destination, If the length of time during which the detected target behavior is being performed is less than a threshold, the device operated by the person during whom the target behavior is detected is determined to be the notification destination; a processing method for determining a worker terminal of the facility as the notification destination when the length of time during which the detected target behavior is being performed is equal to or greater than the threshold value;

10. Computer, behavior analysis means for detecting whether a target person captured in an image of a person in the facility is performing one of a plurality of detection target behaviors; a determination means for determining a notification destination based on at least one of the type of the detected target behavior, the location where the detected target behavior is being performed, the length of time the detected target behavior is being performed, the time when the detected target behavior is being performed, and the degree of certainty of the detection result; a notification means for notifying the determined notification destination of the detection of the target behavior; It functions as The determining means If the length of time during which the detected target behavior is being performed is less than a threshold, the device operated by the person during whom the target behavior is detected is determined to be the notification destination; a program that determines a worker terminal of the facility as the notification destination when the length of time during which the detected target behavior is being performed is equal to or greater than the threshold value;

Images