Monitoring system and monitoring method

Abstract

To provide a monitoring system and a monitoring method, in which even when a plurality of alarming events has been generated sequentially without confirmation of a user, the user confirms not only the newest alarming event but also alarming events generated before then without oversight.SOLUTION: In a method, a face collation processing of comparing the face image of a visitor with a registered face image of a monitoring target person to determine whether the visitor is the same as the monitoring target person is executed and, when performing alarming on detection of the monitoring target person on the basis of its result, a terminal device acquires alarming storage outline information on an alarming event that has not yet been confirmed by a user and in a stored state and displays an alarming screen 401 including the alarming storage outline information. Especially, in an alarming storage outline display unit 431, the total alarming storage number (alarm pop-up storage number), which is the total number of alarming events in a stored state while having not been confirmed by the user, and a target person number, which is the number of monitoring target persons related to the alarming events in the stored state, are displayed as the alarming storage outline information using a pop-up displayed alarming screen.SELECTED DRAWING: Figure 15

Description

Technical Field

[0001] The present invention relates to a monitoring system and a monitoring method that execute, by a processor, a process of performing notification regarding detection of a person to be monitored as a notification event based on an image obtained by photographing a visitor to a facility.

Background Art

[0002] In stores such as supermarkets and shopping malls, for crime prevention, a person who has engaged in problematic behavior (such as theft, violence, drinking, etc.) in the store in the past is registered as a person to be monitored, and the detection of the visit of the person to be monitored is used as a notification event, and a monitoring system that notifies store managers, store employees, security guards, etc. is known.

[0003] In such a monitoring system for crime prevention, it is possible to detect the visit of a person to be monitored by photographing a visitor with a camera and performing face verification. Further, when the visit of a person to be monitored as a notification event is detected, notification is performed for a monitor, a store manager, etc. For example, a technique is known in which a message notifying of the visit of a person to be monitored is displayed on a store terminal, or sound or light is emitted from the store terminal to notify a store employee of the visit of the person to be monitored (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the conventional technology, in order to notify a user (store manager or store clerk) of the visit of a person to be monitored, a message notifying the visit of the person to be monitored is displayed on a terminal. On the other hand, an alarm screen can also be pop-up displayed on the terminal. However, the user cannot always view the alarm screen displayed on the terminal. For this reason, if alarm events (visits of persons to be monitored) occur continuously in a short period during the period when the user is not viewing, even if the latest alarm event can be confirmed on the displayed alarm screen, there is a problem that the user does not notice the alarm events that occurred before that.

[0006] Therefore, the main object of the present invention is to provide a monitoring system and a monitoring method that, when an alarm screen is displayed on a terminal device to notify a user of the occurrence of an alarm event, even if a plurality of alarm events occur continuously without the user confirming, the user can confirm not only the latest alarm event but also the alarm events that occurred before that without overlooking them.

Means for Solving the Problems

[0007] The monitoring system of the present invention is a monitoring system in which a processor executes a process of performing an alarm regarding detection of a person to be monitored as an alarm event based on an image obtained by photographing a visitor to a facility, including a camera that acquires a photographed image within the facility, a registered image storage unit that stores registered face images for each person to be monitored, and a terminal device that displays a predetermined alarm screen in response to an instruction from the processor and displays a screen for the user to confirm the alarm status for each person to be monitored. The processor executes a face verification process of comparing a face image of a visitor with the registered face image of the person to be monitored to determine whether they are the same person, and when performing an alarm regarding detection of the person to be monitored based on the result, acquires alarm accumulation summary information regarding an alarm event that is in an unconfirmed and accumulated state in the terminal device, and causes the terminal device to display the alarm screen including the alarm accumulation summary information.

[0008] The monitoring method of the present invention is a monitoring method in which a processor executes a process of issuing a report regarding the detection of a person to be monitored as a reporting event based on an image of a visitor to a facility, and performs a face verification process of comparing a face image of the visitor with a registered face image of the person to be monitored to determine whether they are the same person. When issuing a report regarding the detection of the person to be monitored based on the result, the terminal device acquires report accumulation summary information regarding a reporting event that is unconfirmed and in an accumulation state by the user, and causes the terminal device to display a reporting screen including the report accumulation summary information.

Effect of the Invention

[0009] According to the present invention, by viewing the reporting screen, the user can confirm the latest reporting event, and further, by the report accumulation summary information in the reporting screen, the user can confirm the outline of the reporting event in the accumulation state. As a result, when the reporting screen is displayed on the terminal device to notify the user of the occurrence of the reporting event, even if a plurality of reporting events occur continuously without the user confirming, the user can confirm not only the latest reporting event but also the reporting events that occurred before that without overlooking them.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Mode for Carrying Out the Invention

[0011] The first invention made to solve the above problems is a monitoring system that executes, by a processor, a process of performing a notification regarding detection of a monitoring target person as a notification event based on an image obtained by photographing a visitor to a facility, the monitoring system including: a camera that acquires a photographed image within the facility; a registered image storage unit that stores registered face images for each monitoring target person; and a terminal device that displays a predetermined notification screen in response to an instruction from the processor and displays a screen for a user to confirm the notification state for each monitoring target person. The processor executes a face verification process of comparing the face image of the visitor with the registered face image of the monitoring target person to determine whether they are the same person, and when performing a notification regarding detection of the monitoring target person based on the result, acquires notification accumulation summary information regarding a notification event that is in an unconfirmed and stored state by the user in the terminal device, and causes the terminal device to display the notification screen including the notification accumulation summary information.

[0012] According to this, by the user viewing the transmission report screen, the user can check the latest transmission report event, and further, based on the transmission report accumulation summary information in the transmission report screen, the user can check the summary of the transmission report events in the accumulation state. As a result, when the terminal device displays the transmission report screen to notify the user of the occurrence of the transmission report event, even if a plurality of transmission report events occur continuously without the user checking, the user can check not only the latest transmission report event but also the transmission report events that occurred before that without overlooking them.

[0013] Further, in the second invention, the processor causes the terminal device to display the transmission report screen including, as the transmission report accumulation summary information, the total number of transmission report events in the accumulation state, represented by the total transmission report accumulation number, and the number of monitored persons related to the transmission report events in the accumulation state, represented by the number of target persons.

[0014] According to this, the user can appropriately check the summary of the transmission report events in the accumulation state based on the total transmission report accumulation number and the number of persons in the transmission report accumulation.

[0015] Further, in the third invention, the processor causes the terminal device to display the transmission report screen including an operation unit that instructs a transition to a detailed screen showing detailed information on the transmission report events in the accumulation state, and causes the terminal device to display the detailed screen in response to a user operation on the operation unit in the transmission report screen.

[0016] According to this, when the user views the transmission report screen to check the occurrence of the transmission report event and the summary of the transmission report events in the accumulation state, the user can quickly switch to the detailed screen by operating the operation unit and check the details of the transmission report events in the accumulation state. In this case, the display area of the transmission report accumulation summary information may be set in the operation unit that instructs a transition to the detailed screen.

[0017] Moreover, in the fourth invention, the processor is configured to cause the terminal device to display the detailed screen including the list information in which information for each reporting event is arranged, limited to the reporting events in the accumulated state, as the detailed information on the accumulated reports.

[0018] According to this, the user can quickly grasp the content of the reporting events in the accumulated state.

[0019] Moreover, in the fifth invention, the processor is configured to cause the terminal device to display the detailed screen including the summary information regarding the accumulation status of the reporting events for each monitored person, as the detailed information on the accumulated reports.

[0020] According to this, the user can quickly grasp the content of the reporting events in the accumulated state.

[0021] Moreover, in the sixth invention, the processor causes the terminal device to display the detailed screen including an operation unit for instructing the release of the filtering state, and in response to the user's operation on the operation unit in the detailed screen, releases the filtering state limited to the reporting events in the accumulated state, and causes the terminal device to display a list screen in which information for each reporting event that occurred in the past is arranged.

[0022] According to this, the user can easily confirm the reporting events that occurred in the past without being limited to the reporting events in the accumulated state.

[0023] Moreover, in the seventh invention, the processor is configured to generate the reporting screen based on the screen configuration dedicated to the mobile terminal and transmit the reporting screen to the terminal device which is the mobile terminal.

[0024] According to this, the user can easily view the reporting screen on the terminal device which is the mobile terminal.

[0025] Further, an eighth invention is a monitoring method in which a processor executes a process of issuing a report regarding detection of a person to be monitored as a reporting event based on an image of a visitor to a facility, the method including: performing a face matching process of comparing a face image of the visitor with a registered face image of the person to be monitored to determine whether they are the same person; and when issuing a report regarding detection of the person to be monitored based on the result, acquiring report accumulation summary information regarding a reporting event that is unconfirmed and in an accumulated state by a user in a terminal device, and causing the terminal device to display a reporting screen including the report accumulation summary information.

[0026] According to this, similar to the first invention, a user can check the latest reporting event by viewing the reporting screen, and further, can check an outline of the reporting event in the accumulated state based on the report accumulation summary information in the reporting screen. As a result, when a reporting screen is displayed on the terminal device to notify the user of the occurrence of a reporting event, even if a plurality of reporting events occur continuously without the user checking, the user can check not only the latest reporting event but also the reporting events that occurred before that without overlooking them.

[0027] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0028] (First Embodiment) FIG. 1 is an overall configuration diagram of a monitoring system according to the first embodiment.

[0029] This system detects the visit of a person to be monitored (a person registered as a monitoring target) as a reporting event for security purposes in a store such as a supermarket or a shopping mall, and issues a report to a store manager, a store clerk, a security guard, etc. The person to be monitored is a person who has taken problematic actions (e.g., theft, violence, drinking, shoplifting, customer harassment, etc.) in the store in the past and is registered as a monitoring target when discovered. Note that the facility to which this system is applied is not limited to a store.

[0030] This system includes a face recognition camera 1, an in-store surveillance camera 2, a control terminal 3, a monitoring terminal 4 (terminal device, notification device), an alarm light 5, and a staff terminal 6. The face recognition camera 1, the in-store surveillance camera 2, the control terminal 3, the monitoring terminal 4, the alarm light 5, and the staff terminal 6 are installed in the store. The face recognition camera 1, the in-store surveillance camera 2, the control terminal 3, the monitoring terminal 4, the alarm light 5, and the staff terminal 6 are interconnected via a LAN (in-store network) and are also connected to a wide-area network such as the Internet via a router.

[0031] In addition, this system includes a recording server 11, a monitoring server 12, a face authentication server 13, a user management server 14, and a personal information storage server 15. The recording server 11, the monitoring server 12, the face authentication server 13, the user management server 14, and the personal information storage server 15 are configured in the cloud. The recording server 11, the monitoring server 12, the face authentication server 13, the user management server 14, and the personal information storage server 15 are connected to a wide-area network such as the Internet.

[0032] The face recognition camera 1 is installed at the entrance of the store to photograph visitors to the store (visitors to the facility). Note that the face recognition camera 1 may be installed at an appropriate location inside the store to photograph the people staying inside the store.

[0033] The in-store surveillance camera 2 is installed at an appropriate location inside the store to photograph the people staying inside the store.

[0034] The control terminal 3 controls the face recognition camera 1, the in-store surveillance camera 2, and the warning light 5. Further, the control terminal 3 extracts the face image and the full body image of the customer from the video captured by the face recognition camera 1. Here, although the control terminal 3 is described as controlling the face recognition camera 1, the in-store surveillance camera 2, and the warning light 5 as the devices to be controlled, the devices to be controlled are not limited to these. For example, as the device to be controlled, only the face recognition camera 1 may be controlled, and the in-store surveillance camera 2 and the warning light 5 may be controlled by other control devices (not shown). Also, as the device to be controlled, the face recognition camera 1 and the in-store surveillance camera 2 may be controlled, and the warning light 5 may be controlled by other control devices (not shown).

[0035] The monitoring terminal 4 is configured by a PC. A monitoring application is installed on the monitoring terminal 4. A monitor or a store manager operates the monitoring terminal 4 to view the screen displayed on the monitoring terminal 4. The monitoring terminal 4 displays the videos captured by the face recognition camera 1 and the in-store surveillance camera 2. In the monitoring terminal 4, in addition to the real-time videos being captured by the face recognition camera 1 and the in-store surveillance camera 2, the past videos stored in the recording server 11 are also displayed. Further, the monitoring terminal 4 displays a reporting screen 401 based on the reporting information transmitted from the monitoring server 12. Thereby, the occurrence of a reporting event, that is, the visit of the person to be monitored, can be notified to the monitor or the store manager. When displaying the real-time video on the monitoring terminal 4, it is possible to directly acquire and display the videos captured by the face recognition camera 1 and the in-store surveillance camera 2, but it is also possible to display the same videos via the recording server 11.

[0036] Also, the monitoring terminal 4 has a function as a management terminal. That is, on the monitoring terminal 4, a user (monitor or store manager) can perform an operation to register the face image and other information about the person to be monitored. Also, on the monitoring terminal 4, based on the past reporting implementation status, the user can perform a setting operation to exclude from the monitoring target or temporarily exclude from the reporting target without excluding from the monitoring target.

[0037] The warning light 5 performs the operation of lighting the lamp and outputting a warning sound as a reporting operation in response to an instruction of the reporting operation from the monitoring server 12. Thereby, the occurrence of the reporting event, that is, the arrival of the monitored person at the store, can be notified to the monitor or the store manager.

[0038] The worker terminal 6 is composed of a mobile terminal such as a smartphone or a tablet terminal. Workers such as store managers and store clerks possess the worker terminal 6, operate the worker terminal 6, and view the screen displayed on the worker terminal 6. The worker terminal 6 receives an e-mail transmitted from the monitoring server 12 and displays the reporting information included in the e-mail. Thereby, the occurrence of the reporting event, that is, the arrival of the monitored person at the store, can be notified to workers such as store managers and store clerks. Further, the worker terminal 6 acquires and displays detailed reporting information including the face image of the monitored person who is the party to the reporting event from the monitoring server 12 based on the link information included in the e-mail.

[0039] The recording server 11 stores the videos captured by the face recognition camera 1 and the in-store monitoring camera 2.

[0040] The monitoring server 12 acquires the face image of the customer from the control terminal 3, requests the face authentication server 13 to perform a face verification to determine whether the customer is a monitored person (a person registered as a monitoring target), and based on the face verification result obtained from the face authentication server 13, when detecting the arrival of the monitored person, performs a reporting to notify the store manager and the store clerk to that effect.

[0041] In addition, the monitoring server 12 stores information other than face images among the information of the persons to be monitored. Specifically, information such as person ID, person name (common name), gender, and category for each person to be monitored is stored. These pieces of information are presented to the user as a target person list together with the face images. Further, the monitoring server 12 stores information associated with the face images of the visitors. Specifically, information such as person ID, camera ID, store visit date and time, and registration date and time for each visitor is stored. Also, the monitoring server 12 stores information (reporting history information) regarding the reports made in the past for each person to be monitored.

[0042] In response to a face matching request from the monitoring server 12, the face authentication server 13 performs face matching by comparing the face image of a visitor with the face image of a person to be monitored, and determines whether the visitor is a person to be monitored. Also, the face authentication server 13 (registered image storage unit) stores the face images of the persons to be monitored. Here, the face authentication server 13 usually sets a matching group using the face images of legitimate target persons and executes face authentication processing, such as entry / exit management by face authentication and settlement by face authentication. However, by newly setting a matching group for the face images of the persons to be monitored and executing face authentication processing, the reporting function of the persons to be monitored can be realized.

[0043] The user management server 14 holds user information. The user management server 14 cooperates with the monitoring server 12 to manage the login of users (monitors) on the monitoring terminal 4. Specifically, when a user logs in on the monitoring terminal 4 and the monitoring terminal 4 accesses the user management server 14, user authentication is performed on the user management server 14, and if the user authentication is successful, the login is permitted.

[0044] The personal information storage server 15 stores the personal information of the visitors. Specifically, the personal information storage server 15 (visitor image storage unit) acquires and stores the face images and full-body images of the visitors from the control terminal 3 via the monitoring server 12. Also, the personal information storage server 15 acquires and stores the face images and full-body images regarding the persons to be monitored for whom reports have been made from the monitoring server 12.

[0045] Incidentally, in this system, when a person to be monitored, that is, a person who has taken problematic actions in the store, is discovered by a person engaged in the work (such as a store clerk or a security guard), the face image of that person is registered in the database of the face authentication server 13, and information other than the face image regarding that person is registered in the database of the monitoring terminal 4 (new registration). In this case, the user can register the face image of that person by finding a captured image in which that person appears from the recorded videos of the face camera 1 for face comparison or the in-store monitoring camera 2 and cutting out and creating a face image from the captured image using an image editing application or the like. Also, the face image of that person can be registered by searching for the face image of that person from among the face images of customers stored in the personal information storage server 15. At this time, the user can specify search conditions (recording date and time and recording location) based on the date and time and location where that person was discovered, and search for face images corresponding to the search conditions from among the face images of customers stored in the personal information storage server 15, and the face images presented to the user are narrowed down.

[0046] Also, regarding a person already registered as a monitoring target, the user can additionally register a face image when there are few registered face images or when no appropriate face image is registered (additional registration). In particular, in the case of a person whose degree or frequency of problematic actions makes it desirable not to miss them as much as possible, by additionally registering a face image and increasing the variations of the face images, that person can be reliably detected. In this case, the user can specify an appropriate face image from among the already registered face images of that person as a search source, and face images similar to the search source face image are searched from among the face images of customers stored in the personal information storage server 15, and the corresponding face images are presented to the user.

[0047] Next, the schematic configurations of the monitoring server 12 and the monitoring terminal 4 will be described. FIG. 2 is a block diagram showing the schematic configuration of the monitoring server 12. FIG. 3 is a block diagram showing the schematic configuration of the monitoring terminal 4. FIG. 4 is a block diagram showing the outline of the process related to visitor monitoring. FIG. 5 is a block diagram showing the outline of the process related to face image registration. FIG. 6 is a block diagram showing the outline of the process related to target person management. FIG. 7 is a block diagram showing the outline of the process related to confirmation of notification history.

[0048] As shown in FIG. 2, the monitoring server 12 includes a communication unit 31, a storage unit 32, and a processor 33.

[0049] The communication unit 31 communicates with the control terminal 3, the monitoring terminal 4, the recording server 11, the face authentication server 13, the user management server 14, and the personal information storage server 15.

[0050] The storage unit 32 stores programs executed by the processor 33 and the like. Further, the storage unit 32 stores information other than face images among the information of the monitoring target persons, specifically, information such as person ID, person name (common name), gender, and category for each of the monitoring target persons. In addition, the storage unit 32 stores information associated with the face images of the visitors, specifically, information such as person ID, camera ID, visit date and time, and registration date and time for each of the visitors. Further, the storage unit 32 stores information related to notifications made in the past for each monitoring target person (notification history information).

[0051] The processor 33 performs various processes by executing the programs stored in the storage unit 32. In the present embodiment, the processor 33 performs processes such as visitor monitoring processing, face search processing, notification history management processing, and target person management processing.

[0052] The visitor monitoring process includes a notification event detection process, a notification determination process, and a notification control process.

[0053] As shown in FIG. 4, in the event detection process, the processor 33 detects the arrival of the monitored person, which is the target event to be reported. Specifically, the face image of the arriving person acquired from the control terminal 3 is provided to the face authentication server 13, and the face authentication server 13 performs a face matching process to compare the face image of the arriving person with the face image of the monitored person. As the face matching result, the success or failure of the face matching, and when the face matching is successful, the ID of the monitored person determined to be the same person as the arriving person and the matching score are acquired. The matching score represents the similarity between the face image of the arriving person and the face image of the monitored person.

[0054] In the event reporting determination process, when the processor 33 detects an event to be reported (the arrival of the monitored person) through the event detection process, it determines whether to report according to whether the event to be reported meets the predetermined event reporting conditions. Specifically, it is determined whether the matching score acquired as the face matching result is equal to or higher than the alarm sensitivity (threshold for event reporting determination). At this time, the processor 33 acquires the alarm sensitivity associated with the ID of the monitored person as the face matching result acquired from the face authentication server 13 and makes a determination.

[0055] In the event reporting control process, when the detected event to be reported meets the event reporting conditions, that is, when the matching score is equal to or higher than the alarm sensitivity, the processor 33 performs predetermined event reporting control on the monitoring terminal 4, the warning light 5, and the staff terminal 6. Specifically, the display information of the event reporting screen 401 (see FIG. 15) is transmitted to the monitoring terminal 4. Also, an instruction to turn on the lamp and output an alarm sound as the event reporting operation is transmitted to the warning light 5. Further, an e-mail including the event reporting information is transmitted to the staff terminal 6.

[0056] As shown in FIG. 2, the face search process includes an image search process, a face state detection process, and a search result presentation process. In the face search process, the face search screen 101 (see FIG. 11) is displayed on the display 42 of the monitoring terminal 4.

[0057] As shown in FIG. 5, in the image search process, the processor 33 acquires the search conditions (shooting date and shooting location) specified by the user on the face search screen 101 (see FIG. 11), and requests the personal information storage server 15 to provide a face image corresponding to the search conditions (shooting date and shooting location) from among the face images of past visitors stored in the personal information storage server 15.

[0058] Also, in the image search process, when the user designates a face image as the search source on the face search screen 101, the processor 33 causes the face authentication server 13 to execute a face matching process, and selects a face image similar to the search source face image from among the face images of past visitors corresponding to the search conditions (shooting date and shooting location). In the face matching process of the face authentication server 13, the face images of past visitors corresponding to the search conditions are compared with the search source face image, and a matching score representing the similarity between the two is obtained. In the image search process, a face image for which the matching score is equal to or greater than a predetermined threshold for face search is selected as the search result, that is, a face image similar to the search source face image. Note that the threshold for face search may be set lower than the threshold for face authentication.

[0059] In the face state detection process, the processor 33 detects a characteristic state of the face as face reference information that clarifies the differences from the search source face image for each face image of the search result. Specifically, characteristic states regarding accessories (such as glasses, hats) and the orientation of the face (such as sideways) are detected. Note that the face images of the search result and the search source face image may be compared, and information regarding the differences between the two (information regarding the relationship) may be detected as face reference information.

[0060] In the search result presentation process, the processor 33 displays the face images selected as the search result in the image search process on the face search screen 101 (see FIG. 11). Also, in the search result presentation process, the processor 33 displays face reference information representing the characteristic state of the face detected in the face state detection process on the face search screen 101. Further, in the search result presentation process, the processor 33 displays the matching score obtained in the face matching process performed by the face authentication server 13 on the face search screen 101 as the similarity between the search source face image and the image of the search result.

[0061] As shown in FIG. 6, in the transmission history management process, each time a transmission corresponding to a transmission event (the arrival of a monitored person at the store) occurs in the transmission control process, the processor 33 accumulates information regarding the transmission event in the storage unit 32 as transmission history information.

[0062] Also, as shown in FIG. 7, in the transmission history management process, the processor 33 acquires accumulated transmission information (identification information of a transmission event in the accumulated state) from the monitoring terminal 4, and based on the accumulated transmission information, acquires transmission history information regarding the transmission event in the accumulated state from the storage unit 32, and provides the transmission history information to the monitoring terminal 4.

[0063] As shown in FIG. 6, in the target person management process, the processor 33 acquires registration information regarding each monitored person based on the content operated by the user on the target person list screen 201 (see FIG. 13) and the target person registration / editing screen 301 (see FIG. 14) displayed on the monitoring terminal 4, and stores it in the storage unit 32. Also, in the target person management process, the processor 33 provides the registration information regarding each monitored person stored in the storage unit 32 to the monitoring terminal 4, and causes the monitoring terminal 4 to display the target person list screen 201 (see FIG. 13) and the target person registration / editing screen 301 (see FIG. 14).

[0064] As shown in FIG. 3, the monitoring terminal 4 includes a communication unit 41, a display 42, an input device 43, a storage unit 44, and a processor 45.

[0065] The communication unit 41 communicates with the recording server 11, the monitoring server 12, and the user management server 14.

[0066] The display 42 displays various screens. The input device 43 is for the user to operate on the screen displayed on the display 42.

[0067] The storage unit 44 stores programs executed by the processor 45 and the like.

[0068] The processor 45 performs various processes by executing the programs stored in the storage unit 44. In the present embodiment, the processor 45 performs screen control processing, transmission accumulation information acquisition processing, and the like.

[0069] In the screen control processing, the processor 45 generates display information regarding various screens based on the information provided from the monitoring server 12 and displays the screen on the display 42 based on the display information. Also, in the screen control processing, the processor 45 acquires operation input information in response to an operation of the input device 43 by the user and executes a predetermined process based on the operation input information.

[0070] Specifically, as shown in FIG. 4, when monitoring visitors, the transmission screen 401 (see FIG. 15) is displayed on the display 42. Also, as shown in FIG. 5, when registering a face image, the face search screen 101 (see FIGS. 11 and 12) is displayed on the display 42. Also, as shown in FIG. 6, when managing target persons, the target person list screen 201 (see FIG. 13) and the target person registration / editing screen 301 (see FIG. 14) are displayed on the display 42. Also, as shown in FIG. 7, when confirming the transmission history, the transmission accumulation details screen 601 (see FIG. 16) and the transmission history list screen 701 (see FIG. 17) are displayed on the display 42 after the transmission screen 401 (see FIG. 15).

[0071] As shown in FIG. 7, in the transmission accumulation information acquisition processing, the processor 45 acquires information (transmission accumulation information) regarding transmission events in an unconfirmed accumulation state by the user. Specifically, the total number of transmission events in an unconfirmed accumulation state by the user (total transmission accumulation number) and the number of monitored target persons related to the transmission events (number of target persons) are obtained. Also, the number of transmission events for each monitored target person in an unconfirmed accumulation state by the user (individual transmission accumulation number) is obtained.

[0072] Note that in the present embodiment, the transmission accumulation information acquisition processing is performed by the monitoring terminal 4, but the transmission accumulation information acquisition processing may be performed by the monitoring server 12.

[0073] Next, an overview of the processing during customer monitoring performed by this system will be described. Figure 8 is a sequence diagram showing an overview of the processing at the time of notification.

[0074] The face recognition camera 1 captures a person (customer) entering the store from the store entrance and transmits the video to the control terminal 3. The control terminal 3 extracts the face image and full body image of the customer from the video received from the face recognition camera 1, and transmits the face image and full body image of the customer to the monitoring server 12.

[0075] The monitoring server 12 transmits the face image and full body image of the customer received from the control terminal 3 to the personal information storage server 15. The personal information storage server 15 accumulates the face image and full body image of the customer received from the monitoring server 12, and transmits a response of image accumulation to the monitoring server 12. The monitoring server 12 stores the accompanying information (such as the shooting date and time) of the face image in its own database.

[0076] The monitoring server 12 transmits a face verification request including the face image of the customer to the face authentication server 13. The face authentication server 13 performs a face verification to compare the face image of the customer with the face image of the person to be monitored in response to the face verification request, and determines whether the customer is the person to be monitored, and transmits a face verification response including the face verification result to the monitoring server 12.

[0077] Based on the face verification result included in the face verification response received from the face authentication server 13, the monitoring server 12 determines whether to issue a notification regarding the arrival of the person to be monitored. Here, if it is determined to issue a notification, the monitoring server 12 performs predetermined notification control on the monitoring terminal 4, the warning light 5, and the staff terminal 6. Specifically, the monitoring server 12 transmits the display information of the notification screen 401 to the monitoring terminal 4. In addition, the monitoring server 12 transmits an instruction to turn on the lamp and output a warning sound as a notification operation to the warning light 5. In addition, the monitoring server 12 transmits an email including the notification information to the staff terminal 6.

[0078] Next, the processing during visitor monitoring performed by the monitoring server 12 will be described. FIG. 9 is a flowchart showing the procedure of the processing during visitor monitoring performed by the monitoring server 12.

[0079] In the monitoring server 12, first, a face image of a visitor transmitted from the control terminal 3 is received (ST101). At this time, in the control terminal 3, processing for extracting the face image of the visitor from the captured image output from the face photographing camera 1 is performed.

[0080] Next, in the monitoring server 12, based on the control of the processor 33, the communication unit 31 transmits a face verification request to the face authentication server 13 (ST102). The face verification request includes the face image of the visitor.

[0081] At this time, in response to the face verification request from the monitoring server 12, the face authentication server 13 compares the face image of the visitor with the face image of the person to be monitored, obtains a verification score representing the similarity between the two, compares the verification score with a predetermined threshold for face verification, and determines whether the face verification has succeeded (face verification process). Specifically, it is determined that the face verification has succeeded when the verification score is equal to or higher than the threshold. Next, the face authentication server 13 transmits a response to the face verification to the monitoring server 12. The response to the face verification includes the success or failure of the face verification as the face verification result. Further, when the face verification has succeeded, the response to the face verification includes the ID of the person to be monitored determined to be the same person as the visitor and the verification score.

[0082] Next, in the monitoring server 12, the communication unit 31 receives the response to the face verification from the face authentication server 13 (ST103). Next, in the monitoring server 12, the processor 33 determines whether the face verification has succeeded based on the face verification result included in the response to the face verification (ST104).

[0083] Here, when face verification is successful (Yes in ST104), next, the processor 33 determines whether the reporting condition is satisfied, specifically, whether the verification score is equal to or higher than the alarm sensitivity (threshold value for reporting determination) (reporting determination process) (ST105). At this time, the processor 33 obtains the alarm sensitivity associated with the ID of the person under surveillance included in the response of the face verification and makes the determination.

[0084] Here, when the reporting condition is satisfied, that is, when the verification score is equal to or higher than the alarm sensitivity (Yes in ST105), next, the processor 33 performs a predetermined reporting control process on the monitoring terminal 4, the warning lamp 5, and the staff terminal 6 (ST106). At this time, for example, on the monitoring terminal 4, a reporting screen 401 (see FIG. 15) is displayed as a reporting operation. Further, on the reporting screen 401, the verification score included in the response of the face verification is displayed as the similarity between the face image of the person under surveillance determined to be the same person in the face verification and the face image of the visitor. When a report is sent to the staff terminal 6, the reporting screen 401 can be displayed by an operation of instructing the link displayed in the received e-mail.

[0085] On the other hand, when the face verification fails, that is, when the verification score is less than the threshold value for face verification (No in ST104), or when the reporting condition is not satisfied, that is, when the verification score is less than the alarm sensitivity (threshold value for reporting determination) (No in ST105), the processor 33 performs a process of discarding the face verification result (ST107).

[0086] Incidentally, the threshold value for face verification is set to a uniform value and the face verification process is performed. On the other hand, the alarm sensitivity (threshold value for reporting determination) is set individually for each person under surveillance according to the user's designation. For example, the alarm sensitivity is set low for a person who should not be missed as much as possible. As a result, although the possibility of false alarms is high, reporting is surely carried out. Also, the alarm sensitivity is set high for a person with significant false alarms. Thereby, false alarms can be suppressed.

[0087] In addition, in this embodiment, categories (for example, most important, important, attention) are individually assigned to the persons to be monitored. Therefore, the alarm sensitivity for each person to be monitored may be set according to the category to which the person to be monitored belongs, regardless of the user's designation. For example, the alarm sensitivity may be set low for a person whose category is most important, and may be set high for a person whose category is attention.

[0088] Next, the face search process performed by the monitoring server 12 will be described. FIG. 10 is a flowchart showing the procedure of the face search process.

[0089] In the monitoring server 12, first, in response to an operation in which the user designates a source face image and other search conditions (shooting date and time, and shooting location) on the monitoring terminal 4, the processor 33 acquires the source face image and other search conditions (ST201). Specifically, a face search screen 101 (see FIG. 11) is displayed on the monitoring terminal 4, and the user performs an operation of designating a source face image and other search conditions on this face search screen 101.

[0090] Next, in the monitoring server 12, based on other search conditions (shooting date and time, and shooting location), the processor 33 acquires, from the personal information storage server 15, face images of past visitors stored in the personal information storage server 15 that correspond to the search conditions (shooting date and time, and shooting location) (ST202). At this time, based on the control of the processor 33, the communication unit 31 transmits a request for a face image to the personal information storage server 15, and receives, from the personal information storage server 15, a face image that corresponds to the search conditions as a response to the face image request.

[0091] Next, in the monitoring server 12, based on the control of the processor 33, a similarity determination loop is started (ST203), and the communication unit 31 transmits a face verification request to the face authentication server 13 (ST204). The face verification request includes the source face image designated by the user and the face images of past visitors that correspond to the search conditions (shooting date and time, and shooting location).

[0092] At this time, in response to the face matching request from the monitoring server 12, the face authentication server 13 compares the source face image with the face images of past visitors, and obtains a matching score representing the similarity between the two (face matching process). Next, the face authentication server 13 sends a response of the face matching to the monitoring server 12. The response of the face matching includes the similarity (matching score) regarding each face image that is the matching result.

[0093] Next, in the monitoring server 12, the communication unit 31 receives the response of the face matching sent from the face authentication server 13 (ST205).

[0094] Next, in the monitoring server 12, the processor 33 determines whether the similarity (matching score) regarding each face image included in the response of the face matching is equal to or greater than a predetermined threshold for face search (ST206).

[0095] Here, if the similarity of the face image to be determined is less than the threshold for face search (No in ST206), the processor 33 performs a process of discarding the face matching result of the face image (ST208).

[0096] On the other hand, if the similarity of the face image to be determined is equal to or greater than the threshold for face search (Yes in ST206), the processor 33 sets the face image to be determined as a face image (registered candidate face image) to be presented as a search result (ST207).

[0097] When the process ends for all the face images included in the response of the face matching, the similarity determination loop ends (ST209). Next, the processor 33 presents the face images of the search results (registered candidate face images) to the user (ST210). At this time, the matching scores of each face image included in the response of the face matching are presented to the user as the similarity with the source face image specified by the user. Specifically, on the face search screen 101 (see FIG. 11) displayed on the monitoring terminal 4, the face images of the search results are displayed together with the similarity.

[0098] In addition, in this embodiment, the number of face images (registered candidate face images) presented to the user as search results is limited to the upper limit value specified by the user. When the number of face images with a similarity equal to or higher than the threshold exceeds the upper limit value, face images are selected in order from the highest similarity until the number of face images reaches the upper limit value.

[0099] In the example shown in FIG. 10, a similar image search is performed with the face image of the search source specified. However, the face image search process may be performed based only on other search conditions (date and time of shooting and shooting location) without specifying the face image of the search source.

[0100] Next, the face search screen 101 displayed on the monitoring terminal 4 will be described. FIGS. 11 and 12 are explanatory diagrams showing the face search screen 101.

[0101] When the user newly registers a face image of a person to be monitored or adds and registers a face image different from the registered face image, in the face search screen 101, the user inputs search conditions to execute the face image search process and views the search results, thereby being able to find an appropriate face image.

[0102] The face search screen 101 is provided with a tab 102 for "alarm", a tab 103 for "face search", a tab 104 for "target list", a tab 105 for "device monitoring", and a tab 106 for "settings". The face search screen 101 is displayed when the user operates the tab 103 for "face search". When the user operates the tab 102 for "alarm", the system transitions to the alarm history list screen 701 (see FIG. 17). When the user operates the tab 104 for "target list", the system transitions to the target list screen 201 (see FIG. 13). When the user operates the tab 105 for "device monitoring", the system transitions to a device monitoring screen (not shown). When the user operates the tab 106 for "settings", the system transitions to a screen (not shown) related to system settings, operation settings, and user settings.

[0103] In addition, on the face search screen 101, a display box 107 for the "store name" and a display box 108 for the "user name" are provided. In the display box 107 for the "store name", the name of the target store is displayed. Also, in the display box 107 for the "store name", the user can switch the target store by operating the pull-down menu. In the display box 108 for the "user name", the user name of the currently logged-in user is displayed. Also, in the display box 108 for the "user name", the user can switch the currently logged-in user by instructing the execution of logout by operating the pull-down menu.

[0104] In addition, on the face search screen 101, a search condition setting section 111, a "search" button 112, and a search result display section 113 are provided. In the search condition setting section 111, the user can specify search conditions. When the user specifies search conditions in the search condition setting section 111 and operates the "search" button 112, the search results are displayed in the search result display section 113. In this embodiment, among the face images of visitors stored in the personal information storage server 15, the face images (registration candidate face images) that meet the search conditions are displayed as search results. Also, when the user designates a source face image (search face image), a similar image search is performed, and the face images similar to the source face image are displayed as search results.

[0105] The search condition setting section 111 is provided with a search target period specifying section 121, a source face image specifying section 122, a search target camera specifying section 123, and a search result display number specifying section 124.

[0106] In the search target period specifying section 121, the user can specify the search target period (start date / time, end date / time). The search target period is set based on the date / time when the visit of the person of interest is predicted. In the face image search process, the face images whose shooting date / time is included in the search target period are selected as search results.

[0107] In the search target period specifying section 121, the user can input the start date and time into the input box 132 by turning on (checking) the "Start" checkbox 131. Also, the user can input the end date and time into the input box 134 by turning on the "End" checkbox 133. If both the "Start" checkbox 131 and the "End" checkbox 133 are off (unchecked), the current date and time is set as the end date and time, and the time one hour before the current date and time is set as the start date and time. Also, if the "Start" checkbox 131 is off and the "End" checkbox 133 is on, the time one hour before the input end date and time is set as the start date and time. If the "Start" checkbox 131 is on and the "End" checkbox 133 is off, the time one hour after the input start date and time is set as the end date and time.

[0108] In the search source face image specifying section 122, the user can specify the search source face image when performing a similar image search. Also, in the search source face image specifying section 122, the user can specify a threshold value regarding similarity for narrowing down the search results. The search source face image specifying section 122 is provided with two face image display frames 135, an input box 136 for similarity, and a "Clear" button 137.

[0109] The face image specified by the user is displayed in the face image display frame 135. When the user operates the face image display frame 135, an operation menu (not shown) is displayed. When the user selects the "File Selection" item in the operation menu, a file selection dialog (not shown) is displayed, and in this file selection dialog, the user can select a face image created and saved in advance as the search source face image. Also, when the user selects the "Select from Target Person List" item in the operation menu, the user is redirected to the target person list screen 201 (see Fig. 13), and when the user selects a person under surveillance here, the representative face image related to the selected person under surveillance is set as the search source face image. Also, when the user selects the "Clear" item in the operation menu, the set face image is deleted.

[0110] In the case of newly registering a face image of a person to be monitored, a face image acquired by the user from a video or the like at the date and time when the person to be monitored was discovered may be set as the face image for search. In the case of additionally registering a face image of a person to be monitored, a registered face image, particularly a representative face image, may be set as the face image for search.

[0111] In the similarity input box 136, the user can specify the similarity threshold used when selecting a face image from the search results. In the similar image search, a face image whose similarity to the face image for search is equal to or higher than the threshold is selected as the search result. Note that an initial value of the similarity threshold is set in advance, and when the user has not input, the initial value is displayed in the input box 346.

[0112] When the user operates the "Clear" button 137, the set face image for search is deleted and the similarity is initialized.

[0113] Note that the image search process may be performed without specifying the face image for search in the face image for search specifying unit 122. In this case, the similar image search is not performed, and a face image corresponding to the search conditions regarding the date and time and the location, specifically, the search conditions specified by the search target period specifying unit 121 and the search target camera specifying unit 123 is searched.

[0114] The search target camera specifying unit 123 is provided with a check box 138 for each face detection camera 1. The user can specify whether or not to target each face detection camera 1 for search by operating the check box 138. By specifying the face detection camera 1, the user can limit the face image displayed as the search result by the shooting location. The user specifies the face detection camera 1 based on the location where the appearance of the person of interest is predicted.

[0115] The search result display number specifying section 124 is provided with an input box 139 for the "search result display number". The user can input the upper limit value of the number of face images to be displayed as search results in the input box 139 in the search result display section 113. When the upper limit value of the number of face images is set, face images with high similarity within the range of the upper limit value are selected and displayed in the search result display section 113.

[0116] The search result display section 113 is provided with a tab 141 for "time / location" and a tab 142 for "list". The example shown in FIG. 11 is a case of the first display mode displayed when the user operates the tab 141 for "time / location". The example shown in FIG. 12 is a case of the second display mode displayed when the user operates the tab 142 for "list".

[0117] As shown in FIG. 11, in the case of the first display mode, the face images of the search results are arranged side by side according to the shooting time and shooting location. Specifically, the face images of the search results are divided and arranged for each face recognition camera 1 corresponding to the shooting location. In addition, the face images taken by the same face recognition camera 1 are arranged side by side in the horizontal direction in chronological order according to the shooting time.

[0118] On the other hand, as shown in FIG. 12, in the case of the second display mode, the face images of the search results are arranged in descending order of similarity. Also, when the face image of the search source is not set as a search condition, that is, when similar image search is not performed, the face images of the search results are arranged in chronological order according to the shooting time.

[0119] Note that among the face images presented to the user as search results, there are those in which a visitor is detected as the same person as the monitored person by face recognition during visitor monitoring and a report is issued, and there are also those in which the visitor is not detected as the same person as the monitored person.

[0120] The search result display unit 113 is provided with a check box 143 for "displaying a full-body image". The example shown in FIG. 11 is the case where the check box 143 for "displaying a full-body image" is off (unchecked). The example shown in FIG. 12 is the case where the check box 143 for "displaying a full-body image" is on (checked).

[0121] As shown in FIG. 11, when the check box 143 for "displaying a full-body image" is off, only the face image of the search result is displayed on the search result display unit 113. Also, as shown in FIG. 12, when the check box 143 for "displaying a full-body image" is on, the face image of the search result and the full-body image of the same person as that face image are displayed on the search result display unit 113. Note that the full-body image may be an image including the entire body of a person, or an image including a body part above the upper body. Since the user can confirm that it is the person being monitored from the appearance such as the clothes shown in the full-body image, the user can efficiently select the face image of the person being monitored with reference to the full-body image.

[0122] The search result display unit 113 is provided with an image display size adjustment unit 144. In the image display size adjustment unit 144, when the user operates the "small" button, the search result image (face image, full-body image) is displayed enlarged by one step. When the user operates the "large" button, the search result image is displayed reduced by one step. When the user operates the slider of the slider bar, the search result image is enlarged or reduced step by step and displayed.

[0123] The search result display unit 113 is provided with a menu button 151 for each face image of the detection result. When the user operates the menu button 151, an operation menu (not shown) is displayed.

[0124] When the user selects the item "Set for face search" in the operation menu, the face image corresponding to the operated menu button 151 is set as the search source face image in a state where it overwrites the set search source face image displayed in the face image display frame 135. This operation can be performed individually for the set search source face images displayed in each of the two face image display frames 135.

[0125] Also, when the user selects the item "New registration" in "Register to target person list" in the operation menu, the target person registration editing screen 301 (see FIG. 14) is displayed. On the target person registration editing screen 301, the face image corresponding to the operated menu button 151 is displayed at the top of the registered face image list display section 323 and is also displayed as the representative face image. Also, when the user selects the item "Additional registration" in "Register to target person list" in the operation menu, the target person list screen 201 (see FIG. 13) is displayed. Then, when the user selects a person to be monitored on the target person list screen 201, the target person registration editing screen 301 is displayed. On the target person registration editing screen 301, the face image corresponding to the operated menu button 151 is displayed in a state where it is added at the end of the registered face image list display section 323. Note that on the target person registration editing screen 301, when the user operates the "Confirm" button 302, the face image corresponding to the operated menu button 151 is registered in association with the person to be monitored.

[0126] Also, when the user selects the item "Playback from shooting time" in the operation menu, the video monitoring screen 501 (see FIG. 15) is displayed. On the video monitoring screen 501, the video of the face camera 1 including the shooting image that is the source of the face image corresponding to the operated menu button 151 is played back from the shooting date and time. Also, when the user selects the item "Display in original size" in the operation menu, the face image corresponding to the operated menu button 151 is pop-up displayed in the original size.

[0127] In addition, in the search result display section 113, for each face image in the search results, a shooting date and time display column 152 and a similarity display column 153 are provided. In the shooting date and time display column 152, the shooting date and time of the face image in the search results are displayed. In the similarity display column 153, the similarity between the face image in the search results and the reference face image is displayed.

[0128] In addition, in the search result display section 113, a face reference information display column 154 is provided for each face image in the search results. In the face reference information display column 154, information regarding characteristic states of the face, as face reference information for clarifying differences from the reference face image, is displayed for each face image in the search results. Specifically, characters representing characteristic states regarding face accessories (such as glasses, hats, etc.) and the orientation of the face (such as sideways) are displayed. These reference information may be displayed not only as characters but also as fixed-form images such as icons.

[0129] As described above, in this embodiment, as information for reference when the user selects a face image to register from among the face images of the detection results (registered candidate face images), the similarity between the reference face image and the face images in the search results, and face reference information regarding the characteristic states of the face in the face images of the search results are displayed. Thereby, the user can quickly identify the face image of the monitored person of interest and can quickly find face images in characteristic states different from the registered face images.

[0130] Next, the target person list screen 201 displayed on the monitoring terminal 4 will be described. FIG. 13 is an explanatory diagram showing the target person list screen 201.

[0131] On the target person list screen 201, a "New Registration" button 202 and an "Import" button 203 are provided. When the user operates the "New Registration" button 202, the screen transitions to the target person registration editing screen 301 (see FIG. 14). When the user operates the "Import" button 203, a target person list import screen (not shown) is displayed. On the target person list import screen, the user can instruct the reading of the target person list file stored in the folder specified by the user.

[0132] In addition, a registered user number display section 204 is provided on the target person list screen 201. The total number of registered surveillance target persons is displayed on the registered user number display section 204.

[0133] In addition, a target person specific display section 211 for displaying information for each surveillance target person is arranged on the target person list screen 201.

[0134] The target person specific display section 211 is provided with a category display column 221, a face image display column 222, a registration date and time display column 223, a person name display column 224, an age display column 225, a gender display column 226, and a remarks display column 227. A category mark and category name representing the category to which the surveillance target person belongs are displayed in the category display column 221. A representative face image of the surveillance target person is displayed in the face image display column 222. The date and time when the surveillance target person was registered is displayed in the registration date and time display column 223. The name (common name) given to the surveillance target person is displayed in the person name display column 224. The age of the surveillance target person is displayed in the age display column 225. The gender (male, female, unspecified) of the surveillance target person is displayed in the gender display column 226. Reference information regarding the surveillance target person is displayed as remarks in the remarks display column 227.

[0135] In addition, a check box 231 for "alarm target" is provided in the target person specific display section 211. By operating the check box 231 for "alarm target", the user can switch whether or not to set the corresponding surveillance target person as an alarm target. Specifically, when the check box 231 for "alarm target" is checked, the corresponding surveillance target person is set as an alarm target. In this case, when face verification is successful and the visitor is determined to be a surveillance target person and the alarm condition is satisfied, an alarm is issued. On the other hand, when the check box 231 for "alarm target" is not checked and the corresponding surveillance target person is not set as an alarm target, even if face verification is successful and the visitor is determined to be a surveillance target person and the alarm condition is satisfied, no alarm is issued.

[0136] In addition, the per-subject display section 211 is provided with a check box 232 for "Delete from the list". By operating the check box 232 for "Delete from the list", the user can switch whether to exclude the corresponding monitored subject from the monitoring targets. Specifically, when the check box 232 for "Delete from the list" is checked, the corresponding monitored subject is excluded from the monitoring targets.

[0137] In addition, the per-subject display section 211 is provided with a button 233 for "Go to detailed settings". When the user operates the button 233 for "Go to detailed settings", the screen transitions to the subject registration editing screen 301 (see FIG. 14) for the corresponding monitored subject.

[0138] In addition, the per-subject display section 211 is provided with a reporting elapsed period display column 241 and a reporting count display column 242. In the reporting elapsed period display column 241, as summary information on the reporting history, the reporting elapsed period for the monitored subject, that is, the elapsed period from the last report to the present (the most recent non-reporting period) is displayed. In the reporting count display column 242, as summary information on the reporting history, the number of reports for the monitored subject, that is, the number of times reports have been made in the past is displayed. Based on the reporting elapsed period and the reporting count, the user can determine whether to exclude the person registered as a monitored subject from the monitoring targets or whether to exclude the person registered as a monitored subject from the reporting targets. For example, the user can determine to exclude from the reporting targets when the reporting elapsed period is equal to or longer than a predetermined period (for example, one year) defined by the operation rules.

[0139] In addition, in the reporting elapsed time display column 241, when the reporting elapsed time (reporting history summary information) of the monitored person corresponds to a predetermined attention - calling level, the reporting elapsed time of that monitored person is highlighted. Specifically, when the reporting elapsed time is equal to or greater than a predetermined threshold value defined by the operation rules, the reporting elapsed time is highlighted. Also, in the highlighting of the reporting elapsed time, for example, the characters representing the reporting elapsed time are displayed in different colors or thicknesses. Also, the background of the reporting elapsed time display column 241 is displayed in different colors or patterns. Thereby, the user can easily grasp the monitored person who should be noted. Note that in the reporting count display column 242, when the reporting count (reporting history summary information) corresponds to a predetermined attention - calling level, the characters representing the reporting count or the reporting count display column 242 may be highlighted.

[0140] In addition, the target - person - specific display section 211 is provided with a registered face - image count display column 243. In the registered face - image count display column 243, the number of registered face images, that is, the number of face images registered for the monitored person, is displayed. The user can judge the necessity of additional registration of face images based on the number of registered face images.

[0141] Also, on the target - person list screen 201, sort buttons 251 (operation section) for descending and ascending order are provided for each item of category, registration date and time, person name, age, gender, alarm target, reporting elapsed time, reporting count, and number of registered face images. When the user operates either the descending - order or ascending - order sort button 251, the target - person - specific display section 211 is sorted in descending or ascending order for each item.

[0142] Also, on the target - person list screen 201, a button 205 for "confirm" and a button 207 for "cancel" are provided. When the user operates the button 205 for "confirm", the content input this time on the target - person list screen 201 is set. When the user operates the button 207 for "cancel", the content input this time on the target - person list screen 201 is discarded.

[0143] Next, the subject registration / editing screen 301 displayed on the monitoring terminal 4 will be described. FIG. 14 is an explanatory diagram showing the subject registration / editing screen 301.

[0144] The subject registration / editing screen 301 is provided with a registration method guidance section 311 and a face image registration section 312. In the registration method guidance section 311, characters guiding the first face image registration method are displayed. In the first face image registration method, a face image can be registered from an operation menu displayed by operating the menu button 151 of the face image on the face search screen 101 (see FIG. 11). Also, a face image can be registered from an operation menu displayed by operating the menu button 652 of the transmission accumulation details screen 601 (see FIG. 16). In the face image registration section 312, the user can select a face image file in order to register a face image according to the second face image registration method. When the user operates the "Reference" button 313, a file selection dialog (not shown) is displayed, and a face image file can be selected in this file selection dialog.

[0145] In addition, the subject registration / editing screen 301 is provided with a representative face image display section 321 and a check box 322 for "always display on the alarm list screen". In the representative face image display section 321, a representative face image of the monitored person is displayed. The representative face image is selected by the user from the registered face images displayed in the registered face image list display section 323. When there is only one registered face image, that face image is set as the representative face image. Also, the user can select whether to display the representative face image of the monitored person as the face image of an important person on the alarm history list screen 701 (see FIG. 17) by operating the check box 322 for "always display on the alarm list screen".

[0146] In addition, the subject registration / editing screen 301 is provided with a registered face image list display section 323. In the registered face image list display section 323, the registered face images related to the monitored person are arranged and displayed.

[0147] In the registered face image list display section 323, when the user performs an operation to select a face image (for example, mouse over), menu buttons 324 appear for each face image. When the user operates the menu button 324, an operation menu (not shown) is displayed.

[0148] When the user selects the "representative face image" item in the operation menu, the face image corresponding to the operated menu button 324 is set as the representative face image. Also, when the user selects the "clear" item in the operation menu, the face image corresponding to the operated menu button 324 is deleted. Further, when the user selects the "display in original size" item in the operation menu, the face image and the full body image corresponding to the operated menu button 324 are popped up and displayed in the original size.

[0149] In addition, in the registered face image list display section 323, a reporting frequency display column 325 is provided for each registered face image. In the reporting frequency display column 325, the number of times of correct reports and false reports is displayed. Thereby, the user can determine whether the registered face image is appropriate or not. For example, when there are no reports or when the number of false reports is significantly larger compared to the number of correct reports, the user can determine that the registered face image is inappropriate. When the user determines that the registered face image is inappropriate, the user may perform an operation to delete the face image. Also, if there is a shortage of appropriate face images, the user may perform an operation to additionally register a face image.

[0150] In addition, on the subject registration and editing screen 301, a reporting elapsed period display column 331 and a reporting frequency display column 332 are provided. The display contents of the reporting elapsed period display column 331 and the reporting frequency display column 332 are the same as those of the reporting elapsed period display column 241 and the reporting frequency display column 242 on the subject list screen 201.

[0151] In addition, on the subject registration and editing screen 301, a registration information input section 335 is provided.

[0152] The registration information input section 335 is provided with an input box 341 for "person's name", an input box 342 for "age", and a check box 343 for "gender". In the input box 341 for "person's name", the user can input the person's name (common name) of the person to be monitored. In the input box 342 for "age", the user can input the age of the person to be monitored. In the check box 343 for "gender", the user can select the gender (male, female, unspecified) of the person to be monitored.

[0153] The registration information input section 335 is provided with an input box 344 for "remarks". The user can input reference information regarding the person to be monitored as remarks in the input box 344. Specifically, information regarding the importance of monitoring the person to be monitored (e.g., habitual, most important, attention required, monitor immediately upon discovery, etc.) and information regarding the specific actions of the person to be monitored (e.g., theft four times in the past, violence in the store, presence of alcohol, etc.) are input.

[0154] The registration information input section 335 is provided with an input box 345 for "category". The user can select the category (e.g., most important, important, attention) of the person to be monitored in the input box 345. When the category of the person to be monitored is selected, the category mark corresponding to that category is displayed. Note that the candidates for the category can be input by the user on the operation setting screen (not shown).

[0155] The registration information input section 335 is provided with an input box 346 for "alarm sensitivity". The user can input the alarm sensitivity (threshold value for alarm reporting determination) of the person to be monitored in the input box 346. Also, an initial value of the alarm sensitivity is set in advance, and the initial value of the alarm sensitivity is displayed in the input box 346 when the user has not input anything.

[0156] In addition, the registration information input unit 335 is provided with an expiration date setting unit 347. The expiration date setting unit 347 is provided with a check box 351 for "expiration date" and an input box 352 for date and time (year, month, day, hour, minute, and second). By operating the check box 351 for "expiration date", the user can switch between the validity and invalidity of the expiration date. Specifically, when the check box 351 for "expiration date" is checked, the expiration date becomes valid and input operations related to the expiration date become possible. The expiration date is the expiration date of the period during which regular monitoring is carried out. When the expiration date elapses, the regular monitoring ends.

[0157] In addition, the expiration date setting unit 347 is provided with a radio button 353 for "alarm stop" and a radio button 354 for "registration cancellation" regarding the action when the expiration date has passed. When the user turns on the radio button 353 for "alarm stop", when the expiration date has passed, the monitored person is excluded from the reporting target, and the check box 231 for "alarm target" on the target person list screen 201 (see FIG. 13) becomes unchecked. When the user turns on the radio button 354 for "registration cancellation", when the expiration date has passed, the monitored person is excluded from the monitoring target, and the monitored person is deleted from the target person list screen 201.

[0158] In addition, the registration information input unit 335 is provided with a button 361 for "reset registration information". When the user operates the button 361 for "reset registration information", the input contents of each item in the registration information input unit 335 are returned to the initial values.

[0159] In addition, on the subject registration editing screen 301, a "Confirm" button 302 and a "Cancel" button 303 are provided. When the user operates the "Confirm" button 302, the content entered this time on the subject registration editing screen 301 is registered. In this case, when returning to the subject list screen 201 (see FIG. 13), the display content is updated according to the content entered this time on the subject registration editing screen 301. Also, when the user operates the "Cancel" button 303, the content entered this time on the subject registration editing screen 301 is discarded and the previous setting content is restored.

[0160] Next, the alarm reporting screen 401 displayed on the monitoring terminal 4 will be described. FIG. 15 is an explanatory diagram showing the alarm reporting screen 401.

[0161] On the monitoring terminal 4, when an alarm report is carried out in the monitoring server 12, the alarm reporting screen 401 is pop-up displayed on the video monitoring screen 501. Note that on the monitoring terminal 4, when the face search screen 101 (see FIG. 11) or the like is in the display state, the alarm reporting screen 401 is pop-up displayed on the face search screen 101.

[0162] Note that on the video monitoring screen 501, a "Video" tab 502, a "Live" tab 503, and a video display unit 504 are provided. The user can play back the video stored in the recording server 11 on the video display unit 504 by selecting the "Video" tab 502. Also, the user can display the real-time video output from the face recognition camera 1 and the in-store monitoring camera 2 on the video display unit 504 by selecting the "Live" tab 503. Also, a camera selection unit 511 is provided on the video monitoring screen 501. In the camera selection unit 511, a checkbox 521 related to the face recognition camera 1 and a checkbox 522 related to the in-store monitoring camera 2 are provided. The user can select the face recognition camera 1 and the in-store monitoring camera 2 for displaying the video on the video display unit 504 by operating the checkboxes 521 and 522.

[0163] On the notification screen 401, a category display section 411, a person name display section 412, and a remarks display section 413 are provided. In the category display section 411, a category mark and a category name are displayed as information regarding the category to which the monitored person, who is the party to the notified event (the arrival of the monitored person at the store), belongs. In the person name display section 412, the person name (common name) of the monitored person is displayed. In the remarks display section 413, reference information regarding the monitored person is displayed.

[0164] Also, on the notification screen 401, a date and time display section 414, a store name display section 415, and a camera name display section 416 are provided. In the date and time display section 414, the date and time (year, month, day, hour, minute, second) when the notification was made is displayed. In the store name display section 415, the name of the store related to the notification is displayed. In the camera name display section 416, the name of the face recognition camera 1 related to the notification is displayed.

[0165] Also, on the notification screen 401, a similarity display section 417 and an image display section 418 are provided. In the similarity display section 417, the similarity between the face image of the arriving customer determined to be a successful face match related to the notification and the face image of the monitored person is displayed. In the image display section 418, the detected image at the time of notification, that is, the full body image and face image of the arriving customer are displayed. Also, in the image display section 418, the representative face image of the monitored person determined to be the same person as the arriving customer is displayed. The user can visually compare the detected image (full body image and face image) at the time of notification with the representative face image of the monitored person to confirm that the arriving customer is the same person as the monitored person, that is, that the current notification is a correct notification.

[0166] In this system, when face matching between the face image of a visitor and any one of a plurality of face images registered for a person under surveillance is successful, the visit of the person under surveillance is detected and a notification is issued. Therefore, the face image of the person under surveillance when the face matching for the notification is successful may be the representative face image or any other registered face image. For this reason, in the example shown in FIG. 15, the representative face image of the person under surveillance is displayed on the image display unit 418, but instead of the representative face image, the face image itself used in the successful face matching may be displayed. Also, when face matching is successful with a plurality of face images, the face image with the highest matching score may be displayed.

[0167] In addition, the notification screen 401 is provided with a button 421 for "Details", a button 422 for "Live Video", and a button 423 for "Playback". When the user operates the button 421 for "Details", the screen transitions to a notification details screen (not shown). On the notification details screen, all the contents of the items that were only partially displayed on the notification screen 401 due to space constraints are displayed. When the user operates the button 422 for "Live Video", the notification screen 401 is closed and the video surveillance screen 501 (see FIG. 15) is displayed, and on the video surveillance screen 501, a live video, that is, a real-time video captured by the face matching camera 1 is displayed. When displaying a real-time video on the surveillance terminal 4, it is possible to directly acquire and display the video captured by the face matching camera 1, but it is also possible to display the video via the recording server 11. When the user operates the button 423 for "Playback", the notification screen 401 is closed and the video surveillance screen 501 is displayed, and on the video surveillance screen 501, a recorded video, that is, a video stored in the recording server 11 is displayed. At this time, the video is played back starting from a predetermined time (for example, 10 seconds) before the notification time.

[0168] In addition, the notification screen 401 is provided with a button 402 for "×". When the user views the notification screen 401 and confirms the content of the generated notification event, the user operates the button 402 for "×". Thereby, the notification screen 401 is closed.

[0169] In addition, an alarm transmission summary display section 431 is provided on the alarm transmission screen 401. The alarm transmission summary display section 431 displays, as alarm transmission summary information, the total number of alarm transmissions (the total number of alarm pop-up accumulations) and the number of target persons. The total number of alarm transmissions represents the total number of alarm events that are in the accumulation state without being confirmed by the user through the popped-up alarm transmission screen 401. The number of target persons represents the number of monitored persons related to the alarm events in the accumulation state. In the example shown in FIG. 15, there are 5 alarm events in the accumulation state, and 3 monitored persons are related to those alarm events.

[0170] Here, on the alarm transmission screen 401, only information regarding the latest alarm event is displayed. For this reason, if a new alarm event occurs while the user does not operate the "×" button 402, that is, while the alarm transmission screen 401 is open, on the alarm transmission screen 401, the display content regarding the previous alarm event is successively overwritten by the display content regarding the newly occurred alarm event. Therefore, just by browsing the alarm transmission screen 401, the user cannot confirm the content of the alarm event before it is overwritten. Thus, in the present embodiment, the alarm transmission summary display section 431 is provided on the alarm transmission screen 401.

[0171] In addition, the alarm events in the accumulation state include the latest alarm event displayed on the currently displayed alarm transmission screen 401 and the alarm events that were overwritten without the user's confirmation previously. Therefore, the total number of alarm transmissions represents the number obtained by adding 1 for the currently displayed alarm event to the number of times the alarm transmission screen 401 has been overwritten due to newly occurring alarm events without the user operating the "×" button 402. Note that after the "×" button 402 is operated and the alarm transmission screen 401 is closed, when a new alarm transmission screen 401 pops up, the total number of alarm transmissions on that alarm transmission screen 401 becomes "1", and thereafter, each time a new alarm event occurs without the user operating the "×" button 402, the total number of alarm transmissions becomes a value increased by 1. In the example shown in FIG. 15, there are 4 alarm events that were overwritten by newly occurring alarm events without the user's confirmation.

[0172] When the user operates the transmission accumulation summary display section 431, the screen transitions to the transmission accumulation details screen 601 (see Fig. 16).

[0173] Next, the transmission accumulation details screen 601 displayed on the monitoring terminal 4 will be described. Fig. 16 is an explanatory diagram showing the transmission accumulation details screen 601.

[0174] On the left side of the transmission accumulation details screen 601, a "List" tab 602 and a "Search" tab 603 are provided. The example shown in Fig. 16 is the case of the list mode displayed when the user selects the "List" tab 602. On the other hand, when the user operates the "Search" tab 603, the screen transitions to a search mode (not shown) where the user can input search conditions to instruct the search for transmission events. In the search mode, similar to the face search screen 101 (see Fig. 11), the user can input detection conditions. Specifically, the user can specify the search target period, the face image of the search source, and the search target camera.

[0175] As shown in Fig. 16, in the list mode, an important person display section 611 is provided on the left side of the transmission accumulation details screen 601. In the important person display section 611, the face images of important persons among the monitored persons are displayed in a list by category (for example, most important, important, attention). Specifically, for each category, a category mark and a category name are displayed, and the representative face image and person name of the important persons belonging to that category are displayed. An important person is a person among the monitored persons for whom the checkbox 322 "Always display on the alarm list screen" is checked on the subject registration / editing screen 301 (see Fig. 14).

[0176] In addition, the transmission report accumulation details screen 601 is provided with a transmission report accumulation summary display section 621. The transmission report accumulation summary display section 621 displays summary information regarding the accumulation status of transmission report events for each person under surveillance. Specifically, the person's name and the individual transmission report accumulation count (number of accumulations) are displayed for each person under surveillance who is the subject of the transmission report event. In the example shown in FIG. 16, there are 5 transmission report events in the accumulation state, there are 3 persons under surveillance related to the transmission report event, and the individual transmission report accumulation counts of the 3 persons under surveillance are 2 times, 2 times, and 1 time, respectively. Note that the transmission report events in the accumulation state include the latest transmission report event and the transmission report event overwritten by a newly generated transmission report event without the user having confirmed it.

[0177] In addition, the transmission report accumulation details screen 601 is provided with a transmission report event list display section 631. In the transmission report event list display section 631, display sections 632 for each transmission report event related to each transmission report event in the accumulation state are arranged side by side. In the example shown in FIG. 16, five display sections 632 for each of the 5 transmission report events related to the latest transmission report event and the 4 transmission report events overwritten by newly generated transmission report events without the user having confirmed them are displayed.

[0178] The event notification type display section 632 is provided with a category display column 641, a notification date and time display column 642, a camera display column 643, an image display column 645, a person name display column 646, a remarks display column 647, and a similarity display column 648. In the category display column 641, a category mark and a category name representing the category to which the monitored person, who is the party to the event notification, belongs are displayed. In the notification date and time display column 642, the date and time when the event notification occurred is displayed. In the camera display column 643, the name of the face camera 1 that captured the face of the customer determined to be the same person as the monitored person is displayed. In the image display column 645, as images related to the event notification, the detection image at the time of notification, that is, the full body image and face image of the customer, and the representative face image of the monitored person determined to be the same person as the customer are displayed. In the person name display column 646, the name (common name) assigned to the monitored person is displayed. In the remarks display column 647, reference information regarding the monitored person is displayed. In the similarity display column 648, the similarity between the face image of the customer and the face image of the monitored person when it is determined that the face matching related to the notification has been successful is displayed.

[0179] In addition, the event notification type display section 632 is provided with a "Delete Alarm" button 651. When the user operates the "Delete Alarm" button 651, a deletion confirmation dialog (not shown) is displayed. When the user performs an operation to approve the deletion of the alarm in the deletion confirmation dialog, the corresponding event notification is deleted. At this time, the user browses the notification screen 401 (see FIG. 15) to determine the correctness of the notification, and if it is determined to be a false alarm, the user operates the "Delete Alarm" button 651.

[0180] In addition, the event notification type display section 632 is provided with a menu button 652. When the user operates the menu button 652, an operation menu (not shown) is displayed. The content of the operation menu is the same as that of the menu button 151 on the face search screen 101 (see FIG. 11).

[0181] In addition, the transmission report accumulation details screen 601 is provided with a "Filter Release" button 661. When the user operates the "Filter Release" button 661, the system transitions to the transmission report history list screen 701 (see FIG. 17).

[0182] Next, the transmission report history list screen 701 displayed on the monitoring terminal 4 will be described. FIG. 17 is an explanatory diagram showing the transmission report history list screen 701.

[0183] The transmission report history list screen 701 is the result of releasing the filter applied to the transmission report accumulation details screen 601, which displays information on each transmission report event by narrowing down to transmission report events in the accumulation state. On the transmission report history list screen 701, all transmission report events that have occurred up to the present, including those in the accumulation state, are listed.

[0184] The transmission report history list screen 701 is provided with a transmission report event list display section 702. The transmission report event list display section 702 is provided with a display section 632 for each transmission report event related to each transmission report event that occurred in the past. The display section 632 for each transmission report event is the same as that of the transmission report accumulation details screen 601 (see FIG. 16).

[0185] In addition, the transmission report event list display section 702 is provided with a fixed display section 711 and a scroll display section 712. The fixed display section 711 displays the most recent predetermined number of (for example, 3) transmission report events. The scroll display section 712 displays transmission report events that occurred before the transmission report events displayed in the fixed display section 711. In the scroll display section 712, a scroll bar 713 is displayed. By operating the scroll bar 713, the user can scroll the display section 632 for each transmission report event and view transmission report events that are not within the screen display range.

[0186] Next, the screen displayed on the operator terminal 6 at the time of transmission report will be described. FIG. 18 is an explanatory diagram showing the mail screen 801 and the transmission report details screen 901 displayed on the operator terminal 6 at the time of transmission report.

[0187] When a notification is issued by the monitoring server 12, the worker terminal 6 receives an email from the monitoring server 12 notifying the worker (store manager or store clerk) of the occurrence of the notified event (the visit of the monitored person). The email screen 801 shown in Fig. 18(A) is displayed by the email application.

[0188] On the email screen 801 shown in Fig. 18(A), text notifying the occurrence of the notified event (the visit of the monitored person) is displayed. Also, on the email screen 801, the name of the store where the notified event occurred, the date and time when the notified event occurred, the name of the face camera 1 that captured the notified event, the name of the category to which the monitored person who is the subject of the notified event belongs, the personal name (common name) of the monitored person who is the subject of the notified event, and remarks (reference information) regarding the monitored person who is the subject of the notified event are displayed.

[0189] Also, a URL is displayed on the email screen 801. When the user operates the URL, the notification details screen 901 shown in Fig. 18(B) is displayed by the browser.

[0190] On the notification details screen 901 shown in Fig. 18(B), a category display section 911, a personal name display section 912, a remarks display section 913, a date and time display section 914, a store name display section 915, a camera name display section 916, a similarity display section 917, and an image display section 918 are provided. The display contents of the category display section 911, the personal name display section 912, the remarks display section 913, the date and time display section 914, the store name display section 915, the camera name display section 916, the similarity display section 917, and the image display section 918 are the same as those of the category display section 411, the personal name display section 412, the remarks display section 413, the date and time display section 414, the store name display section 415, the camera name display section 416, the similarity display section 417, and the image display section 418 on the notification screen 401 (see Fig. 15) displayed on the monitoring terminal 4.

[0191] Note that the example shown in Fig. 18(B) is a case where the worker terminal 6 is a smartphone. Since the display is small, the notification screen 401 is vertically configured, and the user can view the entire notification screen 401 by performing a scroll operation.

[0192] In addition, the alert details screen 901 displayed on the staff terminal 6 may be provided with an alert accumulation summary display section (not shown) similar to the alert accumulation summary display section 431 provided on the alert screen 401 (see Figure 15) displayed on the monitoring terminal 4.

[0193] Second embodiment Next, a second embodiment will be described. Note that points not specifically mentioned here are the same as those in the above embodiment. Fig. 19 is an overall configuration diagram of a monitoring system according to the second embodiment.

[0194] In the first embodiment, a monitoring terminal 4 is installed in a store, and a security guard or a store manager stationed in the store operates the monitoring terminal 4 to view the screen displayed on the monitoring terminal 4. On the other hand, in the present embodiment, a monitoring terminal 21 is installed in a security company's business premises (such as a monitoring center), and a security guard of the security company operates the monitoring terminal 21 to view the screen displayed on the monitoring terminal 21. In addition, a warning light 22 is installed in the security company's business premises. In addition, a security guard of the security company (such as a patrol security guard) carries an employee terminal 23, and operates the employee terminal 23 to view the screen displayed on the employee terminal 23. If unnecessary, the employee terminal 6 and the warning light 5 on the store side may be omitted.

[0195] As described above, the embodiments have been described as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited to these, and can be applied to embodiments in which modifications, substitutions, additions, omissions, etc. are made. In addition, it is also possible to combine the components described in the above embodiments to create new embodiments. [Industrial Applicability]

[0196] The monitoring system and monitoring method according to the present invention have the effect that when a reporting screen is displayed on a terminal device to notify a user of the occurrence of a reporting event, even if a plurality of reporting events occur continuously without the user's confirmation, the user can confirm not only the latest reporting event but also the reporting events that occurred before without overlooking them. It is useful as a monitoring system and monitoring method that execute, by a processor, a process of performing a report related to detection of a monitoring target person as a reporting event based on an image obtained by photographing a visitor to a facility.

Explanation of Signs

[0197] 1: Camera for face verification 2: In-store monitoring camera 3: Control terminal 4: Monitoring terminal (terminal device, notification device) 5: Warning light 6: Staff terminal 11: Recording server 12: Monitoring server (reporting history storage unit) 13: Face authentication server (registered image storage unit) 14: User management server 15: Personal information storage server (visitor image storage unit) 21: Monitoring terminal 22: Warning light 23: Staff terminal 101: Face search screen 201: Target person list screen 301: Target person registration / editing screen 401: Reporting screen 601: Reporting storage details screen 701: Reporting history list screen 154: Face reference information display column 241: Reporting elapsed time display column 242: Reporting count display column 243: Registered face image count display column 251: Sort button (operation unit) 325: Reporting count display column 331: Reporting elapsed time display column 332: Reporting count display column 345: Input box for "category" 346: Input box for "Alarm sensitivity" 347: Expiration date setting section 431: Transmission accumulation summary display section 621: Transmission accumulation summary display section 631: Transmission event list display section

Claims

1. A monitoring system that executes, by a processor, a process of issuing a report regarding detection of a person to be monitored as a reporting event based on an image of a visitor to a facility, a camera that acquires a photographed image within the facility, a registered image storage unit that stores registered face images for each person to be monitored, a terminal device that displays a predetermined reporting screen in response to an instruction from the processor and displays a screen for a user to confirm the reporting status for each person to be monitored, and is provided with, The processor, executes a face matching process of comparing a face image of a visitor with a registered face image of the person to be monitored to determine whether they are the same person, and when issuing a report regarding detection of the person to be monitored based on the result, acquires reporting accumulation summary information regarding a reporting event that is in an unconfirmed and accumulated state in the terminal device, and causes the terminal device to display the reporting screen including the reporting accumulation summary information, and is characterized by a monitoring system.

2. The processor, causes the terminal device to display the reporting screen including, as the reporting accumulation summary information, the total number of reporting events in an accumulated state and the number of persons to be monitored related to the reporting events in an accumulated state, and is characterized by the monitoring system according to claim 1.

3. The processor, causes the terminal device to display the reporting screen including an operation unit that instructs a transition to a detailed screen showing reporting accumulation detailed information regarding a reporting event in an accumulated state, and causes the terminal device to display the detailed screen in response to a user operation on the operation unit in the reporting screen, and is characterized by the monitoring system according to claim 1.

4. The processor, The monitoring system according to claim 3, characterized in that the detailed screen including list information in which information for each reporting event is arranged, limited to reporting events in an accumulated state, is displayed on the terminal device as the reporting accumulation detailed information.

5. The processor The monitoring system according to claim 3, characterized in that the detailed screen including summary information regarding the accumulation status of reporting events for each monitoring target person is displayed on the terminal device as the reporting accumulation detailed information.

6. The processor Displays the detailed screen including an operation unit for instructing the release of the filtering state on the terminal device, The monitoring system according to claim 4, characterized in that in response to a user's operation on the operation unit in the detailed screen, the filtering state limited to reporting events in an accumulated state is released, and a list screen in which information for each reporting event that occurred in the past is arranged is displayed on the terminal device.

7. The processor The monitoring system according to claim 1, characterized in that the reporting screen is generated based on a screen configuration dedicated to a mobile terminal, and the reporting screen is transmitted to the terminal device which is a mobile terminal.

8. A monitoring method in which a processor executes a process of performing a report regarding detection of a monitoring target person as a reporting event based on an image of a visitor to a facility, When performing a face matching process of comparing a face image of a visitor with a registered face image of a monitoring target person to determine whether they are the same person, and performing a report regarding detection of the monitoring target person based on the result, A monitoring method characterized in that the terminal device acquires report accumulation summary information regarding a reporting event in an unconfirmed and accumulated state by the user, and a reporting screen including the report accumulation summary information is displayed on the terminal device.

Images