Surveillance system

Abstract

To provide a monitoring system with which it is possible to prevent a legitimate user from being mistakenly detected as an intruder due to unsuccessful face authentication.SOLUTION: A monitoring system monitors an intrusion into a monitored area, and comprises: a sensor that is installed in the monitored area, for detecting an intrusion; a storage unit that preliminarily stores the registered face images of registrants; an imaging unit that images the face of a user and acquires an input face image; authentication means that verifies the input face image against the registered face images and authenticates the user; estimation means that establishes a first index that indicates the probability of the user being a registrant, and a second index that indicates the acceptability of the imaging environment around the imaging unit; determination means that determines that there is an intrusion into the monitored area, when user authentication is unsuccessful for a prescribed retention time after a detection signal is acquired from the sensor; and execution means that executes a suppress process to suppress determination by the determination means that there is an intrusion, when the first index indicates high probability of being a registrant, or the second index indicates that the imaging environment is not good.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a monitoring system.

Background Art

[0002] Conventionally, there is known a security device that uses a sensor to monitor intrusion into facilities such as a residence and reports to a security center when an intrusion occurs. A user who first enters a facility where such a security device is installed operates the security device after entering the facility to end the monitoring by the security device. The security device holds the report until a predetermined time has elapsed after the intrusion into the facility is detected so that the first person to enter the facility is not reported as an intruder. The security device does not make a report if the security device is operated and the monitoring is ended while the report is being held.

[0003] Patent Document 1 describes a security device that extends the time not to make a report when an image sensor having a face recognition function arranged in a monitoring area is collated with user data. According to the security device of Patent Document 1, false alarms due to misdetection of a user as an intruder are suppressed.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, face recognition accuracy decreases due to various factors such as the orientation of the user's face and the surrounding brightness. Therefore, when using face recognition, there is a possibility that even a legitimate user of the facility may fail in authentication.

[0006] The present invention has been made to solve the above-described problems, and an object thereof is to provide a monitoring system that can prevent a legitimate user from being erroneously detected as an intruder even though face authentication fails.

Means for Solving the Problems

[0007] The monitoring system according to the present invention is a monitoring system that monitors intrusion into a monitoring area, and includes a sensor that is disposed in the monitoring area and detects intrusion, a storage unit that stores in advance a registered face image of a registrant, an imaging unit that captures an image of a user's face to obtain an input face image, an authentication unit that collates the registered face image and the input face image to authenticate the user, an estimation unit that obtains a first index indicating the probability that the user is a registrant or a second index indicating the quality of the imaging environment around the imaging unit, a determination unit that determines that there has been an intrusion into the monitoring area when authentication of the user has not been successful during a predetermined hold time after a detection signal is obtained from the sensor, and an execution unit that executes a suppression process for suppressing the determination by the determination unit that there has been an intrusion when the first index indicates a high probability that the user is a registrant or when the second index indicates that the imaging environment is not good.

[0008] Further, the monitoring system according to the present invention further includes a reception unit that receives a release operation for releasing a warning mode in which a user monitors intrusion into a monitoring area, and it is preferable that the determination unit determines that there has been no intrusion into the monitoring area when authentication of the user is successful and a release operation is received during a predetermined hold time after a detection signal is obtained from the sensor.

[0009] Further, in the monitoring system according to the present invention, it is preferable that the estimation unit obtains, as the first index, an index based on an estimation result as to whether or not the user has an intention to be authenticated, and the execution unit executes a suppression process when the user has an intention to be authenticated.

[0010] In addition, in the monitoring system according to the present invention, the estimation means determines the orientation of the face of the user included in the input face image, and preferably estimates that the user has the intention to receive authentication when the determined face orientation is included in a predetermined range including the direction of the imaging unit.

[0011] In addition, in the monitoring system according to the present invention, the estimation means obtains an index based on the similarity between the input face image and the registered face image as a first index, and the execution means preferably executes a suppression process when the similarity is equal to or greater than a threshold value.

[0012] In addition, the monitoring system according to the present invention further includes a second sensor that detects the brightness of the shooting environment. The estimation means obtains an index based on the detection result of the second sensor as a second index, and the execution means preferably executes a suppression process when the brightness of the shooting environment is less than a predetermined standard.

[0013] In addition, in the monitoring system according to the present invention, as the suppression process, the execution means preferably executes a process of extending the hold time or a process of waiting for a predetermined time to determine that there has been an intrusion into the monitoring area.

[0014] In addition, the monitoring system according to the present invention further includes a communication unit that communicates with the user's communication terminal. The determination means determines that there has been an intrusion into the monitoring area when the user's authentication fails and no predetermined input signal is received from the communication terminal during the hold time after the detection signal is acquired. The execution means preferably transmits, to the communication terminal, a notification for prompting the user to perform an operation for transmitting the input signal as the suppression process.

Effect of the Invention

[0015] The monitoring system according to the present invention can prevent being misdetected as an intruder even though the user is a legitimate user and the face authentication fails.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Embodiments for Carrying Out the Invention

[0017] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. Note that the technical scope of the present invention is not limited to those embodiments, and extends to the invention described in the claims and its equivalents.

[0018] FIG. 1 is a functional block diagram of the monitoring system 1 according to the embodiment. The monitoring system 1 monitors intrusion into the monitoring area. The monitoring area is, for example, a building such as a residence, commercial facility, office, or public facility. The monitoring area may include the outer periphery of the building or the land on which the building is constructed. The monitoring system 1 includes a security device 10, a plurality of monitoring sensors 20, a security center device 30, and a communication terminal 40. The monitoring sensor 20 is communicably connected to the security device 10. The security center device 30 and the communication terminal 40 are communicably connected to the security device 10 via a network 50.

[0019] The security device 10 is arranged inside the monitoring area. For example, the security device 10 is arranged inside the building that is the monitoring area. The security device 10 monitors the monitoring area in response to the operation of the user, and reports to the security center device 30 when an abnormality suspected of being an intrusion into the monitoring area is detected. The security device 10 includes an interface unit 11, a communication unit 12, a storage unit 13, an operation unit 14, an imaging unit 15, a display unit 16, an audio output unit 17, an illuminance sensor 18, and a control unit 19.

[0020] The interface unit 11 is configured to enable the security device 10 to communicate with the monitoring sensor 20, and includes an interface circuit. The interface circuit is, for example, a communication interface circuit such as a wired LAN (Local Area Network) or a wireless LAN. The interface circuit may also be an image transmission interface circuit such as HD-SDI (High Definition Serial Digital Interface). The interface unit 11 supplies the data supplied from the monitoring sensor 20 to the control unit 19.

[0021] The communication unit 12 is configured to enable the security device 10 to communicate with the security center device 30 and the communication terminal 40, and includes a communication interface circuit. The communication interface circuit is, for example, a communication interface circuit such as a wired LAN, a wireless LAN, or LTE (Long Term Evolution). The communication unit 12 supplies the data received from the security center device 30 and the communication terminal 40 to the control unit 19, and transmits the data supplied from the control unit 19 to the security center device 30 or the communication terminal 40.

[0022] The storage unit 13 is configured to store data and programs, and includes, for example, a semiconductor memory. The storage unit 13 stores an operating system program, a driver program, an application program, data, etc. used for the processing by the control unit 19. The program is installed in the storage unit 13 from a computer-readable and non-transitory portable storage medium such as a CD (Compact Disc)-ROM (Read Only Memory) or a DVD (Digital Versatile Disc)-ROM.

[0023] The storage unit 13 stores a registered user information table 131 and a sensor information table 132 as data.

[0024] The operation unit 14 is configured to receive a user operation on the security device 10, and includes, for example, a keypad. The operation unit 14 generates a signal according to the user operation and supplies it to the control unit 19.

[0025] The imaging unit 15 is configured to image the periphery of the security device 10 and includes a camera. The camera includes an imaging optical system for forming an image on a light-receiving surface, a photoelectric conversion element such as a CCD (Charge Coupled Device) sensor that is two-dimensionally arranged on the light-receiving surface and outputs an electrical signal corresponding to the incident light amount, and an image generation circuit that generates an image based on the output of the photoelectric conversion element. The imaging unit 15 supplies the generated image to the control unit 19. Note that the imaging unit 15 may be configured as a means for acquiring a captured image from an externally connected camera.

[0026] The display unit 16 is configured to display an image and includes a display such as a liquid crystal display or an organic EL (Electro Luminescence) display. The display unit 16 generates and displays an image based on a signal supplied from the control unit 19.

[0027] The voice output unit 17 is configured to output voice and includes a speaker. The voice output unit 17 outputs voice based on a signal supplied from the control unit 19.

[0028] The illuminance sensor 18 is an example of the second sensor and is configured to detect the brightness of the periphery of the security device 10. The illuminance sensor 18 has a light-receiving element that outputs a current corresponding to the incident light amount, and a signal generation circuit that generates a signal based on the output of the light-receiving element. The illuminance sensor 18 supplies the generated signal to the control unit 19.

[0029] The control unit 19 is configured to comprehensively control the operation of the security device 10 and includes one or more processors and their peripheral circuits. The control unit 19 includes, for example, a CPU (Central Processing Unit). The control unit 19 may include a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The control unit 19 executes processing based on the program stored in the storage unit 13. Also, the control unit 19 controls the operation of each component of the security device 10 so that the processing is appropriately executed.

[0030] The control unit 19 has acquisition means 191, timing means 192, authentication means 193, reception means 194, determination means 195, setting means 196, notification means 197, estimation means 198, and execution means 199 as functional blocks. These functional blocks are functional modules realized based on the program executed by the control unit 19. These functional blocks may be firmware installed in the security device 10.

[0031] The plurality of monitoring sensors 20 are arranged in the monitoring area and detect an abnormality in the monitoring area that is suspected to be an intrusion into the monitoring area. For example, the monitoring sensor 20 is a magnet sensor that detects the operation of an opening / closing part such as a door or window of a building as an abnormality in the monitoring area. The monitoring sensor 20 may be an infrared sensor that detects a moving object inside the monitoring area as an abnormality in the monitoring area based on a change in the amount of infrared light received. The monitoring sensor 20 may also be an image sensor that detects a moving object inside the monitoring area as an abnormality in the monitoring area based on a difference signal between the captured image and a background image stored in advance. When an abnormality in the monitoring area is detected, the monitoring sensor 20 transmits a detection signal to the security device 10.

[0032] The security center device 30 is an information processing device such as a server or a PC (Personal Computer) arranged in a security center where security guards are stationed. The security center device 30 receives reports from the security device 10. When a report is received, the security center device 30 notifies the content of the report to the security guards stationed in the security center by, for example, displaying the content of the report.

[0033] The communication terminal 40 is an information processing terminal such as a smartphone or a mobile phone held by a user. The communication terminal 40 transmits a predetermined input signal for operating the security device 10 to the security device 10.

[0034] FIG. 2(A) is a diagram showing the data structure of a registrant information table 131 that is stored in the storage unit 13 and manages information about a registrant (referring to a person who has been registered in advance as a person who operates the security device 10 among the users of the facility). The registrant information table 131 stores the registrant ID, the registered face image, the notification destination, etc. in association with each other.

[0035] The registrant ID is information that uniquely identifies a registrant and is, for example, an arbitrary character string. The registered face image is an image of the registrant's face. The notification destination is information that identifies the communication terminal held by the registrant, which is the destination of the notification by the security device 10. The notification destination is, for example, a device token for transmitting a push notification or an email address or a phone number for transmitting a message.

[0036] The data in the registrant information table 131 is set in advance by the administrator of the security device 10.

[0037] FIG. 2(B) is a diagram showing the data structure of a sensor information table 132 that is stored in the storage unit 13 and manages information about the monitoring sensor 20. The sensor information table 132 stores the sensor ID, the hold time setting, etc. in association with each other.

[0038] The sensor ID is information that uniquely identifies the monitoring sensor 20, and may be, for example, an arbitrary character string. The hold time setting is information indicating whether or not a hold time is set for each monitoring sensor 20. The hold time is the time that the security device 10 holds off on reporting to the security center device 30 after the monitoring sensor 20 outputs a detection signal. When a monitoring sensor 20 for which a hold time is not set outputs a detection signal, the security device 10 immediately reports to the security center device 30 that there has been an intrusion into the monitored area. When a monitoring sensor 20 for which a hold time is set outputs a detection signal, the security device 10 holds off on reporting for a preset hold time (for example, one minute) before reporting to the security center device 30 that there has been an intrusion into the monitored area.

[0039] The security device 10 is set to either an alert mode in which intrusion into the monitoring area is monitored, or a deactivation mode in which intrusion into the monitoring area is not monitored. For example, if the monitoring area is the residence of a registrant, the registrant sets the security device 10 to alert mode when going out of the residence, and deactivates the alert mode and sets the security device 10 to deactivation mode when returning home. Since the security device 10 is placed inside the monitoring area, the monitoring sensor 20 detects the registrant's return as an abnormality in the monitoring area during the period from when the registrant returns home to when the security device 10 is set to deactivation mode. At this time, in order to prevent the security device 10 from being notified to the security center device 30 by determining that an intrusion into the monitoring area has occurred, the security device 10 holds back the report for a hold time that is preset as a standard time from when the monitoring sensor 20 detects an abnormality until when the registrant deactivates the alert mode. If the alert mode is deactivated during the hold time from when the registrant is detected by the monitoring sensor 20, no report is made.

[0040] The data in the sensor information table 132 is set in advance by the administrator of the security device 10. In particular, the hold time setting data is set based on the location where each monitoring sensor 20 is placed. For example, if the monitoring area is the residence of a registrant, a hold time is set for the monitoring sensor 20 that first detects the registrant when the registrant returns home, and a hold time is not set for the other monitoring sensors 20. An example of a monitoring sensor 20 for which a hold time is set is a magnet sensor that detects the opening and closing of a front door. Examples of monitoring sensors 20 for which a hold time is not set are a magnet sensor that detects the opening and closing of a window, an infrared sensor that detects a moving object, etc.

[0041] 3 is a flow diagram showing the flow of the detection process executed by the security device 10 of the monitoring system 1. The detection process is executed when the security device 10 is set to the alert mode and any of the monitoring sensors 20 outputs a detection signal. The detection process is realized by the control unit 19 cooperating with each component of the security device 10 based on a program stored in the storage unit 13.

[0042] First, the acquiring means 191 acquires a detection signal output from the monitoring sensor 20 via the interface unit 11 (step S101). The detection signal includes the sensor ID of the monitoring sensor 20.

[0043] Next, the timer 192 judges whether or not a hold time is set for the monitoring sensor 20 that output the detection signal (step S102). The timer 192 refers to the sensor information table 132, and judges that a hold time is set when the hold time setting for the sensor ID included in the detection signal is set to "Yes".

[0044] If a hold time is set for the monitoring sensor 20 that output the detection signal (step S102-Yes), the clocking means 192 starts clocking the hold time (step S103). For example, the clocking means 192 stores the time when the detection signal is acquired in the storage unit 13 as the start time of the hold time.

[0045] Next, the authentication means 193 authenticates the user (step S104). The authentication means 193 controls the imaging unit 15 to image the user's face and acquire an input face image including the user's face. The authentication means 193 authenticates the user by comparing each registered face image stored in the registered user information table 131 with the acquired input face image.

[0046] The authentication means 193 uses a known face recognition technology to extract a face region corresponding to the user's face from the input face image. For example, the authentication means 193 extracts a change region where the difference in luminance values between each pixel of the input face image and each pixel of a background image stored in advance is equal to or greater than a predetermined value. The authentication means 193 extracts a part of the extracted change region as a person region based on feature amounts such as the size and shape of the change region. The authentication means 193 applies an edge pixel extraction process such as a Sobel filter to the extracted person region to extract edge pixels. The authentication means 193 detects an elliptical edge distribution approximated to the contour shape of the head among the extracted edge pixels, and extracts the region surrounded by the detected edge distribution as the face region.

[0047] Note that the method for extracting the face region is not limited to such an example, and the authentication means 193 may extract the face region from the input face image using a discriminator trained to extract the face region from the image. The discriminator is, for example, an Adaboost discriminator.

[0048] The authentication means 193 collates the registered face image with the face area extracted from the input face image. For example, the authentication means 193 collates the registered face image with the face area by pattern matching. In this case, the authentication means 193 calculates the sum of the squares of the differences in luminance values between each pixel of the registered face image and the corresponding pixel of the face area while changing the correspondence between each pixel of the registered face image and each pixel of the face area. The authentication means 193 specifies the smallest one among the sums of squares calculated for each correspondence. The authentication means 193 calculates the reciprocal of the value obtained by dividing the specified sum of squares by the number of pixels in the face area as the similarity. When the maximum value among the similarities calculated for all the registered face images is equal to or greater than the first threshold, the authentication means 193 determines that the person shown in the input face image is the user corresponding to the registered face image with the maximum similarity, and determines that the authentication has succeeded. On the other hand, when the maximum value among the similarities calculated for all the registered face images is less than the first threshold, the authentication means 193 does not determine that the authentication has succeeded.

[0049] Next, the authentication means 193 determines whether the authentication has succeeded (step S105).

[0050] If the authentication has not succeeded (step S105 - No), the timing means 192 determines whether a preset length of hold time has elapsed since the start time stored in step S103 (step S106).

[0051] If the hold time has not elapsed (step S106 - No), the detection process proceeds to step S104, and the authentication means 193 authenticates the user again. That is, the authentication means 193 repeatedly authenticates the user until the hold time elapses or the authentication succeeds.

[0052] If the authentication has succeeded (step S105 - Yes), the reception means 194 receives a release operation by the user to release the warning mode (step S107). The reception means 194 displays information indicating that the authentication has succeeded and the release operation is possible on the display unit 16, and receives the release operation.

[0053] Next, the determination means 195 determines whether a release operation on the operation unit 14 has been received (step S108). The determination means 195 determines that the release operation has been received when a signal corresponding to the release operation is supplied from the operation unit 14.

[0054] If the release operation has not been received (step S108 - No), the timing means 192 determines whether a preset length of hold time has elapsed since the start time stored in step S103 (step S109).

[0055] If the hold time has not elapsed (step S109 - No), the detection process proceeds to step S107, and the reception means 194 receives the release operation again. That is, the reception means 194 continuously receives the release operation until the hold time elapses.

[0056] If the release operation has been received (step S108 - Yes), the determination means 195 determines that there has been no intrusion into the monitoring area (step S110). That is, the determination means 195 determines that there has been no intrusion into the monitoring area when the user authentication is successful and the release operation is received during the hold time after the detection signal from the monitoring sensor 20 is acquired. The determination means 195 determines that there has been an intrusion into the monitoring area when the user authentication is not successful or the release operation is not received although the user authentication is successful during the hold time after the detection signal is acquired.

[0057] Next, the setting means 196 releases the warning mode and sets the security device 10 to the release mode (step S111). The setting means 196 changes the information indicating the current mode stored in the storage unit 13 from the information indicating the warning mode to the information indicating the release mode. Thus, the detection process ends.

[0058] When no hold time is set for the monitoring sensor 20 that has output a detection signal (step S102 - No), when the hold time has elapsed before authentication is successful (step S106 - Yes), or when the hold time has elapsed before a release operation is received (step S109 - Yes), the determination means 195 outputs a warning (step S112). The determination means 195 displays a predetermined warning on the display unit 16 and causes the audio output unit 17 to output a predetermined warning sound.

[0059] Next, the determination means 195 determines that there has been an intrusion into the monitoring area (step S113). That is, the determination means 195 determines that there has been an intrusion into the monitoring area when the user's authentication has not been successful during the hold time after the detection signal from the monitoring sensor 20 has been acquired, or when a release operation has not been received during the hold time after the detection signal has been acquired.

[0060] Next, the notification means 197 notifies the security center device 30 that there has been an intrusion into the monitoring area (step S114). The notification means 197 generates a notification signal including the sensor ID of the monitoring sensor 20 that has output the detection signal. The notification means 197 transmits the notification signal to the security center device 30 via the communication unit 12. Thus, the detection process ends.

[0061] Figure 4 is a flowchart showing the flow of the notification suppression process executed by the security device 10 of the monitoring system 1. The notification suppression process is executed in parallel with the detection process from when the hold time starts in step S103 of the detection process until it is determined that authentication is successful in step S105, or until it is determined that the hold time has elapsed in step S106 or S109. The notification suppression process is realized by the control unit 19 cooperating with each component of the security device 10 based on a program stored in the storage unit 13.

[0062] First, the estimation means 198 obtains a first index indicating the probability that the user is a registered person and a second index indicating the quality of the shooting environment around the imaging unit 15 (step S201). For example, the first index is information indicating that the probability that the user is a registered person is high, or information indicating that the probability that the user is a registered person is low. The second index is information indicating that the area around the imaging unit 15 is bright, or information indicating that the area around the imaging unit 15 is dark.

[0063] The first index is, for example, an index based on the estimation result of whether the user has the intention to receive authentication. In this case, the estimation means 198 estimates whether the user has the intention to receive authentication, and obtains the first index based on the estimation result. For example, when the estimation result indicates that the user has the intention to receive authentication, the estimation means 198 obtains a first index indicating that the probability that the user is a registered person is high. When the estimation result indicates that the user has no intention to receive authentication, the estimation means 198 obtains a first index indicating that the probability that the user is a registered person is low.

[0064] When the user is a registered person, since the user recognizes that he / she must receive authentication by the security device 10 and set the security device 10 to the release mode, the user usually has the intention to receive authentication. When the user is not a registered person (for example, an intruder), since the user does not recognize that he / she needs to receive authentication by the security device 10, the user usually does not have the intention to receive authentication. Therefore, based on whether the user has the intention to receive authentication, a first index indicating the probability that the user is a registered person is obtained.

[0065] For example, the estimation means 198 determines the orientation of the user's face included in the input face image. When the determined face orientation is included in a predetermined range including the direction of the imaging unit 15, the estimation means 198 estimates that the user has the intention to receive authentication.

[0066] In this case, the estimation means 198 extracts facial feature points such as the outer corners of the eyes, the tip of the nose, and the corners of the mouth from the input face image. For example, the estimation means 198 extracts edge pixels from the input face image by applying edge pixel extraction processing to the input face image, and extracts facial feature points based on the distribution of the edge pixels. The estimation means 198 determines the orientation of the user's face based on the type, positional relationship, etc. of the extracted feature points.

[0067] The estimation means 198 determines whether or not the direction of the estimated face orientation is included in a predetermined range including the direction of the imaging unit 15. For example, the estimation means 198 calculates the angular difference between the estimated face orientation and the direction directly facing the imaging unit 15. The estimation means 198 determines that the orientation of the user's face is included in a predetermined range including the direction of the imaging unit 15 when the angular difference is equal to or less than a predetermined value. When the orientation of the user's face is included in a predetermined range including the direction of the imaging unit 15, the estimation means 198 estimates that the user has the intention to undergo authentication.

[0068] A user who has the intention to undergo authentication usually faces the imaging unit 15 in order to undergo face authentication. A user who does not have the intention to undergo authentication usually does not face the imaging unit 15. In particular, when the user is an intruder, it is considered particularly difficult to face the imaging unit 15 in order to avoid being imaged by the imaging unit 15. Therefore, based on the orientation of the user's face, it is possible to estimate whether or not the user has the intention to undergo authentication.

[0069] Note that the method for estimating whether or not the user has the intention to undergo authentication is not limited to the example described above. For example, the estimation means 198 may estimate that the user has the intention to undergo authentication when the user is staying in the vicinity of the imaging unit 15.

[0070] In this case, the estimation means 198 acquires a plurality of input face images with different imaging times. The estimation means 198 extracts a face region from each of the acquired input face images, and specifies the three-dimensional position of the user's face for each input face image. Further, the estimation means 198 estimates the amount and direction of movement of the user's face between the plurality of input face images using a known tracking technique or the like. The estimation means 198 determines whether the user has continuously stayed within a range within a predetermined distance from the imaging unit 15 for a predetermined time or more based on the estimated three-dimensional position, amount of movement, and direction of movement of the user's face. The estimation means 198 estimates that the user has the intention to receive authentication when the user has continuously stayed within a range within a predetermined distance from the imaging unit 15 for a predetermined time or more.

[0071] The first index may be an index based on the similarity between the input face image and the registered face image. In this case, the estimation means 198 calculates the similarity between the input face image and the registered image by the same method as in step S104 of the detection process. The estimation means 198 calculates a first index indicating that the probability that the user is the registered person is high when the similarity is equal to or greater than a second threshold value. The second threshold value is a value smaller than the first threshold value used in step S105 of the detection process. The estimation means 198 calculates a first index indicating that the probability that the user is the registered person is low when the similarity is less than the second threshold value.

[0072] The second index is an index indicating the quality of the shooting environment around the imaging unit 15, that is, whether the shooting environment degrades the authentication accuracy. The second index is, for example, an index based on the brightness of the shooting environment. In this case, the estimation means 198 acquires a detection signal from the illuminance sensor 18 as a detection result by the illuminance sensor 18. When the illuminance indicated by the detection signal is equal to or higher than a predetermined standard, the estimation means 198 obtains a second index indicating that the shooting environment is good, and when the illuminance is lower than the predetermined standard, the estimation means 198 obtains a second index indicating that the shooting environment is not good. The second index may be an index based on whether the shooting environment is front-lit or back-lit. In this case, the estimation means 198 acquires a captured image from the imaging unit 15. The estimation means 198 determines whether the shooting environment is front-lit or back-lit based on the distribution of the luminance values of the acquired captured image. For example, in the captured image, the estimation means 198 determines that the shooting environment is back-lit when the ratio of the pixels having a luminance value equal to or higher than a predetermined value is equal to or higher than a predetermined standard, and determines that the shooting environment is front-lit when it is lower than the predetermined standard. When the shooting environment is front-lit, the estimation means 198 obtains a second index indicating that the shooting environment is good, and when the shooting environment is back-lit, the estimation means 198 obtains a second index indicating that the shooting environment is not good.

[0073] When the shooting environment around the imaging unit 15 degrades the authentication accuracy (for example, in a dark place or back-lit), there is a high possibility that the authentication will fail even though the user is a registered person. That is, at the stage where the authentication has not been successful, the probability that the user is a registered person is higher when the authentication accuracy is low than when the authentication accuracy is high. Therefore, the estimation means 198 can use a second index indicating the quality of the shooting environment around the imaging unit 15 instead of the first index indicating the probability that the user is a registered person.

[0074] Next, the execution means 199 determines whether or not a suppression condition for executing the suppression process is satisfied (step S202). The suppression condition is that the first index indicates that the probability that the user is a registered person is high, or the second index indicates that the shooting environment is not good. The suppression process is a process for suppressing the determination that there has been an intrusion into the monitoring area by the determination means 195.

[0075] That is, when the first index is an index based on the estimation result of whether the user has the intention to receive authentication, the execution means 199 executes a suppression process when the user has the intention to receive authentication. When the first index is an index based on the similarity between the input face image and the registered face image, the execution means 199 executes a suppression process when the similarity is equal to or greater than a second threshold value. When the second index is an index based on the detection result by the illuminance sensor 20, that is, the brightness of the shooting environment, the execution means 199 executes a suppression process when the brightness of the shooting environment is less than a predetermined standard. When the second index is an index based on whether the shooting environment is front-lit or back-lit, the execution means 199 executes a suppression process when the ratio of the pixels whose luminance values included in the captured image are equal to or greater than a predetermined value is equal to or greater than a predetermined standard.

[0076] When the suppression condition is satisfied (step S202 - Yes), the execution means 199 executes a suppression process (step S203). The execution means 199 may execute a suppression process when both the suppression condition for the first index and the suppression condition for the second index are satisfied, or may execute a suppression process when at least one of them is satisfied. Thus, the notification suppression process ends.

[0077] The suppression process is, for example, a process of extending the hold time. For example, the execution means 199 extends the hold time by changing the length of the preset hold time to be longer. When the length of the hold time is changed, in steps S106 and S109 of the detection process, the timing means 192 determines whether the changed hold time has elapsed from the start time of the hold time. The execution means 199 may extend the hold time by updating the start time of the hold time. In this case, the start time of the hold time is updated, for example, at the time when it is determined in step S202 that the suppression condition is satisfied.

[0078] When the suppression condition is not satisfied (step S202 - No), the execution means 199 determines whether an end condition for ending the notification suppression process is satisfied (step S204). The end condition is that the user authentication was successful in step S105 of the detection process, or that the hold time has elapsed in step S106 or S109. When the end condition is satisfied (step S204 - Yes), the notification suppression process ends.

[0079] When the end condition is not satisfied (step S204 - No), the notification suppression process returns to step S201, and the estimation means 198 obtains the first index and the second index again. That is, the estimation means 198 repeatedly obtains the first index and the second index until the suppression condition or the end condition is satisfied.

[0080] As described above, the monitoring system 1 includes a monitoring sensor 20, a storage unit 13, an imaging unit 15, an authentication means 193, an estimation means 198, a determination means 195, and an execution means 199. The monitoring sensor 20 is arranged in the monitoring area and detects an intrusion into the monitoring area. The storage unit 13 stores in advance the registered face images of the registered persons. The imaging unit 15 captures the face of the user and generates an input face image. The authentication means 193 authenticates the user by comparing the registered face image and the input face image. The estimation means 198 calculates a first index indicating the probability that the user is a registered person or a second index indicating the quality of the shooting environment around the imaging unit 15. The determination means 195 determines that there has been an intrusion into the monitoring area when the user authentication has not been successful during a predetermined hold time after the detection signal from the monitoring sensor 20 has been acquired. The execution means 199 executes a suppression process for suppressing the determination by the determination means 195 that there has been an intrusion into the monitoring area when the first index indicates a high probability that the user is a registered person or when the second index indicates that the shooting environment is not good. Thereby, the monitoring system 1 can prevent being erroneously detected as an intruder even though the user is a legitimate user and the face authentication fails.

[0081] That is, in face authentication, the face of the user, which is the object of authentication, is imaged in a state where it is away from the imaging unit. Therefore, in face authentication, compared with fingerprint authentication, vein authentication, etc. where the object is imaged in a state of being in contact with the imaging unit, while the burden on the user to bring the object into contact with the imaging unit is reduced, the authentication accuracy of legitimate users may decrease depending on the environment of the imaging space. The monitoring system 1 according to the embodiment suppresses the misjudgment that there has been an intrusion into the monitoring area based on the probability that the user is a legitimate user or the environment of the imaging space. Therefore, even when face authentication is used, the monitoring system 1 prevents legitimate users from being misdetected as intruders.

[0082] Preferably, the monitoring system 1 further has reception means for receiving a release operation for releasing a warning mode for monitoring an intrusion into the monitoring area by the user. The determination means 195 determines that there has been no intrusion into the monitoring area when the user's authentication is successful and a release operation is received within a predetermined hold time after the detection signal from the monitoring sensor 20 is acquired. This prevents an intruder who does not recognize the setting operation of the release mode from being determined to be a registered user. Therefore, the monitoring system 1 can more accurately prevent the misdetection of legitimate users.

[0083] Preferably, the estimation means 198 obtains, as a first index, an index based on the estimation result of whether the user has an intention to undergo authentication. The execution means 199 executes suppression processing when the user has an intention to undergo authentication. Thereby, the monitoring system 1 can prevent the misdetection of legitimate users with high accuracy based on the user's intention.

[0084] Preferably, the estimation means 198 determines the orientation of the user's face included in the input face image, and estimates that the user has an intention to undergo authentication when the determined face orientation is included in a predetermined range including the direction of the imaging unit 15. Thereby, the monitoring system 1 can estimate the user's intention based on the user's behavior and prevent the misdetection of legitimate users with high accuracy.

[0085] Preferably, the estimation means 198 obtains, as a first index, an index based on the degree of similarity between the input face image and the registered face image. The execution means 199 executes a suppression process when the degree of similarity is equal to or greater than a second threshold value. Thereby, the monitoring system 1 can prevent misdetection of a normal user with high accuracy based on an objective index of the user's face.

[0086] Preferably, the monitoring system 1 further includes an illuminance sensor 18 that detects the brightness of the shooting environment. The estimation means 198 obtains, as a second index, an index based on the detection result by the illuminance sensor 18. The execution means 199 executes a suppression process when the brightness of the shooting environment is less than a predetermined standard. Thereby, the monitoring system 1 can prevent misdetection of a normal user with high accuracy with a simple configuration.

[0087] Preferably, the execution means 199 executes, as a suppression process, a process of extending the hold time. Thereby, since time for a user to perform a release mode setting operation is secured, the monitoring system 1 can prevent a normal user from being misdetected as an intruder.

[0088] The following various modifications may be applied to the monitoring system 1.

[0089] In the above-described embodiment, the suppression process is assumed to be a process of extending the hold time, but it is not limited to such an example. The suppression process may be a process of waiting for a predetermined time for reporting.

[0090] In this case, in step S203, the execution means 199 stores, in the storage unit 13, standby information indicating that it waits for a predetermined time for reporting. When the standby information is stored, in step S112 of the detection process, the reporting means 197 waits for a predetermined time after it is determined that there is an intrusion into the monitoring area. If a release mode setting operation is received while waiting, the reporting means 197 ends the detection process without executing step S114.

[0091] That is, when the suppression process is a process that extends the retention time, steps S112, S113, and S114 are not executed until the extended retention time has elapsed. On the other hand, when the suppression process is a process that waits for a notification for a predetermined time, step S112 is executed before the predetermined time has elapsed, but steps S113 and S114 are not executed until the predetermined time has elapsed. Therefore, when the suppression process is a process that waits for a notification for a predetermined time, a warning is output before the predetermined time has elapsed. This suppresses a non-registered user from attempting a release operation.

[0092] In the above-described embodiment, the reception means 194 is configured to receive a release mode setting operation on the operation unit 14, but is not limited to such an example. The reception means 194 may receive a release mode setting signal from the communication terminal 40. In this case, the suppression process may be a process of transmitting a notification for prompting the user to perform an operation on the communication terminal 40 for transmitting the release mode setting signal.

[0093] In this case, in step S203, the execution means 199 generates a message for prompting an operation for transmitting the release mode setting signal and a notification including transmission information for causing the communication terminal 40 to transmit the release mode setting signal. The transmission information is, for example, information for executing an application program for transmitting the release mode setting signal, which is installed in the communication terminal 40 in advance. The information for executing the application program for transmitting the release mode setting signal includes, for example, identification information of the application program. The transmission information may be a URL (Uniform Resource Locator) for accessing a release server (not shown).

[0094] The execution means 199 acquires the notification destinations of the registrant from the registrant information table 131. At this time, the execution means 199 may acquire all the notification destinations stored in the registrant information table 131, or may acquire some of the notification destinations selected by a predetermined method. When some of the notification destinations are acquired, the execution means 199 acquires the notification destination associated with the registered face image having the highest similarity to the input face image at the current time. The execution means 199 transmits the generated notification to the communication terminal 40 corresponding to the acquired notification destination via the communication unit 12.

[0095] FIG. 5 is a diagram showing a cancellation screen G1 displayed on the communication terminal 40 that has received the notification. The cancellation screen G1 includes a message display area A1 in which a message prompting an operation for transmitting a setting signal for the cancellation mode is displayed. The message display area A1 is displayed superimposed on other screens displayed on the communication terminal 40 as a so-called pop-up notification.

[0096] When an operation for selecting the message display area A1 by the user is received, the communication terminal 40 transmits a setting signal for the cancellation mode based on the transmission information included in the notification. When the information for transmitting the setting signal for the cancellation mode is information for executing an application program, the communication terminal 40 executes an application program for transmitting the setting signal for the cancellation mode based on the information and transmits the setting signal for the cancellation mode.

[0097] When the setting signal for the cancellation mode is received before the elapse of the hold time, steps S110 and S111 are executed. That is, when the setting signal for the cancellation mode is received during the hold time, the determination means 195 determines that there has been no intrusion into the monitoring area regardless of whether the user authentication has been successful. Further, the setting means 196 sets the security device 10 to the cancellation mode. Thereby, the detection process ends.

[0098] The communication terminal 40 is usually always carried by the user. Also, the communication terminal 40 may be equipped with an authentication function such as fingerprint authentication or face authentication. Therefore, when the communication terminal 40 is operated and a setting signal for the release mode is transmitted from the communication terminal 40, it is highly likely that the registered user himself / herself is trying to set the security device 10 to the release mode. In such a case, since the possibility of misdetecting an intruder as a legitimate user is small even if the authentication by the authentication means 193 is omitted, the determination means 195 determines that there has been no intrusion into the monitoring area regardless of whether the user's authentication has been successful.

[0099] As described above, the monitoring system 1 further has a communication unit 12 that communicates with the user's communication terminal. The determination means 195 determines that there has been an intrusion into the monitoring area when the user's authentication has not been successful and a predetermined input signal has not been received from the communication terminal during the hold time after the detection signal has been acquired. The execution means 199 transmits, as a suppression process, a notification for prompting the user to perform an operation for transmitting an input signal to the communication terminal 40. Thereby, the monitoring system 1 can prevent a legitimate user from being misdetected as an intruder even when the authentication accuracy is reduced due to the environment around the imaging unit 15.

[0100] In the above description, the determination means 195 determines that there has been no intrusion into the monitoring area when the user's authentication has been successful and a setting operation for the release mode has been received during the hold time. However, it is not limited to such an example. The determination means 195 may determine that there has been no intrusion into the monitoring area when the user's authentication has been successful at the time during the hold time. Thereby, it becomes unnecessary for the user to perform a setting operation for the release mode. Therefore, the monitoring system 1 can reduce the operation burden on the user.

[0101] In the above description, it is assumed that the estimation means 198 calculates the second index based on the detection result of the illuminance sensor 18, but it is not limited to such an example. For example, the estimation means 198 may calculate the second index based on the time zone. In this case, when it is a time zone at night when sunlight cannot be obtained, the estimation means 198 calculates a second index indicating that the periphery of the imaging unit 15 is dark, and when it is a time zone during the day, the estimation means 198 calculates a second index indicating that the periphery of the imaging unit 15 is bright.

[0102] In the above description, it is assumed that the first index is information indicating that the probability that the user is a registered person is high or information indicating that the probability that the user is a registered person is low, but it is not limited to such an example. The first index may be a numerical index indicating the probability that the user is a registered person. In this case, the estimation means 198 calculates the first index such that the probability that the user is a registered person increases as the orientation of the user's face approaches the direction of the imaging unit 15. The estimation means 198 may calculate the first index such that the probability that the user is a registered person increases as the similarity between the input face image and the registered face image increases. Further, the first index may be a multi-stage index that takes three or more values based on the probability that the user is a registered person.

[0103] Similarly, the second index may be a numerical index indicating the degree to which the environment around the imaging unit 15 degrades the authentication accuracy, or may be a multi-stage index.

[0104] When the first index or the second index is a numerical index or a multi-stage index, the execution means 199 may vary the content of the suppression process according to the first index or the second index. For example, the execution means 199 may make the hold time longer as the value of the first index is larger. Thereby, the monitoring system 1 can prevent a legitimate user from being misdetected as an intruder and can also prevent an intruder from being misdetected as a legitimate user.

[0105] In the above description, the first index and the second index are cited as the indicators required by the estimation means 198. However, the estimation means 198 may obtain only one of the first index and the second index. That is, the estimation means 198 obtains the first index or the second index. The configuration may be such that the estimation means 198 obtains only the first index, or the configuration may be such that the estimation means 198 obtains only the second index.

[0106] In obtaining the first index and the second index, a plurality of judgment indicators may be combined. For example, the first index may be obtained from the estimation result of whether there is an intention to receive authentication and the similarity. In this case, if there is an intention to receive authentication and the similarity is equal to or greater than the threshold value, it may be considered that the probability of being a registrant is high. Also, the second index may be obtained from the detection result of the illuminance and whether it is backlit. In this case, if the illuminance is below the standard and it is backlit, it may be considered that the shooting environment is not good.

[0107] It should be understood by those skilled in the art that various changes, substitutions, and modifications can be added without departing from the spirit and scope of the present invention. For example, the above-described embodiments and modifications may be implemented in appropriate combinations within the scope of the present invention.

Explanation of Reference Numerals

[0108] 1 Monitoring system 12 Communication unit 13 Storage unit 15 Imaging unit 18 Illuminance sensor 193 Authentication means 194 Reception means 195 Judgment means 198 Estimation means 199 Execution means

Claims

1. A monitoring system for monitoring intrusion into a monitoring area, comprising: a sensor disposed in the monitoring area for detecting the intrusion; a storage unit for storing in advance a registered face image of a registrant; an imaging unit for imaging a user's face to obtain an input face image; authentication means for authenticating a user by comparing the registered face image and the input face image; estimation means for obtaining a first index indicating the probability that the user is the registrant or a second index indicating the quality of the imaging environment around the imaging unit; determination means for determining that there has been an intrusion into the monitoring area when authentication of the user has not been successful during a predetermined hold time after a detection signal from the sensor has been acquired; execution means for executing a suppression process for suppressing the determination by the determination means that there has been an intrusion when the first index indicates a high probability that the user is the registrant or when the second index indicates that the imaging environment is not good; A monitoring system, characterized by comprising the above.

2. further comprising reception means for receiving a cancellation operation for canceling a warning mode for monitoring intrusion into the monitoring area by a user; the determination means determines that there has been no intrusion into the monitoring area when authentication of the user has been successful during a predetermined hold time after a detection signal from the sensor has been acquired and the cancellation operation has been received; The monitoring system according to claim 1.

3. the estimation means obtains, as the first index, an index based on an estimation result as to whether the user has an intention to be authenticated; the execution means executes the suppression process when the user has an intention to be authenticated; The monitoring system according to claim 1 or 2.

4. the estimation means determines the orientation of the user's face included in the input face image, and estimates that the user has an intention to be authenticated when the determined face orientation is included in a predetermined range including the direction of the imaging unit; The monitoring system according to claim 3.

5. the estimation means obtains, as the first index, an index based on the degree of similarity between the input face image and the registered face image; the execution means executes the suppression process when the degree of similarity is equal to or greater than a threshold value; The monitoring system according to any one of claims 1-4.

6. further comprising a second sensor for detecting the brightness of the imaging environment; the estimation means obtains, as the second index, an index based on the detection result by the second sensor; The execution means executes the suppression process when the brightness of the shooting environment is less than a predetermined standard. The monitoring system according to any one of claims 1 to 5. **Claim 7** The execution means executes, as the suppression process, a process of extending the holding time or a process of waiting for a predetermined time to determine that there has been an intrusion into the monitoring area. The monitoring system according to any one of claims 1 to 6. **Claim 8** Further comprising a communication unit that communicates with the user's communication terminal. The determination means determines that there has been an intrusion into the monitoring area when the user authentication has not been successful and no predetermined input signal has been received from the communication terminal during the holding time after the detection signal has been acquired. The execution means transmits, as the suppression process, a notification for prompting the user to perform an operation for transmitting the input signal to the communication terminal. The monitoring system according to any one of claims 1 to 7.

Images